Cross-system fault diagnosis method and system combined with multi-dimensional anomaly detection

Abstract

The invention provides a cross-system fault diagnosis method and system combined with multi-dimensional anomaly detection, and the method comprises the steps: firstly obtaining an original data set which is generated by cross-system call and comprises fragmented call data, peripheral business transaction data and log data, and carrying out the breakpoint supplement processing to generate a complete data set; then, transverse anomaly diagnosis is carried out on the complete data set, a decision tree algorithm is used for positioning a fault propagation path and an abnormal object, then longitudinal anomaly detection is carried out on the abnormal object, operation logs, error stacks and infrastructure indexes are analyzed based on a multi-dimensional anomaly detection algorithm, and an anomaly reason analysis result is generated; and finally, determining a fault source, a fault object and an influence range according to an analysis result, and outputting a cross-system fault diagnosis report, thereby comprehensively utilizing multi-source data, realizing accurate positioning and deep analysis of cross-system faults, improving fault diagnosis efficiency and accuracy, and ensuring stable operation of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital software systems, in particular to a cross-system fault diagnosis method and system combined with multi-dimensional anomaly detection. BACKGROUND

[0002] In modern complex IT architecture, cross-system calls occur frequently, and different systems cooperate with each other to complete various business functions. However, when cross-system calls fail, fault diagnosis becomes extremely challenging.

[0003] At present, the existing fault diagnosis methods often have many deficiencies. On the one hand, the collected data is usually fragmented, for example, the call data collected by link probes may have breakpoints due to network fluctuations, system load, etc., resulting in the inability to fully present the whole process of cross-system calls. On the other hand, the existing diagnosis methods mostly only focus on a single data source, such as only analyzing log data or only relying on business transaction data, lacking comprehensive utilization of multi-source data. Moreover, when fault locating, it is often only possible to determine the approximate range of the fault, making it difficult to accurately locate the fault propagation path and abnormal object, and even more difficult to analyze the specific reasons for the abnormal object to generate faults, thereby leading to low efficiency of fault diagnosis, requiring a large amount of manpower and time for troubleshooting, and also making it difficult to accurately determine the fault root cause and impact range, posing a serious threat to the stable operation of the system and business continuity. SUMMARY

[0004] In view of the above-mentioned problems, in combination with the first aspect of the present application, the embodiments of the present application provide a cross-system fault diagnosis method combined with multi-dimensional anomaly detection, which comprises: Obtaining a set of original data generated in the cross-system call process, the set of original data comprising fragmented call data collected by link probes, peripheral business transaction data and log data; Performing breakpoint supplement processing on the set of original data, identifying the breakpoint position in the fragmented call data based on application type, and generating a complete data set by fusing the peripheral business transaction data, log data and system correlation information; Performing horizontal anomaly diagnosis processing on the complete data set, locating the fault propagation path and abnormal object through a decision tree algorithm; Performing vertical anomaly detection processing on the abnormal object, analyzing the running log, error stack and infrastructure indicators of the abnormal object based on a multi-dimensional anomaly detection algorithm to generate an abnormal reason analysis result; Determining the fault root cause, fault object and impact range according to the abnormal reason analysis result, and outputting a cross-system fault diagnosis report.

[0005] In still another aspect, the embodiments of the present application also provide a cross-system fault diagnosis system combined with multi-dimensional anomaly detection, comprising a processor, a machine-readable storage medium, the machine-readable storage medium being connected with the processor, the machine-readable storage medium being used for storing programs, instructions or codes, and the processor being used for executing the programs, instructions or codes in the machine-readable storage medium to realize the method described above.

[0006] Based on the above aspects, the embodiments of the present application can quickly and accurately determine the propagation direction of the fault in the cross-system call and the key abnormal point by acquiring the original data set containing the fragmented call data collected by the link probe, the peripheral business transaction data and the log data in the cross-system call process, and performing breakpoint supplement processing thereon, fusing the peripheral business transaction data, the log data and the system correlation information to generate a complete data set, then performing horizontal anomaly diagnosis processing on the complete data set, and locating the fault propagation path and the abnormal object by using the decision tree algorithm, performing longitudinal anomaly detection processing on the abnormal object, analyzing the running log, error stack and infrastructure index of the abnormal object based on the multi-dimensional anomaly detection algorithm to generate a detailed abnormal reason analysis result, and deeply mining the root cause of the fault, and finally determining the fault root cause, the fault object and the impact range according to the abnormal reason analysis result and outputting a cross-system fault diagnosis report, thereby realizing the full-process automation and precision from data integration, fault location to cause analysis, greatly improving the efficiency and accuracy of cross-system fault diagnosis, reducing the fault troubleshooting cost, and ensuring the stable operation of the system and the continuity of the business. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 is the execution flow diagram of the cross-system fault diagnosis method combined with multi-dimensional anomaly detection provided by an embodiment of the present application.

[0008] Figure 2 is the schematic diagram of exemplary hardware and software components of the cross-system fault diagnosis system combined with multi-dimensional anomaly detection provided by an embodiment of the present application. DETAILED DESCRIPTION

[0009] The present application will be described in detail below with reference to the accompanying drawings, Figure 1 is the flow diagram of the cross-system fault diagnosis method combined with multi-dimensional anomaly detection provided by an embodiment of the present application, and the cross-system fault diagnosis method combined with multi-dimensional anomaly detection will be described in detail below.

[0010] Step S110: acquiring an original data set generated in the cross-system call process, the original data set containing fragmented call data collected by a link probe, peripheral business transaction data and log data.

[0011] In this embodiment, in a cross-system service environment, multiple systems cooperate with each other to complete various service functions. For the fragmented call data collected by the link probe, link probe components can be deployed on the key interfaces and communication paths of each system. The link probe components can listen to the call information between systems in real time, including the initiation time of the call, the target system of the call, the parameters of the call, the return result of the call, etc. Due to the influence of network fluctuations, system load and other factors, the collected data is often incomplete and presents a fragmented characteristic. For example, in a cross-system business process, due to network delay, the link probe only records the initiation part of the call, and the subsequent processing and return result information is missing.

[0012] In order to ensure that these fragmented call data can be accurately collected, the link probe component will use high-speed data capture technology. For example, the communication data between systems can be sampled at very short time intervals. The collected raw data packets are preliminarily analyzed and filtered to extract key information related to system calls. Then, these data are sent to the data collection center through a data transmission channel. During data transmission, encryption technology can be used to encrypt the data to prevent data from being stolen or tampered with during transmission.

[0013] Peripheral business transaction data reflects the specific circumstances of business transactions and is usually stored in a business database. In order to obtain peripheral business transaction data, a database interface can be used to connect with the business database. According to the preset query rules, the business transaction data related to cross-system calls is extracted from the business database regularly. These business transaction data include basic information of the transaction, such as the initiator, the receiver, the transaction time, the transaction amount, etc., and detailed content of the transaction, such as the product or service information of the transaction, the status of the transaction, etc. When extracting data, data screening and cleaning can be performed to remove invalid or duplicate data, ensuring the accuracy and consistency of the data.

[0014] Log data records detailed information of each system during operation, including system operation records, error information, state changes, etc. Log data is usually stored in the form of files in the log file system of each system. In order to obtain these log data, log collection tools can be used. These tools will regularly scan the log files of each system, collect newly generated log data, and send it to the data collection center. When collecting log data, log data can be classified and labeled for subsequent analysis and processing.

[0015] At the data collection center, the fragmented call data, peripheral business transaction data and log data collected from the link probe, business database and log file system can be integrated to form a raw data set. During the integration process, the data can be associated and matched according to the time stamp and related identification of the data to ensure the consistency and integrity of the data. At the same time, the data can be stored and managed, and distributed file system and database technology can be used to ensure efficient storage and query of the data.

[0016] Step S120: breakpoint supplement processing is performed on the raw data set, breakpoint positions in the fragmented call data are identified based on application type, and complete data set is generated by fusing the peripheral business transaction data, log data and system correlation information.

[0017] Since the fragmented call data is incomplete, breakpoint supplement processing is needed to construct a complete data set, which involves multiple steps, including application type matching, breakpoint position identification and data fusion.

[0018] Step S121: application type matching processing is performed on the fragmented call data to identify the call protocol characteristics and data interaction mode corresponding to different application types.

[0019] Different applications follow the set call protocol and data interaction mode when making cross-system calls. In order to accurately identify the application type to which the fragmented call data belongs, in-depth analysis and processing of the data are needed.

[0020] First, feature extraction can be performed on the fragmented call data. Natural language processing and machine learning techniques are used to parse and process the text information in the fragmented call data, and key feature information is extracted. For example, for the request and response messages in the fragmented call data, the header information, body content, request method, response status code and other features can be extracted. For binary information in the fragmented call data, data mining and pattern recognition techniques can be used to extract the structural features of the fragmented call data and the distribution features of the fragmented call data.

[0021] Then, an application type feature library can be established, which contains the call protocol characteristics and data interaction modes of various application types. These call protocol characteristics and data interaction modes are obtained by analyzing and summarizing a large amount of call data of known application types. When establishing the application type feature library, classification algorithms can be used to classify and organize the features of different application types to form a hierarchical feature structure.

[0022] Next, the extracted feature information of the fragmented call data is matched with the features in the application type feature library. A similarity calculation algorithm is used to calculate the similarity scores of the data features and each feature in the application type feature library. For example, a cosine similarity algorithm is used to calculate the similarity of text features, and a Euclidean distance algorithm is used to calculate the similarity of numerical features. According to the similarity scores, the application type with the highest score is selected as the application type of the fragmented call data.

[0023] During the matching process, a multi-stage matching strategy can be used. First, coarse matching is performed to quickly filter out the possible matching application type range. Then, fine matching is performed within the application type range to further improve the accuracy of the matching. At the same time, a machine learning model can be used to verify and optimize the matching results. The matching results are input into the machine learning model, which learns the feature distribution and matching rules of different application types, thereby improving the accuracy and stability of the matching.

[0024] Step S122: According to the call protocol features and data interaction modes, mark the missing data nodes in the fragmented call data as breakpoint positions.

[0025] After determining the application type to which the fragmented call data belongs, the breakpoint positions in the data can be identified according to the call protocol features and data interaction modes of the application type.

[0026] Each application type has its own set of rules and processes for call protocols and data interaction modes. For example, in an application call based on the HTTP protocol, a complete call usually includes three stages: request initiation, server processing, and response return, each with its own set of data formats and interaction rules. By analyzing the fragmented call data in detail, it can be checked whether the fragmented call data conforms to the call protocol and data interaction mode of the application type.

[0027] If the data of a certain stage is missing in the fragmented call data, or the format and interaction rules of the fragmented call data do not conform to the normal process, the position can be marked as a breakpoint position. When marking the breakpoint position, a data marking algorithm can be used to record relevant information such as the timestamp of the breakpoint, the data segment where the breakpoint is located, the type of the breakpoint, etc. At the same time, the breakpoint position can be visualized for subsequent analysis and processing.

[0028] To improve the accuracy of breakpoint position identification, a multi-dimensional analysis method can be used. In addition to analyzing the format and interaction rules of the data, the context information and time series information of the data are also combined for comprehensive analysis. For example, the correlation between the data before and after is analyzed to determine whether the missing data will affect the integrity of the entire call flow. At the same time, machine learning algorithms can be used to predict and verify the breakpoint position. The breakpoint position information in the historical data is used as training data to train the machine learning model, so that the machine learning model learns the distribution rules and characteristics of the breakpoint position, thereby improving the accuracy and reliability of the breakpoint position identification.

[0029] Step S123: Extract transaction identification information and interaction parameter set aligned with the timestamp of the breakpoint position from the peripheral business transaction data.

[0030] After marking the breakpoint position, information related to the breakpoint position needs to be extracted from the peripheral business transaction data to supplement the missing data.

[0031] First, according to the timestamp of the breakpoint position, find the transaction record aligned with it in the peripheral business transaction data. Use time series matching algorithm to compare the timestamp of the breakpoint position with the transaction timestamp in the peripheral business transaction data, and find the transaction record with similar time.

[0032] Then, extract transaction identification information and interaction parameter set from these transaction records. Transaction identification information is information that uniquely identifies a transaction, such as transaction number, order number, etc. Interaction parameter set contains various parameter information in the transaction process, such as transaction amount, quantity, commodity or service information, etc.

[0033] When extracting information, data extraction algorithm can be used to extract the required information from transaction records according to the preset rules. At the same time, the extracted information can be cleaned and converted to ensure that the format and quality of the information meet the requirements of subsequent processing. For example, convert transaction amounts of different formats to the same currency unit, and decode and standardize commodity or service information with different encodings.

[0034] Step S124: Extract request initiation record, response reception record and intermediate processing log entry associated with the breakpoint position from the log data.

[0035] Log data records detailed information of the system during operation, from which key information related to the breakpoint position can be extracted.

[0036] Firstly, the log data is preprocessed, including log data parsing, cleaning and classification. The log data is classified according to different log types, such as system operation log, error log, access log, etc. Then, according to the related information of the breakpoint position, the request initiation record, response receiving record and intermediate processing log entry associated with it are found in the log data.

[0037] The log retrieval algorithm is used to retrieve according to the timestamp of the log, the keyword of the log, the source of the log, etc. For example, by searching for keywords such as system name and interface name related to the breakpoint position in the log, the possible relevant log entries are filtered out. Then, the filtered log entries are analyzed in detail, and the request initiation record, response receiving record and intermediate processing log entry are extracted.

[0038] During the extraction process, natural language processing techniques can be used to parse and understand the log entries. The text information of the log entries is converted into structured data, and key information such as request parameters, response status codes, intermediate processing steps and results is extracted. At the same time, the extracted information can be verified and supplemented to ensure the accuracy and completeness of the information.

[0039] Step S125: The transaction identification information, interaction parameter set, request initiation record, response receiving record and intermediate processing log entry are taken as breakpoint supplement data.

[0040] The relevant information extracted from the peripheral business transaction data and log data is integrated to form breakpoint supplement data.

[0041] During the integration process, data correlation and matching can be performed according to the relevance and consistency of the data. For example, the transaction identification information is associated with the transaction identification in the request initiation record and response receiving record to ensure the correct correspondence of the data. At the same time, the data can be cleaned and converted to remove duplicate and invalid data, and unify the format and standard of the data.

[0042] Then, the integrated data is taken as breakpoint supplement data to supplement the missing part in the fragmented call data. When supplementing the data, the breakpoint supplement data can be accurately inserted into the corresponding position in the fragmented call data according to the information of the breakpoint position.

[0043] Step S126: The pre-stored system association information is called, which contains service dependency relationship of cross-system call, interface call rule and infrastructure deployment location information.

[0044] In this embodiment, the system association information reflects the mutual relationship and calling rules between each system. The pre-stored system association information is collected and sorted during the system deployment and configuration process. These system association information includes service dependency relationship of cross-system calling, that is, which services of other systems a service of a system depends on; interface calling rules, that is, the way, parameters and return values of interface calling between systems; infrastructure deployment location information, that is, the physical location and network topology structure where each system is deployed.

[0045] When the system association information needs to be called, it can be queried and obtained through the system association information management module. This module stores and manages the system association information to ensure the accuracy and timeliness of the information. At the same time, the system association information can be updated and maintained. When the service dependency relationship, interface calling rule or infrastructure deployment location of the system changes, the system association information is updated in time.

[0046] Step S127: Fuse the breakpoint supplement data with the system association information, fill in the missing data nodes in the fragmented calling data, and generate a complete data set.

[0047] Fuse the breakpoint supplement data with the system association information to fill in the missing data nodes in the fragmented calling data.

[0048] First, according to the service dependency relationship and interface calling rule in the system association information, the breakpoint supplement data is analyzed and processed. It is judged whether the breakpoint supplement data conforms to the calling rule and service dependency relationship between systems. If it does not conform, the breakpoint supplement data can be adjusted and corrected to ensure the rationality and consistency of the data.

[0049] Then, the processed breakpoint supplement data is merged with the fragmented calling data. According to the information of the breakpoint position, the breakpoint supplement data is accurately inserted into the corresponding position of the fragmented calling data to fill in the missing data nodes. In the merging process, a data fusion algorithm can be used to integrate and optimize the data to ensure the integrity and accuracy of the data.

[0050] Finally, the fused data is associated and integrated with the system association information to generate a complete data set. The complete data set contains complete information of cross-system calling, including the calling process, detailed information of business transactions, running logs of systems and association information between systems.

[0051] Step S130: Perform horizontal anomaly diagnosis processing on the complete data set to locate the fault propagation path and abnormal object through a decision tree algorithm.

[0052] After obtaining the complete data set, it needs to be processed for transverse anomaly diagnosis to locate the propagation path and abnormal object of the fault. The decision tree algorithm is a commonly used machine learning algorithm, which can classify and decide according to the characteristics and rules of data, and is very suitable for fault diagnosis and anomaly positioning.

[0053] Step S131: Extracting a call path feature set from the complete data set, the call path feature set containing the number of call nodes, the call sequence between nodes, the single node processing duration and the node response state identifier.

[0054] The call path feature set reflects the basic characteristics and state of cross-system calls. First, the call records in the complete data set are parsed to extract the relevant information of each call path. By analyzing the call initiator and receiver information in the call record, the number of nodes in the call path and the call sequence between nodes are determined. For example, if a call path is initiated from system A, it successively calls system B, system C, and finally returns the result to system A, then the number of nodes in the call path is 3, and the call sequence between nodes is A->B->C->A.

[0055] Then, according to the timestamp information in the call record, the processing duration of each node is calculated. The time difference between the timestamp of the call initiation and the timestamp of the node returning the response is the processing duration of the node. At the same time, according to the response information of the node, the response state identifier of the node is determined, such as response success, response failure, response timeout, etc.

[0056] The extracted call node number, call sequence between nodes, single node processing duration and node response state identifier are arranged and combined to form a call path feature set. When forming the feature set, the features can be standardized to convert different types of features into a unified format and range for subsequent analysis and processing.

[0057] Step S132: Inputting the call path feature set into the feature screening module of the decision tree algorithm, selecting key features as split attributes according to the information gain rate, the key features containing single node processing duration anomaly identifier and node response state anomaly identifier.

[0058] The feature screening module of the decision tree algorithm is used to select key features as split attributes from the call path feature set to construct a decision tree model. Information gain rate is a commonly used feature selection index, which can measure the importance of a feature for classification.

[0059] After the set of call path features is input into the feature screening module, the module evaluates each feature and calculates its information gain ratio. The information gain ratio is calculated based on the entropy and conditional entropy of the data. The entropy represents the uncertainty of the data, and the conditional entropy represents the uncertainty of the data given a certain feature. The information gain ratio is the ratio of the information gain and the inherent value of the feature, which can effectively avoid selecting features with more values.

[0060] For the single-node processing duration feature, a normal processing duration range can be set according to historical data and business experience. If the processing duration of a node exceeds the processing duration range, the single-node processing duration anomaly identifier of the node is set to abnormal. For the node response state identifier, if the response state of the node is failure or timeout, the node response state anomaly identifier of the node is set to abnormal.

[0061] According to the calculation result of the information gain ratio, the feature with an information gain ratio greater than a set threshold is selected as a key feature. In this embodiment, the single-node processing duration anomaly identifier and the node response state anomaly identifier usually have a high information gain ratio, and are therefore selected as key features as splitting attributes.

[0062] Step S133: constructing a decision tree classification model based on the splitting attributes, wherein the internal nodes of the decision tree classification model represent feature splitting conditions, and the leaf nodes represent fault propagation path types.

[0063] After the splitting attributes are determined, a decision tree classification model can be constructed based on the splitting attributes. The decision tree classification model is a tree structure composed of internal nodes and leaf nodes.

[0064] The internal nodes represent feature splitting conditions and divide the data according to the values of the splitting attributes. For example, if the single-node processing duration anomaly identifier is selected as a splitting attribute, the internal nodes will divide the data into different subsets according to the values of the feature (normal or abnormal). Each subset will continue to be divided according to other splitting attributes until a set stop condition is met.

[0065] The leaf nodes represent fault propagation path types. During the construction of the decision tree, when a data subset meets the stop condition, such as the number of samples in the data subset being less than a certain threshold or the purity of the data subset reaching a set standard, the data subset is marked as a leaf node, and the corresponding fault propagation path type is determined according to the sample features in the data subset.

[0066] The decision tree classification model is constructed in a recursive manner. Starting from the root node, the data is divided according to the split attribute to generate child nodes. Then, the above process is repeated for each child node until all nodes are processed. During the construction process, pruning algorithms can be used to optimize the decision tree, removing unnecessary nodes and branches, and improving the generalization ability and accuracy of the decision tree.

[0067] Step S134: Traverse the call paths in the complete data set through the decision tree classification model, and identify abnormal nodes that meet the split condition as candidate abnormal objects.

[0068] After the decision tree classification model is constructed, the call paths in the complete data set are input into the decision tree classification model for traversal.

[0069] Starting from the root node of the decision tree, compare the feature values of the call paths with the split conditions of the internal nodes, and select the corresponding branches to continue traversal. When traversing to the leaf node, determine whether the call path is abnormal according to the fault propagation path type of the leaf node.

[0070] During the traversal process, if the feature value of a node meets the split condition of an internal node, and the split condition is related to an abnormal situation, the node is identified as a candidate abnormal object. For example, if the split condition of an internal node is a single-node processing time length abnormality identifier, when a call path is traversed, a node in the call path has a single-node processing time length abnormality identifier, then the node is identified as a candidate abnormal object.

[0071] Record and organize the identified candidate abnormal objects to form a candidate abnormal object set. When recording the candidate abnormal objects, record their node identifiers, call paths, related feature values, etc. for subsequent analysis and processing.

[0072] Step S135: Perform path backtracking processing on the candidate abnormal objects, analyze the position of the abnormal node in the call path and the dependency relationship with other nodes, determine the abnormal propagation range of the subsequent nodes triggered by the abnormal node, and generate a fault propagation path containing the abnormal starting node, propagation direction, and affected node set.

[0073] After obtaining the candidate abnormal object set, path backtracking processing needs to be performed on these candidate abnormal objects to determine the fault propagation path.

[0074] The path backtracking process starts from the candidate abnormal object, traces back along the call path, and analyzes the position of the abnormal node in the call path and its dependency with other nodes. In the complete data set, the call relationship of each system node is recorded explicitly, and through these records, it can be clearly seen which pre-node calls triggered the abnormal node and which subsequent nodes it will call.

[0075] First, determine the pre-node of the abnormal node. According to the records of the call path, find the pre-node that directly calls the abnormal node. Analyze the running state and related features of the pre-node to see if the pre-node also has an abnormality. If the pre-node is normal, it can be preliminarily judged that the abnormality is generated in the abnormal node itself; if the pre-node also has an abnormality, it is necessary to further trace the pre-node of the pre-node, and so on, until the starting node of the abnormality is found.

[0076] Next, analyze the dependency relationship between the abnormal node and the subsequent node. After the abnormal node appears abnormal, it may propagate the exception to the subsequent node with which it has a call relationship. By checking the call path records, determine the set of subsequent nodes called by the abnormal node. For each subsequent node, check its running state and related features after the abnormal node appears abnormal. If the response state identifier of the subsequent node changes to abnormal, or the single node processing time changes significantly, it can be considered that the subsequent node is affected by the exception propagation.

[0077] When determining the affected subsequent nodes, the calling order and time relationship between nodes need to be considered comprehensively. If a subsequent node also appears abnormal soon after the abnormal node appears abnormal, and there is a direct call relationship between them, then the subsequent node is likely to be affected by the exception propagation. But if the subsequent node's abnormality occurs a long time after the abnormal node's abnormality, or there is no direct call relationship between them, further analysis is needed to determine whether other factors caused the subsequent node to be abnormal.

[0078] In order to more accurately determine the scope of exception propagation, service dependency relationship in system association information can also be combined. There may be indirect service dependency relationship between some system nodes, even if they have no direct call records in the call path, the exception may also be propagated through the above indirect service dependency relationship. By analyzing the service dependency relationship, potential affected nodes can be found.

[0079] After determining the abnormal starting node, propagation direction, and affected node set, these information are integrated to generate a fault propagation path. The fault propagation path describes in detail the propagation of the anomaly from the starting node, through which nodes, and ultimately affecting which nodes, thereby enabling rapid localization of the fault source and taking appropriate measures to block the propagation of the anomaly and restore normal operation of the system.

[0080] Step S136: Extracting nodes with error response status and nodes with processing duration exceeding the reference threshold from the fault propagation path as the final located abnormal objects.

[0081] After obtaining the fault propagation path, the final abnormal objects need to be extracted from it. The response status being error and the processing duration exceeding the reference threshold are two important indicators for judging node abnormalities.

[0082] For each node in the fault propagation path, check its response status identifier. If the response status identifier shows error, it means that the node has problems in the running process and cannot normally complete the task. This error may be caused by internal code errors, resource shortages, network failures, etc. Extract these nodes with error response status as possible abnormal objects.

[0083] At the same time, set a reference processing duration threshold for each node. The reference processing duration threshold is determined according to the historical running data and business requirements of the system, representing the reasonable time required for the node to complete the task under normal circumstances. For the nodes in the fault propagation path, compare their actual processing duration with the reference processing duration threshold. If the actual processing duration of a node exceeds the reference processing duration threshold, it means that the processing speed of the node has slowed down significantly, and there may be performance problems or it has been affected by other abnormal factors. Extract these nodes with processing duration exceeding the reference threshold as possible abnormal objects.

[0084] Combine the nodes with error response status and the nodes with processing duration exceeding the reference threshold to obtain the final located abnormal object set. These abnormal objects are nodes that have obvious abnormalities in the fault propagation process and need to be analyzed and processed to solve the fault problems in the system.

[0085] Step S140: Perform longitudinal abnormality detection processing on the abnormal objects, analyze the running logs, error stacks, and infrastructure indicators of the abnormal objects based on a multi-dimensional abnormality detection algorithm, and generate an abnormal reason analysis result.

[0086] After locating the abnormal objects, longitudinal anomaly detection processing is needed to analyze the causes of the abnormalities. Multi-dimensional anomaly detection algorithms combine information such as the running logs, error stacks, and infrastructure indicators of abnormal objects, and can more comprehensively and accurately analyze the causes of abnormalities.

[0087] Step S141: Extract the running log data of the abnormal object from the complete data set, which contains timestamps, operation types, input parameters, and output results.

[0088] The complete data set records detailed running information of each system node, from which the running log data of the abnormal object is extracted. Running log data is a detailed record of the system running process, which is of great value for analyzing the causes of abnormalities.

[0089] According to the identification information of the abnormal object, the running log records related to the abnormal object are filtered out from the complete data set. Each running log record contains timestamps, operation types, input parameters, and output results, and other key information. The timestamp records the specific time of the operation, and by analyzing the timestamp, the time sequence and time interval of the abnormality can be understood; the operation type describes the specific operation of the system, such as data query, data update, business calculation, etc.; the input parameter is the parameter information used during operation, and the output result is the return result after operation.

[0090] When extracting the running log data, the data needs to be cleaned and organized. Some useless information such as redundant characters and comment information in the log is removed to ensure that the extracted data is accurate and clear. At the same time, the data is converted to a standard format, and the input parameters and output results are organized according to the specified specifications for subsequent analysis and processing.

[0091] Step S142: Perform lexical analysis on the running log data to extract key operation verbs, data object nouns, and state description adjectives as log feature units.

[0092] In order to extract useful information from the running log data, lexical analysis processing is needed. Lexical analysis is to divide text data into meaningful word units and identify key information.

[0093] Step S1421: Perform text cleaning processing on the running log data to remove non-semantic information and retain operation description text content.

[0094] Some non-semantic information may be contained in the running log data, such as system-generated timestamp format information, log record number, etc. These information are not directly helpful for analyzing the cause of the exception, and need to be removed. Through regular expression matching and string processing techniques, identify and delete these non-semantic information, only keep the text content related to the operation description. For example, remove the above prefix information in the log record "[2024-01-01 12:00:00] INFO:", only keep the operation description text behind.

[0095] Step S1422: input the operation description text content into a word segmentation tool for segmentation processing, to generate a word segmentation result containing multiple word units.

[0096] Segment the operation description text content using a professional word segmentation tool. The word segmentation tool will segment the text into individual word units according to the language grammar rules and vocabulary library. Different languages and business scenarios may need to choose different word segmentation tools to ensure the accuracy and effectiveness of the segmentation. For example, for Chinese text, you can use the Jieba word segmentation tool; for English text, you can use tools such as NLTK. The word segmentation result is a list containing multiple word units, each of which represents a meaningful word in the text.

[0097] Step S1423: match the operation verbs in the word segmentation result according to the preset verb vocabulary, the operation verbs containing words representing system operation behaviors.

[0098] A verb vocabulary is preset, which contains common operation behavior verbs in the system, such as "query", "update", "delete", "calculate", etc. Match the word segmentation result with the verb vocabulary to find the operation verbs. By traversing each word unit in the word segmentation result, check if it is in the verb vocabulary. If so, mark it as an operation verb, so you can accurately identify the system operation behavior described in the running log.

[0099] Step S1424: match the data object nouns in the word segmentation result according to the preset noun vocabulary, the data object nouns containing words representing operation objects.

[0100] Similarly, a noun vocabulary is preset, which contains common data object nouns in the system, such as "user information", "order data", "inventory record", etc. Match the word segmentation result with the noun vocabulary to find the data object nouns. By traversing each word unit in the word segmentation result, check if it is in the noun vocabulary. If so, mark it as a data object noun, so you can clearly identify the data object targeted by the system operation.

[0101] Step S1425: match the state description adjectives in the segmented result according to a preset adjective library, the state description adjectives containing words representing the operation result state.

[0102] A library of adjectives is preset, which contains adjectives representing the operation result state, such as "success", "failure", "normal", "abnormal", etc. The segmented result is matched with the adjective library to find the state description adjectives therein. By traversing each word unit in the segmented result, it is checked whether it is in the adjective library. If it is, it is marked as a state description adjective, so that the result state of system operation can be understood.

[0103] Step S1426: frequency statistical processing is performed on the operation verbs, data object nouns and state description adjectives matched, and words with an appearance frequency exceeding a preset threshold are selected as key log feature units.

[0104] The operation verbs, data object nouns and state description adjectives matched are subjected to frequency statistics. The number of times each word appears in the entire running log data is counted. Then, a preset threshold is set, and words with an appearance frequency exceeding the threshold are selected as key log feature units. These key log feature units represent important information frequently appearing in the running log, and have important reference value for analyzing abnormal reasons. For example, if an operation verb frequently appears, it indicates that the operation may be a key operation in the system; if a data object noun frequently appears, it indicates that the data object may be the focus of attention for abnormal occurrence.

[0105] Step S1427: the key log feature units are arranged and combined in the order of operation verb-data object noun-state description adjective to generate a feature unit set representing the core semantics of the log.

[0106] The key log feature units selected are arranged and combined in the order of operation verb-data object noun-state description adjective. The above arrangement and combination method can clearly express the core semantics of the running log. For example, a combination such as "query-user information-failure" can directly indicate that the system failed when querying user information. All the above arrangement and combination results are collected to generate a feature unit set representing the core semantics of the log. The feature unit set can more accurately reflect the system operation and result state recorded in the running log.

[0107] Step S143: perform syntax analysis processing on the error stack data of the abnormal object, identify the stack call level, exception class name and exception trigger code line number, and generate a stack structure feature.

[0108] The error stack data records the call stack information of the system when the exception occurs. By parsing the syntax of the error stack data, the specific location and cause of the exception can be understood in depth.

[0109] First, the error stack data is parsed. Error stack data is usually recorded in a set format, such as the error stack of Java language, which displays the class name, method name and code line number of each call level in a set format. Through regular expression matching and string processing technology, the error stack data is parsed into individual call level records.

[0110] Then, identify the stack call level. Each call level record represents a method call of the system when the exception occurs. By sorting the parsed call level records, the order of the call level can be determined. Starting from the bottommost call level, gradually analyze upwards to understand how the system goes from one method call to another, ultimately leading to the occurrence of the exception.

[0111] Next, extract the exception class name. In each call level record, the name of the exception class is usually included. Through string extraction technology, the exception class name is extracted from the call level record. The exception class name can reflect the type of exception, such as NullPointerException, SQLException, etc., and different exception class names represent different exception causes.

[0112] Finally, determine the exception trigger code line number. In the call level record, it can be clearly indicated that the exception is triggered at which code line. By parsing the call level record, the exception trigger code line number is extracted. This code line number can accurately locate the specific position of the exception occurrence, and the developer can view the corresponding code according to this code line number to find out the cause of the exception.

[0113] Combine the identified stack call level, exception class name and exception trigger code line number to generate the stack structure feature. This stack structure feature can comprehensively describe the call stack situation of the system when the exception occurs.

[0114] Step S144: input the log feature unit and the stack structure feature into the semantic understanding model, analyze the semantic association relationship between the log content and the stack information, and extract a set of semantic features representing the exception trigger condition.

[0115] The semantic understanding model is a trained artificial intelligence model that can understand the semantic information of text data and analyze the association between different texts.

[0116] The log feature unit set and the stack structure feature are input into a semantic understanding model. The semantic understanding model processes the input data and analyzes the semantic association relationship between the log content and the stack information. For example, the model determines whether the operation failure recorded in the log is related to the exception class and the exception trigger code line number shown in the stack information.

[0117] Through analysis of the semantic association relationship, the model extracts a semantic feature set representing the exception trigger condition. The semantic feature set contains key information that can describe the cause of the exception. For example, if the log records "query-data object-failure", and the stack information shows that it is caused by a database connection exception, the semantic understanding model extracts a semantic feature such as "database connection exception causes data object query failure".

[0118] Step S145: Obtain infrastructure indicator data of the device where the exception object is located from the infrastructure monitoring system, the infrastructure indicator data including computing resource occupation data, network transmission data, and storage access data.

[0119] The infrastructure monitoring system monitors the running state and performance indicators of each device in real time. The infrastructure indicator data of the device where the exception object is located is obtained from the system, which can reflect the hardware resource usage and network communication of the device.

[0120] The computing resource occupation data includes CPU usage, memory usage, etc. CPU usage reflects the busy degree of the central processor of the device at a certain time. If the CPU usage is too high, it may cause the system to run slowly or even appear abnormal. Memory usage reflects the usage of the memory resources of the device. If the memory is insufficient, it may cause the program to run abnormally.

[0121] The network transmission data includes network bandwidth usage, network delay, etc. Network bandwidth usage reflects the busy degree of the device in network transmission. If the network bandwidth usage is too high, it may cause data transmission delay and affect the normal operation of the system. Network delay reflects the time required for data transmission in the network. If the network delay is too large, it may cause the system to respond slowly.

[0122] The storage access data includes disk I / O usage, storage capacity usage, etc. Disk I / O usage reflects the busy degree of the disk read / write operation of the device. If the disk I / O usage is too high, it may cause the data read / write speed to slow down. Storage capacity usage reflects the usage of the storage resources of the device. If the storage capacity is insufficient, it may cause the data to be unable to be normally stored.

[0123] The infrastructure index data of the device where the abnormal object is located is obtained through data interaction with the interface of the infrastructure monitoring system. When obtaining the data, the data needs to be filtered by time range to ensure that the index data within a certain period of time before and after the abnormality occurs is obtained, so as to more accurately analyze the relationship between the abnormality and the infrastructure index.

[0124] Step S146: Perform multi-dimensional anomaly detection processing on the infrastructure index data, calculate the mean, variance and change rate of the index data through a sliding window, and identify the mutation time point and mutation amplitude of the index data.

[0125] In order to more accurately analyze the abnormal situation in the infrastructure index data, it needs to be processed by multi-dimensional anomaly detection.

[0126] Step S1461: Arrange the infrastructure index data in chronological order to generate an index data sequence with time sequence relationship.

[0127] The infrastructure index data is recorded in chronological order, but the order of the data may be chaotic due to data collection and transmission when obtaining. Therefore, the obtained infrastructure index data needs to be sorted by timestamp. By comparing and sorting the timestamps, the data is rearranged into an index data sequence with time sequence relationship. The index data sequence can clearly show the change of the infrastructure index over time.

[0128] Step S1462: Set a sliding window of fixed length to slide through the index data sequence, and the length of the sliding window matches the average processing time of the cross-system call.

[0129] A sliding window of fixed length is set, and the length of the window matches the average processing time of the cross-system call. The purpose of setting it is to be able to statistically analyze the infrastructure index data within a reasonable time range and more accurately capture the changes in the index data.

[0130] The sliding window is placed at the starting position of the index data sequence, and then the sliding traversal is started. Each time the window is slid by one time unit, the index data contained in the window is also updated. During the sliding process, the index data in the window is statistically analyzed.

[0131] Step S1463: Calculate the mean parameter of the index data in each sliding window, which represents the average level of the index data in the window.

[0132] For each sliding window, the mean parameter of the index data in the window is calculated. The mean parameter can reflect the average level of the index data in the window. The mean parameter is calculated by adding all the index data in the window and dividing the sum by the number of data. For example, for a sliding window containing 10 data points, the mean parameter of the index data in the window is calculated by adding the values of the 10 data points and dividing the sum by 10. The mean parameter can be used as a representative value of the index data in the time period for subsequent comparison and analysis.

[0133] Step S1464: The mean of the squared differences between the index data in each sliding window and the mean parameter is calculated as the variance parameter, which represents the dispersion of the index data in the window.

[0134] After the mean parameter of the index data in each sliding window is calculated, the squared differences between each data point and the mean parameter are calculated. Then the squared differences are added and divided by the number of data to obtain the variance parameter. The variance parameter can reflect the dispersion of the index data in the window. If the variance parameter is large, it indicates that the index data in the window fluctuates greatly, which may be an abnormal situation; if the variance parameter is small, it indicates that the index data is relatively stable.

[0135] Step S1465: The ratio of the difference between the mean parameters of adjacent sliding windows and the mean parameter of the previous window is calculated as the change rate parameter, which represents the fluctuation amplitude of the index data.

[0136] The mean parameters of two adjacent sliding windows are compared, and the difference between them is calculated. Then the difference is divided by the mean parameter of the previous window to obtain the change rate parameter. The change rate parameter can reflect the fluctuation amplitude of the index data in adjacent time periods. If the change rate parameter is large, it indicates that the index data has changed greatly in the adjacent time period, which may be caused by an abnormal event; if the change rate parameter is small, it indicates that the change of the index data is relatively smooth.

[0137] Step S1466: A mutation detection threshold is set, which includes a variance parameter threshold and a change rate parameter threshold.

[0138] In order to determine whether the index data has a mutation, a mutation detection threshold needs to be set. The mutation detection threshold includes a variance parameter threshold and a change rate parameter threshold. The variance parameter threshold is used to determine whether the dispersion degree of the index data in the window exceeds the normal range, and the change rate parameter threshold is used to determine whether the fluctuation amplitude of the index data in adjacent time periods exceeds the normal range. The setting of these thresholds needs to be determined in combination with the historical data and business characteristics of the system. The distribution range of the variance parameter and the change rate parameter under normal circumstances can be statistically obtained by analyzing a large amount of historical infrastructure index data, and then appropriate thresholds can be determined according to the distribution range. For example, a certain percentile of the variance parameter in the historical data is taken as the variance parameter threshold, and a certain percentile of the change rate parameter is taken as the change rate parameter threshold.

[0139] Step S1467: The variance parameters and the change rate parameters of all sliding windows are traversed, and the window positions where the variance parameters exceed the variance parameter threshold or the change rate parameters exceed the change rate parameter threshold are identified as mutation time points.

[0140] After calculating the variance parameter and the change rate parameter of each sliding window and setting the mutation detection threshold, all sliding windows are traversed. For each sliding window, it is checked whether the variance parameter exceeds the variance parameter threshold or the change rate parameter exceeds the change rate parameter threshold. If any of the conditions is met, the position of the sliding window is marked as a mutation time point. The mutation time point represents the time when the infrastructure index data changes significantly, which may be related to the occurrence of an abnormal event.

[0141] Step S1468: The absolute value of the difference between the index data corresponding to the mutation time point and the index data under normal conditions, which is the mean parameter of the sliding window that has not occurred mutation, is calculated as the mutation amplitude.

[0142] After identifying the mutation time point, the mutation amplitude needs to be calculated. The index data under normal conditions can be represented by the mean parameter of the sliding window that has not occurred mutation. For each mutation time point, the corresponding index data value is found, and then the absolute value of the difference between the value and the index data under normal conditions (the mean parameter of the sliding window that has not occurred mutation) is calculated, which is the mutation amplitude. The mutation amplitude can reflect the degree of change of the index data at the mutation time point. The greater the mutation amplitude, the more dramatic the change of the index data, and the greater the impact on the normal operation of the system.

[0143] Step S147: Analyze the time correlation between the abnormal trigger conditions in the semantic feature set and the mutation time points of the index data, determine the association relationship between the abnormal cause and the infrastructure index change, and generate an abnormal cause analysis result containing the abnormal trigger condition description, the associated infrastructure index type and the association strength.

[0144] After obtaining the abnormal trigger condition in the semantic feature set and the mutation time point of the index data, the time correlation between them needs to be analyzed. By comparing the time when the abnormal trigger condition appears and the mutation time point of the index data, it is checked whether there is a time sequence or synchronization between the two.

[0145] If the time when the abnormal trigger condition appears is very close to the mutation time point of the index data, it can be preliminarily judged that there is a strong correlation between the abnormal reason and the infrastructure index change. For example, if the abnormal trigger condition in the semantic feature set shows "database query failure", and at a similar time point, the disk I / O usage rate in the infrastructure index data has mutated, it can be speculated that the database query failure may be related to the disk I / O performance problem.

[0146] In order to more accurately determine the correlation strength, statistical analysis methods can be used. For example, the correlation coefficient between the number of times the abnormal trigger condition appears and the number of times the index data mutation time point appears is calculated. If the correlation coefficient is high, it means that the correlation strength between the abnormal reason and the infrastructure index change is large; if the correlation coefficient is low, it means that the correlation strength is small.

[0147] Integrate information such as abnormal trigger condition description, associated infrastructure index type, and correlation strength to generate abnormal reason analysis results. The abnormal reason analysis results can clearly show the cause of the abnormality and the relationship with the infrastructure index change.

[0148] Step S150: Determine the fault root cause, fault object and impact range according to the abnormal reason analysis result, and output the cross-system fault diagnosis report.

[0149] After obtaining the abnormal reason analysis result, the fault root cause, fault object and impact range need to be further determined, and these information is arranged into a cross-system fault diagnosis report.

[0150] Step S151: Extract the abnormal trigger condition description and associated infrastructure index type from the abnormal reason analysis result, and determine the direct cause of the abnormality and the underlying infrastructure factor as the fault root cause.

[0151] The abnormal reason analysis result contains information such as abnormal trigger condition description and associated infrastructure index type. By deeply analyzing these information, the direct cause of the abnormality and the underlying infrastructure factor can be determined.

[0152] The direct cause can usually be obtained directly from the abnormal trigger condition description. For example, if the abnormal trigger condition description is "network connection timeout causes service call failure", then the network connection timeout is the direct cause of the abnormality.

[0153] The underlying infrastructure factor can be determined from the associated infrastructure indicator type. If the associated infrastructure indicator type shows that the network bandwidth usage is too high, then the network bandwidth shortage can be the underlying infrastructure factor that causes the network connection timeout. The direct cause and the underlying infrastructure factor are combined to determine the fault root cause. The fault root cause is the fundamental reason that causes the anomaly to occur. Only when the fault root cause is accurately determined, can effective measures be taken to solve the fault.

[0154] Step S152: In combination with the anomaly starting node and the set of affected nodes in the fault propagation path, the initial fault object corresponding to the fault root cause and the secondary fault object subsequently affected are determined.

[0155] The anomaly starting node and the set of affected nodes are recorded in the fault propagation path. The anomaly starting node is where the fault starts, and the system node corresponding to it is the initial fault object. In combination with the fault root cause, it can be determined that the initial fault object is caused by what kind of reason to cause the occurrence of the fault.

[0156] The nodes in the set of affected nodes are nodes that are affected in the fault propagation process. These nodes are the secondary fault objects. By analyzing the calling relationship and dependency relationship between the nodes in the fault propagation path, it can be determined how the initial fault object propagates the fault to the secondary fault object. For example, if the initial fault object is a database server, the data query fails due to the disk I / O performance problem, and the database server provides data services for multiple application servers, then these application servers can become secondary fault objects, because they depend on the data provided by the database server.

[0157] Step S153: Analyze the service dependency relationship between the initial fault object and the secondary fault object to determine the specific link of the fault propagation through the calling path and the influence degree of each link.

[0158] Step S1531: Extract the calling path information between the initial fault object and the secondary fault object from the complete data set, the calling path information including calling interface identification, data transmission content, and calling timing relationship.

[0159] The calling path information between the nodes in the complete data set is recorded. The calling path information between the initial fault object and the secondary fault object is extracted from the complete data set by the identification information of the initial fault object and the secondary fault object. The calling interface identification in the calling path information can clearly indicate which interfaces the initial fault object and the secondary fault object communicate through; the data transmission content can understand which data is transmitted in the calling process; and the calling timing relationship can master the calling sequence and time interval.

[0160] Step S1532: segmenting the call path information, dividing the call path into interface call segment, data transmission segment and result feedback segment.

[0161] The extracted call path information is segmented according to function and process. The interface call segment is the process of the initial fault object initiating a call to the secondary fault object, including sending and receiving of the call request; the data transmission segment is the process of data transmission from the initial fault object to the secondary fault object in the call process, including data packaging, transmission and analysis; the result feedback segment is the process of the secondary fault object returning the call result to the initial fault object, including result generation and return.

[0162] Step S1533: in the interface call segment, analyze whether the request parameters when the initial fault object initiates a call to the secondary fault object are abnormal, and determine the first influence degree of the interface call segment.

[0163] In the interface call segment, check the request parameters when the initial fault object initiates a call to the secondary fault object. If the request parameters do not conform to the specifications or expectations of the interface, it may cause the call to fail or an exception. For example, a certain key field in the request parameters is empty or has a format error, which may cause the secondary fault object to be unable to correctly process the request. Through analysis of the request parameters, the influence degree of the interface call segment caused by the fault can be evaluated, and the influence degree is defined as the first influence degree. The first influence degree can be quantified according to the severity and frequency of the request parameter abnormalities, for example, the severity of the request parameter abnormalities is divided into three levels of slight, moderate and severe, which correspond to different influence degree scores.

[0164] Step S1534: in the data transmission segment, analyze whether the data content transmitted by the initial fault object to the secondary fault object is complete or the format is correct, and determine the second influence degree of the data transmission segment.

[0165] In the data transmission segment, check the data content transmitted by the initial fault object to the secondary fault object. If the data content is incomplete, it may cause the secondary fault object to be unable to obtain enough information for processing; if the data format is incorrect, it may cause the secondary fault object to be unable to correctly parse the data. Through analysis of the completeness of the data content and the correctness of the format, the influence degree of the data transmission segment caused by the fault can be evaluated, and the influence degree is defined as the second influence degree. Similarly, the second influence degree can be quantified according to the severity and frequency of the data content incompleteness or format error.

[0166] Step S1535: In the result feedback link, analyze whether the time for the secondary fault object to receive the initial fault object response result is overdue or the result content is incorrect, to determine the third influence degree of the result feedback link.

[0167] In the result feedback link, check the situation of the secondary fault object receiving the initial fault object response result. If the return time of the response result is overdue, it may cause the secondary fault object to be unable to obtain the processing result in time, affecting the subsequent business process; if the result content is incorrect, it may cause the secondary fault object to make an incorrect decision. Through the analysis of the return time and content of the response result, the influence degree of the result feedback link affected by the fault can be evaluated, and the influence degree is defined as the third influence degree. The third influence degree can be quantified according to the length of time for the response result to be overdue and the severity of the result content error.

[0168] Step S1536: Calculate a weighted average value according to the first influence degree, the second influence degree, and the third influence degree, to generate a comprehensive evaluation value representing the overall influence degree of the fault propagation link.

[0169] In order to obtain the overall influence degree of the fault propagation link, the first influence degree, the second influence degree, and the third influence degree need to be weighted and averaged. Different links may have different importance in the fault propagation process, so each influence degree needs to be assigned a corresponding weight. The allocation of weights can be determined according to the business characteristics and actual situation of the system. For example, if the data transmission link is crucial to the normal operation of the system, a higher weight can be assigned to the second influence degree. Multiply the first influence degree, the second influence degree, and the third influence degree by the corresponding weight respectively, and then add the results to obtain the weighted average value, which is the comprehensive evaluation value of the overall influence degree of the fault propagation link. The comprehensive evaluation value can more comprehensively reflect the influence degree of the fault propagation in each link.

[0170] Step S154: Count the number of systems, services, and associated business transactions contained in the affected node set to generate a quantitative description of the breadth of fault influence.

[0171] Step S1541: Perform system type classification processing on the nodes in the affected node set, and count the number of nodes belonging to different business systems as the number of affected systems.

[0172] The affected node set can include multiple system nodes of different types. By classifying the system types of these nodes, the number of nodes belonging to different business systems can be counted. For example, in an enterprise information system, there can be different business systems such as a financial system, a human resource system, and a sales system. By classifying the nodes in the affected node set, the number of affected nodes in each business system is counted, and then these numbers are added to obtain the number of affected systems. The number of affected systems can reflect the degree of spread of the fault among different business systems.

[0173] Step S1542: Perform service function classification processing on the nodes in the affected node set, and count the number of service nodes providing different business functions as the number of affected services.

[0174] In addition to classifying by system type, the nodes in the affected node set also need to be classified by service function. Each system node can provide different business functions, for example, a server node can provide user authentication services, data storage services, etc. By classifying the nodes in the affected node set by service function, the number of service nodes providing different business functions is counted, which is the number of affected services. The number of affected services can reflect the scope of the impact of the fault on different business functions.

[0175] Step S1543: Extract the business transaction identifiers associated with the affected node set from the complete data set, and count the number of business transactions with unique identifiers as the number of affected business transactions.

[0176] The complete data set records detailed information of each business transaction. By the node identifier in the affected node set, the business transaction identifier associated with it is extracted from the complete data set. Then the number of business transactions with unique identifiers is counted, which is the number of affected business transactions. The number of affected business transactions can reflect the degree of impact of the fault on actual business transactions.

[0177] Step S1544: Build an impact breadth evaluation model, the input of the impact breadth evaluation model being the number of affected systems, the number of affected services, and the number of affected business transactions, and the output being an impact breadth score.

[0178] To comprehensively evaluate the impact range of the fault, an impact range evaluation model needs to be constructed. The input of the impact range evaluation model is the number of affected systems, the number of affected services, and the number of affected business transactions, and the output is an impact range score. Machine learning algorithms or statistical analysis methods can be used to construct the impact range evaluation model. For example, a linear regression model can be used, with the number of affected systems, the number of affected services, and the number of affected business transactions as independent variables, and the impact range score as the dependent variable. Through training on a large amount of historical data, the parameters of the impact range evaluation model are determined. The impact range score can intuitively reflect the impact range of the fault, and the higher the score, the wider the impact range of the fault.

[0179] Step S1545: generating a corresponding text description according to the impact range score, the text description containing a corresponding relationship description of the number of affected systems, the number of affected services, the number of affected business transactions, and the impact range score.

[0180] According to the impact range score, a corresponding text description is generated. The text description will detail the corresponding relationship between the number of affected systems, the number of affected services, the number of affected business transactions, and the impact range score. For example, "This fault has affected multiple business systems, the number of affected systems is large, multiple service nodes have been affected, the number of affected services is large, and the number of associated business transactions is also large. The comprehensive evaluation impact range score is high, indicating that the impact range of the fault is wide." The above text description can make relevant personnel more intuitively understand the impact range of the fault.

[0181] Step S155: analyzing the impact of the fault root cause on business transaction continuity, determining the business interruption duration and transaction failure quantity caused by the fault, and generating a quantitative description representing the fault impact depth.

[0182] Step S1551: extracting the start timestamp and recovery timestamp of the fault occurrence from the complete data set, and calculating the time difference between the start timestamp and the recovery timestamp as the business interruption duration.

[0183] The complete data set records the relevant time information of the fault occurrence. Through analysis of log data and business transaction data, the start timestamp and recovery timestamp of the fault occurrence are extracted. The start timestamp represents the time when the fault begins to affect business transactions, and the recovery timestamp represents the time when business transactions return to normal. The time difference between the two timestamps is calculated, which is the business interruption duration. The business interruption duration can reflect the impact of the fault on business transaction continuity, and the longer the business interruption duration, the more serious the impact of the fault on business.

[0184] Step S1552: Count the number of transactions with a failed state in the business transactions associated with the set of affected nodes during the start timestamp to the recovery timestamp as a transaction failure number.

[0185] After determining the start timestamp and the recovery timestamp of the fault occurrence, the number of transactions with a failed state in the business transactions associated with the set of affected nodes during the time period is counted. By screening and counting the business transaction data in the complete data set, the business transactions related to the set of affected nodes during the fault occurrence period are found, and the number of transactions with a failed state is counted. The transaction failure number can directly reflect the impact of the fault on the business transaction. The more the transaction failure number, the greater the impact of the fault on the business.

[0186] Step S1553: Obtain the number of normal business transactions in the same time period before the fault occurrence as a baseline transaction number.

[0187] In order to more accurately evaluate the impact of the fault on the business transaction, the number of normal business transactions in the same time period before the fault occurrence is obtained as a baseline transaction number. By querying and counting the historical business transaction data in the complete data set, the number of normal business transactions in the historical time period same as the fault occurrence time period is found. The baseline transaction number can provide a reference standard for comparing the business transaction during the fault occurrence period.

[0188] Step S1554: Calculate the ratio of the transaction failure number to the baseline transaction number as a transaction failure rate.

[0189] The ratio obtained by dividing the transaction failure number by the baseline transaction number is the transaction failure rate. The transaction failure rate can directly reflect the impact of the fault on the success rate of the business transaction. The higher the transaction failure rate, the more serious the impact of the fault on the business transaction.

[0190] Step S1555: Build an impact depth evaluation model, the input of the impact depth evaluation model being the business interruption duration, the transaction failure number and the transaction failure rate, and the output being an impact depth score.

[0191] To comprehensively evaluate the impact depth of the fault, an impact depth evaluation model needs to be built. The inputs of the impact depth evaluation model are the business interruption duration, the number of transaction failures, and the transaction failure rate, and the output is the impact depth score. Machine learning algorithms or statistical analysis methods can be used to build the impact depth evaluation model. For example, a multiple linear regression model can be used, with the business interruption duration, the number of transaction failures, and the transaction failure rate as independent variables, and the impact depth score as the dependent variable. Through training on a large amount of historical data, the parameters of the model are determined. The impact depth score can more comprehensively reflect the degree of impact of the fault on business transaction continuity and success rate. The higher the score, the more serious the impact of the fault.

[0192] Step S1556: generating a corresponding text description according to the impact depth score, which contains the corresponding relationship description of the business interruption duration, the number of transaction failures, the transaction failure rate, and the impact depth score.

[0193] According to the impact depth score, a corresponding text description is generated. The text description will detail the corresponding relationship between the business interruption duration, the number of transaction failures, the transaction failure rate, and the impact depth score. For example, "This fault caused a long business interruption duration, a large number of transaction failures, a high transaction failure rate, and a high impact depth score, indicating that the fault had a very serious impact on business transactions." The above text description can help relevant personnel more intuitively understand the impact depth of the fault.

[0194] Step S156: integrating the fault root cause, initial fault object, secondary fault object, impact breadth quantitative description, and impact depth quantitative description to generate a cross-system fault diagnosis report containing text description and chart display.

[0195] The fault root cause, initial fault object, secondary fault object, impact breadth quantitative description, and impact depth quantitative description are integrated to form a cross-system fault diagnosis report. The report will contain detailed text descriptions that comprehensively explain the causes of the fault, the fault objects, and the impact range. At the same time, to more intuitively display the relevant information of the fault, the cross-system fault diagnosis report will also contain chart displays. For example, a column chart can be used to display the number of affected systems, the number of affected services, and the number of affected business transactions; a line chart can be used to display the change trend of the business interruption duration and the transaction failure rate. Through the combination of text description and chart display, relevant personnel can more clearly and comprehensively understand the fault situation.

[0196] Figure 2An exemplary diagram showing exemplary hardware and software components of a cross-system fault diagnosis system 100 with multi-dimensional anomaly detection provided by some embodiments of the present application is shown. For example, a processor 120 can be used in the cross-system fault diagnosis system 100 with multi-dimensional anomaly detection and for performing the functions in the present application.

[0197] The cross-system fault diagnosis system 100 with multi-dimensional anomaly detection can be a general-purpose server or a special-purpose server, both of which can be used to implement the cross-system fault diagnosis method with multi-dimensional anomaly detection of the present application. Although only one server is shown in the present application, for the sake of convenience, the functions described in the present application can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0198] For example, the cross-system fault diagnosis system 100 with multi-dimensional anomaly detection can include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and different forms of storage media 140, such as a disk, a ROM, or a RAM, or any combination thereof. Exemplarily, the cross-system fault diagnosis system 100 with multi-dimensional anomaly detection can also include program instructions stored in a ROM, a RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application can be implemented according to these program instructions. The cross-system fault diagnosis system 100 with multi-dimensional anomaly detection also includes an I / O interface 150 between the computer and other input / output devices.

[0199] For the sake of illustration, only one processor is described in the cross-system fault diagnosis system 100 with multi-dimensional anomaly detection. However, it should be noted that the cross-system fault diagnosis system 100 with multi-dimensional anomaly detection in the present application can also include multiple processors, so the steps performed by one processor described in the present application can also be jointly performed or individually performed by multiple processors. For example, if the processor of the cross-system fault diagnosis system 100 with multi-dimensional anomaly detection performs step A and step B, it should be understood that step A and step B can also be jointly performed by two different processors or individually performed in one processor. For example, a first processor performs step A, a second processor performs step B, or the first processor and the second processor jointly perform steps A and B.

[0200] In addition, the present application also provides a readable storage medium, in which computer executable instructions are pre-set, when the processor executes the computer executable instructions, the cross-system fault diagnosis method with multi-dimensional anomaly detection is implemented.

[0201] It should be noted that the foregoing description of embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form described, and many modifications, variations, and alternatives are possible.

Claims

1. A cross-system fault diagnosis method combining multidimensional anomaly detection, characterized in that, The method includes: Obtain the raw data set generated during the cross-system call process, the raw data set including fragmented call data collected by the link probe, peripheral business transaction data and log data; The original data set is subjected to breakpoint supplementation processing. The breakpoint positions in the fragmented call data are identified based on the application type. The external business transaction data, log data and system association information are integrated to generate a complete data set. The complete dataset is subjected to lateral anomaly diagnosis and processing, and the fault propagation path and anomaly objects are located by using a decision tree algorithm; The abnormal object is subjected to longitudinal anomaly detection processing. Based on the multi-dimensional anomaly detection algorithm, the runtime log, error stack and infrastructure indicators of the abnormal object are analyzed to generate anomaly cause analysis results. Based on the results of the anomaly analysis, the root cause, the fault object, and the scope of impact are determined, and a cross-system fault diagnosis report is output.

2. The cross-system fault diagnosis method combining multidimensional anomaly detection according to claim 1, characterized in that, The step involves performing breakpoint supplementation on the original data set, identifying breakpoint locations in the fragmented call data based on application type, and integrating the external business transaction data, log data, and system correlation information to generate a complete data set, including: The fragmented call data is processed for application type matching to identify the call protocol characteristics and data interaction patterns corresponding to different application types; Based on the calling protocol characteristics and data interaction mode, missing data nodes are marked as breakpoint locations in the fragmented calling data; Extract transaction identifier information and a set of interaction parameters aligned with the timestamp of the breakpoint from the peripheral business transaction data; Extract request initiation records, response reception records, and intermediate processing log entries associated with the breakpoint location from the log data; The transaction identifier information, interaction parameter set, request initiation record, response reception record, and intermediate processing log entries are used as supplementary breakpoint data. The system association information is pre-stored and includes service dependencies for cross-system calls, interface call rules, and infrastructure deployment location information. The breakpoint supplementary data is fused with the system association information to fill in the missing data nodes in the fragmented call data, generating a complete data set.

3. The cross-system fault diagnosis method combining multidimensional anomaly detection according to claim 1, characterized in that, The process of performing lateral anomaly diagnosis on the complete dataset, and locating the fault propagation path and anomaly objects using a decision tree algorithm, includes: Extract a call path feature set from the complete dataset. The call path feature set includes the number of calling nodes, the calling order between nodes, the processing time of a single node, and the node response status identifier. The call path feature set is input into the feature filtering module of the decision tree algorithm, and key features are selected as splitting attributes according to the information gain ratio. The key features include single node processing time abnormality identifier and node response status abnormality identifier. A decision tree classification model is constructed based on the aforementioned splitting attribute. The internal nodes of the decision tree classification model represent feature splitting conditions, and the leaf nodes represent fault propagation path types. The decision tree classification model traverses the call paths in the complete dataset to identify abnormal nodes that meet the splitting conditions as candidate abnormal objects. The candidate abnormal objects are subjected to path backtracking processing to analyze the position of the abnormal node in the call path and its dependency relationship with other nodes, determine the scope of abnormal propagation of subsequent nodes triggered by the abnormal node, and generate a fault propagation path that includes the abnormal starting node, propagation direction and the set of affected nodes. Nodes with incorrect response status and nodes whose processing time exceeds a baseline threshold are extracted from the fault propagation path and used as the finally located anomaly objects.

4. The cross-system fault diagnosis method combining multidimensional anomaly detection according to claim 1, characterized in that, The process involves performing vertical anomaly detection on the abnormal object, analyzing its runtime logs, error stacks, and infrastructure metrics based on a multi-dimensional anomaly detection algorithm, and generating anomaly cause analysis results, including: Extract the runtime log data of the abnormal object from the complete dataset. The runtime log data includes timestamps, operation types, input parameters, and output results. Lexical analysis was performed on the operation log data to extract key operation verbs, data object nouns, and status description adjectives as log feature units; Syntactic analysis is performed on the error stack data of the exception object to identify the stack call level, exception class name and exception trigger code line number, and generate stack structure features. The log feature units and the stack structure features are input into the semantic understanding model to analyze the semantic relationship between log content and stack information, and to extract a set of semantic features representing abnormal triggering conditions. The infrastructure indicator data of the device where the abnormal object is located is obtained from the infrastructure monitoring system. The infrastructure indicator data includes computing resource usage data, network transmission data, and storage access data. Multidimensional anomaly detection processing is performed on the infrastructure indicator data. By using a sliding window to statistically analyze the mean, variance, and rate of change of the indicator data, the time points and magnitudes of abrupt changes in the indicator data are identified. The temporal correlation between the anomaly triggering conditions in the semantic feature set and the mutation time point of the indicator data is analyzed to determine the correlation between the anomaly cause and the changes in infrastructure indicators, and anomaly cause analysis results containing descriptions of anomaly triggering conditions, types of associated infrastructure indicators, and correlation strength are generated.

5. The cross-system fault diagnosis method combining multidimensional anomaly detection according to claim 4, characterized in that, The step of performing lexical analysis on the operation log data to extract key operation verbs, data object nouns, and state description adjectives as log feature units includes: The operation log data is cleaned to remove non-semantic information and retain the operation description text content; The operation description text is input into a word segmentation tool for segmentation, generating a word segmentation result containing multiple word units; The word segmentation results are matched with operation verbs based on a preset verb lexicon, and the operation verbs include words that represent system operation behaviors; The data object nouns in the word segmentation results are matched according to a preset noun lexicon, and the data object nouns contain words that represent the operation objects; The state description adjectives in the word segmentation results are matched according to a preset adjective lexicon, and the state description adjectives include words that indicate the state of the operation result; Frequency statistics are performed on the matched operation verbs, data object nouns, and state description adjectives, and words with a frequency exceeding a preset threshold are selected as key log feature units. The key log feature units are arranged and combined in the order of operation verb - data object noun - state description adjective to generate a set of feature units representing the core semantics of the log.

6. The cross-system fault diagnosis method combining multidimensional anomaly detection according to claim 4, characterized in that, The multidimensional anomaly detection processing of the infrastructure indicator data, which involves using a sliding window to statistically analyze the mean, variance, and rate of change of the indicator data, to identify the time points and magnitudes of abrupt changes in the indicator data, includes: The infrastructure indicator data are arranged in timestamp order to generate an indicator data sequence with a time series relationship; A fixed-length sliding window is set to slide across the indicator data sequence, and the length of the sliding window is matched with the average processing time of cross-system calls; Calculate the mean parameter for the indicator data within each sliding window, where the mean parameter represents the average level of the indicator data within the window; The average of the squared differences between the index data and the mean parameter within each sliding window is calculated as the variance parameter, which represents the degree of dispersion of the index data within the window. The ratio of the difference between the mean parameters of adjacent sliding windows to the mean parameter of the previous window is used as the rate of change parameter, which represents the fluctuation range of the indicator data. Set a mutation detection threshold, which includes a variance parameter threshold and a rate of change parameter threshold; Iterate through the variance and rate of change parameters of all sliding windows, and identify the window positions where the variance parameter exceeds the variance parameter threshold or the rate of change parameter exceeds the rate of change parameter threshold as the abrupt change time points. The absolute value of the difference between the indicator data at the time point of mutation and the indicator data under normal conditions is calculated as the mutation amplitude. The indicator data under normal conditions is the mean parameter of the sliding window where no mutation has occurred.

7. The cross-system fault diagnosis method combining multidimensional anomaly detection according to claim 1, characterized in that, The process involves determining the root cause, fault object, and scope of impact based on the anomaly analysis results, and outputting a cross-system fault diagnosis report, including: Extract the description of the anomaly triggering conditions and the types of related infrastructure indicators from the anomaly cause analysis results, and determine the direct cause of the anomaly and the underlying infrastructure factors as the root cause of the failure. By combining the abnormal starting node and the set of affected nodes in the fault propagation path, the initial fault object corresponding to the fault root cause and the subsequent affected secondary fault objects are determined. Analyze the service dependencies between the initial fault object and the secondary fault objects to determine the specific steps in the fault propagation through the call path and the degree of impact of each step; The number of systems, services, and associated business transactions included in the affected node set are counted to generate a quantitative description representing the breadth of the fault's impact. Analyze the impact of the root cause of the failure on the continuity of business transactions, determine the duration of business interruption and the number of failed transactions caused by the failure, and generate a quantitative description representing the depth of the failure's impact. The root cause of the fault, the initial fault object, the secondary fault object, the quantitative description of the breadth of impact, and the quantitative description of the depth of impact are integrated to generate a cross-system fault diagnosis report that includes text descriptions and charts.

8. The cross-system fault diagnosis method combining multidimensional anomaly detection according to claim 7, characterized in that, The analysis of the service dependencies between the initial faulty object and the secondary faulty object determines the specific stages of fault propagation through the call path and the degree of impact of each stage, including: Extract the call path information between the initial fault object and the secondary fault object from the complete dataset. The call path information includes the call interface identifier, data transmission content, and call sequence relationship. The call path information is broken down into stages, dividing the call path into interface call stage, data transmission stage, and result feedback stage; In the API call phase, analyze whether the request parameters when the initial faulty object makes a call to the secondary faulty object are abnormal, and determine the first degree of impact of the API call phase. In the data transmission process, analyze whether the data content transmitted from the initial fault object to the secondary fault object is complete or the format is correct, and determine the second degree of impact of the data transmission process. In the result feedback stage, analyze whether the time for the secondary fault object to receive the response result from the initial fault object exceeds the time limit or whether the result content is incorrect, and determine the third degree of influence of the result feedback stage. A weighted average is calculated based on the first degree of impact, the second degree of impact, and the third degree of impact to generate a comprehensive assessment value representing the overall impact of the fault propagation process.

9. The cross-system fault diagnosis method combining multidimensional anomaly detection according to claim 7, characterized in that, The statistical analysis includes the number of systems, services, and associated business transactions contained in the affected node set, generating a quantitative description representing the breadth of the fault's impact, including: The nodes in the affected node set are classified by system type, and the number of nodes belonging to different business systems is counted as the number of affected systems. The nodes in the affected node set are classified according to their service functions, and the number of service nodes providing different business functions is counted as the number of affected services. Extract the business transaction identifiers associated with the set of affected nodes from the complete data set, and count the number of business transactions with unique identifiers as the number of affected business transactions; Construct an impact breadth assessment model, the input of which is the number of affected systems, the number of affected services, and the number of affected business transactions, and the output is the impact breadth score; A corresponding text description is generated based on the impact breadth score. The text description includes an explanation of the correspondence between the number of affected systems, the number of affected services, the number of affected business transactions, and the impact breadth score.

10. A cross-system fault diagnosis system combining multi-dimensional anomaly detection, characterized in that, The system includes a processor and a memory, the memory being connected to the processor. The memory is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the memory to implement the cross-system fault diagnosis method combining multidimensional anomaly detection as described in any one of claims 1-9.

Images