A method, device, medium and equipment for locating root cause of microservice application system

Through multi-dimensional abnormality detection and call chain analysis, the root cause node of microservices is automatically determined, which solves the problem of low efficiency of fault location of artificially dependent microservices in the existing technology, and realizes fast and accurate positioning and efficient microservice failure analysis.

CN114528175BActive Publication Date: 2024-12-03ASIAINFO TECH CHINA INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011195383.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-10-30
Publication Date
2024-12-03
Estimated Expiration
2040-10-30

AI Technical Summary

Technical Problem

The existing microservice failure root cause location methods rely on manual experience and labeling, resulting in high labor costs, low efficiency and poor accuracy, and the root cause nodes of microservice abnormalities cannot be quickly and accurately positioned.

Method used

Using multi-dimensional anomaly detection method, through absolute threshold detection method and call chain analysis, combined with search algorithm and pruning algorithm, the abnormal nodes and call chains of microservices are automatically detected, and the root cause nodes of microservices are determined, and multiple abnormal detections are passed. Methods Analyze the platform indicators and determine the root cause indicators.

Benefits of technology

It realizes rapid and accurate positioning of microservice failures, reduces labor costs, improves fault positioning efficiency and accuracy, and adapts to complex microservice call relationship changes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114528175B_ABST
    Figure CN114528175B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a method, device, medium and equipment for locating the root cause of a microservice application system, which is applied to the field of information technology. The method includes: performing anomaly detection on the microservice application system to obtain the abnormal microservice and the abnormal time corresponding to the abnormal microservice; then, detecting the call chain of the abnormal microservice and the corresponding microservice node, and locating the fault according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node. The embodiment of the present application realizes the rapid and accurate positioning of the fault of the microservice application system, and solves the technical problems of the existing fault positioning technology, such as large workload, low efficiency and large limitations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of information technology, and more specifically, to a method, device, medium and equipment for locating root causes of a microservice application system. Background Art

[0002] With the increasing popularity of cloud computing and big data technologies, and the rapid development of distributed technologies, more and more Internet and software companies are beginning to build microservice architectures. Under the microservice architecture, a large and complex application can be split into multiple microservices with high cohesion, low coupling, and statelessness. Each microservice is built around business functions and can be independently deployed through an automatic deployment mechanism and run in its own process. Usually, the deployment of each microservice includes multiple different microservice nodes, and the call relationship between microservice nodes forms a call chain. In a complete call chain, if any node is delayed or abnormal, it may cause the final result to be abnormal. Therefore, the stability of microservices is crucial, and when an abnormality occurs, the result needs to be located quickly and accurately.

[0003] Existing solutions for locating the root cause of microservice failures usually rely on expert experience or manually annotated data to locate the root cause node. Therefore, not only is the labor cost high, but the efficiency and accuracy of root cause node location are also low. Summary of the invention

[0004] The present application provides a method, device, medium and equipment for locating the root cause of a microservice application system, which are used to solve the problems of high labor cost, low efficiency and great limitations in the process of locating microservice faults.

[0005] In a first aspect, the present application provides a method for locating root causes of a microservice application system, the method comprising:

[0006] Perform anomaly detection on the microservice application system to obtain abnormal microservices and the abnormal time corresponding to the abnormal microservices;

[0007] Detect the call chain of abnormal microservices and the corresponding microservice nodes, and locate the fault according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node.

[0008] In an embodiment of the present application, after detecting the call chain of an abnormal microservice and the corresponding microservice node, locating the fault according to the abnormal time and the call logic of the call chain, and obtaining the microservice root cause node, the following is also included:

[0009] According to the microservice root cause node and abnormal time, the platform indicators of the corresponding microservice are obtained to perform anomaly detection and analysis to determine the root cause indicators.

[0010] In the embodiment of the present application, anomaly detection is performed on the microservice application system to obtain an abnormal microservice and an abnormal time corresponding to the abnormal microservice, including:

[0011] The absolute threshold detection method is used to detect the golden index anomaly of the microservice application system and obtain the fault type;

[0012] Determine the abnormal microservice and the abnormal time corresponding to the abnormal microservice according to the fault type.

[0013] In the embodiment of the present application, an absolute threshold detection method is used to perform golden index anomaly detection on the microservice application system to obtain the fault type, including:

[0014] Detect the average latency and success rate of microservice requests in microservice application systems;

[0015] If the average delay is not less than the preset average delay threshold and the success rate is not less than the preset success rate threshold, the fault type is time consumption abnormality;

[0016] If the average delay is less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, the fault type is abnormal success rate;

[0017] If the average delay is less than the average delay threshold and the success rate is zero, the fault type is database abnormality;

[0018] If the average delay is not less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, then the fault type is that both the time consumption and the success rate are abnormal;

[0019] If the average delay is not less than the average delay threshold and the success rate is zero, the fault type is that both the time consumption and the database are abnormal.

[0020] In the embodiment of the present application, the call chain of the abnormal microservice and the corresponding microservice node are detected, and the fault is located according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node, including:

[0021] Filter the call chain of abnormal microservices to obtain the abnormal call chain;

[0022] According to the calling logic of the abnormal call chain, a search algorithm is used to perform abnormal analysis on the nodes of the abnormal call chain to obtain abnormal nodes and pending nodes. The abnormal nodes and pending nodes are candidate root cause nodes of microservices.

[0023] The call chain location algorithm is used to locate and analyze the candidate root cause nodes to obtain the microservice root cause node.

[0024] In an embodiment of the present application, the call chain of the abnormal microservice is screened to obtain the abnormal call chain, including:

[0025] Use normal distribution to count the call chain time consumed in each microservice request of abnormal microservices;

[0026] Filter out abnormal time-consuming data in the call chain time-consuming statistics based on the abnormal time;

[0027] The call chain corresponding to the abnormal time-consuming data is determined to be the abnormal call chain.

[0028] In an embodiment of the present application, according to the abnormal time, the abnormal time-consuming data in the call chain time-consuming statistics is filtered out, including:

[0029] Obtain the data in the preset time period before the abnormal moment in the call chain time consumption statistics as the data to be screened;

[0030] The data to be screened that are within the confidence interval under the preset confidence level in the normal distribution statistics are screened out as abnormal time-consuming data.

[0031] In an embodiment of the present application, according to the call logic of the abnormal call chain, a search algorithm is used to perform abnormal analysis on the nodes of the abnormal call chain to obtain abnormal nodes and pending nodes, which are candidate root cause nodes of microservices, including:

[0032] Use normal distribution to count the call time of nodes in abnormal call chains;

[0033] According to the calling logic of the abnormal call chain and the calling time of the node, the pruning algorithm is used to search the nodes of the abnormal call chain layer by layer to obtain the abnormal nodes and pending nodes. The abnormal nodes and pending nodes are the candidate root cause nodes of the microservice.

[0034] In the embodiment of the present application, a call chain location algorithm is used to locate and analyze the candidate root cause nodes to obtain the microservice root cause nodes, including:

[0035] Count the number of failures and failure rates of candidate root cause nodes during multiple calls;

[0036] According to the depth, number of failures and failure rate of the candidate root cause nodes, the candidate root cause nodes are analyzed layer by layer to determine the root cause node.

[0037] In the embodiment of the present application, according to the microservice root cause node and the abnormal time, the platform indicator of the corresponding microservice is obtained to perform abnormal detection analysis and determine the root cause indicator, including:

[0038] Detect the platform indicators of the components corresponding to the microservice root cause nodes within the preset time;

[0039] According to the platform indicators and abnormal moments, a combination of multiple anomaly detection methods is used to perform anomaly detection and analysis on the platform indicators to obtain the root cause indicators.

[0040] In the embodiment of the present application, according to the platform indicators and the abnormal time, a combination of multiple abnormal detection methods is used to perform abnormal detection analysis on the platform indicators to obtain root cause indicators, including:

[0041] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the test results are all abnormal, the platform indicator is taken as the root cause indicator;

[0042] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the detection result obtained is that one of them is abnormal, if the detection result obtained by the absolute threshold method to detect the platform indicator is abnormal, the platform indicator is used as the root cause indicator;

[0043] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the test results obtained are normal, the platform indicator is regarded as a normal indicator;

[0044] If both the deviation analysis method and the fluctuation point analysis method are used to detect platform indicators, and one of the detection results is abnormal, if the absolute threshold method is used to detect platform indicators and the result is normal, the platform indicator will be regarded as a normal indicator.

[0045] In an embodiment of the present application, the platform indicators include one or more of CPU utilization, memory utilization, number of database connections, and network transceiver queue length.

[0046] In a second aspect, the present application provides a root cause location device for a microservice application system, the device comprising:

[0047] The anomaly detection module is used to detect anomalies in the microservice application system and obtain the abnormal microservice and the abnormal time corresponding to the abnormal microservice;

[0048] The root cause node location module is used to detect the call chain of abnormal microservices and the corresponding microservice nodes, and locate the fault according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node.

[0049] In an embodiment of the present application, the above-mentioned microservice application system root cause location device further includes:

[0050] The root cause indicator determination module is used to obtain the platform indicators of the corresponding microservices according to the microservice root cause nodes and abnormal time, perform abnormality detection and analysis, and determine the root cause indicators.

[0051] In an embodiment of the present application, the above-mentioned abnormality detection module includes:

[0052] The anomaly detection unit is used to perform golden indicator anomaly detection on the microservice application system using an absolute threshold detection method to obtain the fault type;

[0053] The abnormality determination unit is used to determine the abnormal microservice and the abnormal time corresponding to the abnormal microservice according to the fault type.

[0054] In the embodiment of the present application, the above-mentioned abnormality detection unit is specifically used for:

[0055] Detect the average latency and success rate of microservice requests in microservice application systems;

[0056] If the average delay is not less than the preset average delay threshold and the success rate is not less than the preset success rate threshold, the fault type is time consumption abnormality;

[0057] If the average delay is less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, the fault type is abnormal success rate;

[0058] If the average delay is less than the average delay threshold and the success rate is zero, the fault type is database abnormality;

[0059] If the average delay is not less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, then the fault type is that both the time consumption and the success rate are abnormal;

[0060] If the average delay is not less than the average delay threshold and the success rate is zero, the fault type is that both the time consumption and the database are abnormal.

[0061] In an embodiment of the present application, the root cause node positioning module includes:

[0062] A screening unit is used to screen the call chain of abnormal microservices to obtain the abnormal call chain;

[0063] A search unit is used to perform abnormal analysis on the nodes of the abnormal call chain using a search algorithm according to the call logic of the abnormal call chain to obtain abnormal nodes and pending nodes, which are candidate root cause nodes of the microservice;

[0064] The node locating unit is used to locate and analyze the candidate root cause nodes using a call chain locating algorithm to obtain the microservice root cause node.

[0065] In an embodiment of the present application, the screening unit comprises:

[0066] The statistical subunit is used to use normal distribution to count the call chain time consumption in each microservice request of abnormal microservices;

[0067] The filtering subunit is used to filter out abnormal time-consuming data in the call chain time-consuming statistics according to the abnormal time;

[0068] The call chain determination subunit is used to determine that the call chain corresponding to the abnormal time-consuming data is an abnormal call chain.

[0069] In the embodiments of the present application, the above screening subunit is specifically used for:

[0070] Obtain the data in the preset time period before the abnormal moment in the call chain time consumption statistics as the data to be screened;

[0071] The data to be screened that are within the confidence interval under the preset confidence level in the normal distribution statistics are screened out as abnormal time-consuming data.

[0072] In the embodiment of the present application, the above-mentioned search unit is specifically used for:

[0073] Use normal distribution to count the call time of nodes in abnormal call chains;

[0074] According to the calling logic of the abnormal call chain and the calling time of the node, the pruning algorithm is used to search the nodes of the abnormal call chain layer by layer to obtain the abnormal nodes and pending nodes. The abnormal nodes and pending nodes are the candidate root cause nodes of the microservice.

[0075] In the embodiment of the present application, the node positioning unit is specifically used for:

[0076] Count the number of failures and failure rates of candidate root cause nodes during multiple calls;

[0077] According to the depth, number of failures and failure rate of the candidate root cause nodes, the candidate root cause nodes are analyzed layer by layer to determine the root cause node.

[0078] In an embodiment of the present application, the root cause indicator determination module includes:

[0079] A platform indicator detection unit, used to detect the platform indicator of the component corresponding to the microservice root cause node within a preset time;

[0080] The root cause indicator determination unit is used to perform anomaly detection and analysis on the platform indicators according to the platform indicators and the abnormal time by combining multiple anomaly detection methods to obtain the root cause indicators.

[0081] In an embodiment of the present application, the root cause indicator determination unit is specifically used to:

[0082] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the test results are all abnormal, the platform indicator is taken as the root cause indicator;

[0083] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the detection result obtained is that one of them is abnormal, if the detection result obtained by the absolute threshold method to detect the platform indicator is abnormal, the platform indicator is used as the root cause indicator;

[0084] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the test results obtained are normal, the platform indicator is regarded as a normal indicator;

[0085] If both the deviation analysis method and the fluctuation point analysis method are used to detect platform indicators, and one of the detection results is abnormal, if the absolute threshold method is used to detect platform indicators and the result is normal, the platform indicator will be regarded as a normal indicator.

[0086] In an embodiment of the present application, the above-mentioned platform indicators include one or more of CPU utilization, memory utilization, number of database connections, and network transceiver queue length.

[0087] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for locating the root cause of a microservice application system as shown in any embodiment of the first aspect of the present application.

[0088] In a fourth aspect, the present application provides an electronic device, the electronic device comprising: a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface communicate with each other via the communication bus;

[0089] The memory is used to store at least one executable instruction, which enables the processor to perform operations corresponding to the root cause location method of the microservice application system as shown in any embodiment of the first aspect of the present application.

[0090] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0091] The present application performs multi-dimensional anomaly detection on the microservice application system, first discovers faults in the microservice system to obtain abnormal microservices and abnormal times, and then further detects the call chain and nodes of the abnormal microservices to obtain the root cause nodes, thereby achieving decoupling analysis of microservice operation and maintenance scenarios, helping to clarify the actual business of each scenario, while reducing the complexity of the algorithms for each scenario, overcoming the technical problem of the existing technology that requires a lot of manpower costs, and ensuring the efficiency and stability of the root cause location method of the microservice application system. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in describing the embodiments of the present application.

[0093] Figure 1 A schematic diagram of a process for locating the root cause of a microservice application system provided in an embodiment of the present application;

[0094] Figure 2A schematic diagram of a microservice node call relationship in a microservice application system root cause location method provided in an embodiment of the present application;

[0095] Figure 3 A schematic diagram of a complete microservice call chain in a microservice application system root cause location method provided in an embodiment of the present application;

[0096] Figure 4 A schematic diagram of the structure of a root cause location device for a microservice application system provided in an embodiment of the present application;

[0097] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0098] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0099] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0100] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0101] As the calling levels between business systems become deeper and deeper and the calling relationships become increasingly complex, the number of microservices may be tens of thousands, and these microservices may be deployed on thousands of different servers. Various microservices carry and call many components, and the number of monitoring indicators of each component varies. In such a complex environment, it is difficult to troubleshoot and locate the specific components and indicators that cause microservice abnormalities, and it is impossible to quickly locate and solve the root cause of the problem.

[0102] The existing microservice fault root cause location solutions are mainly the following:

[0103] 1. Expert experience positioning: After a fault occurs, the first-end microservice information called by the current business is manually determined, and then based on the call link tracking information, the call relationship and node abnormal status are analyzed layer by layer to finally find the root cause microservice node. Then, based on the existing topological relationship of the microservice and expert experience, the possible root cause of the platform indicator is inferred.

[0104] However, since there are tens of thousands of microservices and their calling relationships are complex, manual troubleshooting based on expert prior knowledge is too costly in terms of manpower investment and can only locate partial faults, and cannot adapt to the frequent changes in microservice calling relationships.

[0105] 2. Random forest algorithm: Manually select effective indicators from a large number of data samples, and then mark the abnormal root cause nodes corresponding to the data samples. Then use the manually marked data for training, and train multiple random decision trees at the same time. Vote on the positioning results of each decision tree, and locate the abnormal root cause nodes based on the voting results.

[0106] However, for manual labeling, relatively accurate abnormal samples in the real environment are relatively scarce, making it difficult to cover various abnormal situations. In addition, the workload of manual labeling is very large and time-consuming.

[0107] 3. Random walk algorithm: Collect multi-indicator data, build an influence diagram based on abnormal characteristics and indicator data weights, and use the random walk algorithm to analyze the previous and subsequent self-calls to calculate the root cause.

[0108] Since the implementation of the random walk algorithm is based on indicator weights, it is necessary to define the range of indicators that affect the operation of microservices based on manual experience. This method is suitable for specific environments where the microservice call relationship is relatively stable and simple, and its universality is not high.

[0109] In view of this, the embodiment of the present application proposes a root cause location method for a microservice application system. First, a business golden indicator anomaly detection model is established to actively discover business anomalies. Secondly, based on big data statistical methods, the call chain tracking information is comprehensively analyzed horizontally and vertically to locate the root cause node of the faulty microservice. Finally, based on the platform indicator anomaly detection and root cause location model, and combined with the static topological relationship, the real root cause of the microservice anomaly is finally located, thereby achieving active detection of business anomalies and rapid and accurate location of the real root cause, without the need for manual labeling and without relying on expert prior knowledge, solving the problems of large workload, large limitations, and long time consumption of manual analysis or manual labeling. In addition, the method is more general and more universal.

[0110] The method proposed in the embodiment of the present application may include the following steps:

[0111] First, golden indicator anomaly detection: detect the abnormal situation of golden indicators in ESB (Enterprise ServiceBus) business through absolute threshold detection method, and return the abnormal time and abnormal microservice name;

[0112] Second, locate the root cause of the call chain: Based on the detection results of the golden indicator in the first step, further detect the abnormal conditions of all microservice call chains within the preset time period before the abnormal moment of the specific microservice, and locate the abnormal node according to the call chain relationship, and return the abnormal time, abnormal microservice name, and the root cause node of the microservice;

[0113] Third, platform indicator anomaly detection: Based on the microservice abnormal node at a certain abnormal moment located in the second step, the topological relationship of the associated platform architecture is used to further detect the abnormal conditions of various indicators on the platform side and finally find the root cause indicator.

[0114] In order to better understand the root cause location method of the microservice application system provided in the embodiment of the present application, Figure 1-5 The method is described in detail.

[0115] In an embodiment of the present application, a method 10 for locating the root cause of a microservice application system is provided. Figure 1 As shown, the method may include the following steps:

[0116] S1, perform anomaly detection on the microservice application system to obtain the abnormal microservice and the abnormal time corresponding to the abnormal microservice.

[0117] S2, detect the call chain of the abnormal microservice and the corresponding microservice node, and locate the fault according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node.

[0118] In the microservice architecture, a large and complex application can be split into multiple microservices with high cohesion, low coupling, and statelessness, such as Figure 2 As shown in the figure, it is a schematic diagram of the call relationship of microservice nodes. The deployment of each microservice includes multiple different microservice nodes, and the call relationship between the microservice nodes forms a call link. Figure 2 In the , the entire microservice system is divided into three categories of microservices: A, B, and C. Each category of microservices contains multiple microservice nodes. For example, category A microservices include microservice nodes such as A1, A2, and A3. Each microservice node can be a component of the microservice system, which can be a terminal or a technical component (message middleware, network server) or a database. When the system issues a business request, a call link is formed such as A2-A3, A3-B1, B1-C1, etc.

[0119] The present application performs multi-dimensional anomaly detection on the microservice application system, first discovers faults in the microservice system to obtain abnormal microservices and abnormal times, and then further detects the call chain and nodes of the abnormal microservices to obtain the root cause nodes, thereby achieving decoupling analysis of microservice operation and maintenance scenarios, helping to clarify the actual business of each scenario, while reducing the complexity of the algorithms for each scenario, overcoming the technical problem of the existing technology that requires a lot of manpower costs, and ensuring the efficiency and stability of the root cause location method of the microservice application system.

[0120] A possible implementation method is provided in an embodiment of the present application, and step S1 may specifically include:

[0121] S11, use the absolute threshold detection method to perform golden indicator anomaly detection on the microservice application system to obtain the fault type.

[0122] S12, determining the abnormal microservice and the abnormal time corresponding to the abnormal microservice according to the fault type.

[0123] Among them, the golden indicator detection is to detect the key indicators of microservices. When the business response time becomes longer, it means that the system's processing performance has declined; if the request success rate drops suddenly or the error rate rises suddenly, it means that there may be problems with the system's own processing logic. In this way, the different business indicators of various microservices are classified into the above two categories of response time and request success rate for anomaly detection, which improves the efficiency and effect of microservice fault detection.

[0124] Another possible implementation method is provided in the embodiment of the present application. Step S11 may specifically include:

[0125] Detect the average latency and success rate of microservice requests in microservice application systems;

[0126] If the average delay is not less than the preset average delay threshold and the success rate is not less than the preset success rate threshold, the fault type is time consumption abnormality;

[0127] If the average delay is less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, the fault type is abnormal success rate;

[0128] If the average delay is less than the average delay threshold and the success rate is zero, the fault type is database abnormality;

[0129] If the average delay is not less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, then the fault type is that both the time consumption and the success rate are abnormal;

[0130] If the average delay is not less than the average delay threshold and the success rate is zero, the fault type is that both the time consumption and the database are abnormal.

[0131] In practical applications, the average delay threshold and the success rate threshold can be set based on the abnormal proportion in actual experience. For example, the average delay threshold is 1s, and the success rate threshold is 90%. In the embodiment of the present application, the golden indicator abnormality detection of the microservice application system is performed by adopting the absolute threshold detection method and the classification idea, and the fault types are carefully classified, which solves the technical problem that it is difficult to cover multiple types of abnormal situations in the prior art, and achieves comprehensive coverage of different fault types in actual business.

[0132] Another possible implementation method is provided in the embodiment of the present application, and step S2 may specifically include:

[0133] S21, filter the call chain of the abnormal microservice to obtain the abnormal call chain;

[0134] Specifically, in this embodiment, a complete call chain of a microservice request presents a one-way propagation relationship of the link, and each call chain has a unique identifier to distinguish it from other call chains.

[0135] S22, according to the call logic of the abnormal call chain, use the search algorithm to perform abnormal analysis on the nodes of the abnormal call chain, and obtain abnormal nodes and pending nodes, which are candidate root cause nodes of the microservice;

[0136] S23, using the call chain location algorithm to locate and analyze the candidate root cause nodes to obtain the microservice root cause node.

[0137] Another possible implementation method is provided in the embodiment of the present application. Step S21 may specifically include:

[0138] S211, using normal distribution to count the call chain time consumed in each microservice request of the abnormal microservice; wherein the call chain time consumed is the sum of the time consumed by all microservice nodes included in the above call chain.

[0139] S212, filtering out abnormal time-consuming data in the call chain time-consuming statistics according to the abnormal time.

[0140] S213, determining that the call chain corresponding to the abnormal time-consuming data is an abnormal call chain.

[0141] Another possible implementation method is provided in the embodiment of the present application, and step S212 specifically includes:

[0142] Obtain the data in the preset time period before the abnormal moment in the call chain time consumption statistics as the data to be screened;

[0143] The data to be screened that are within the confidence interval under the preset confidence level in the normal distribution statistics are screened out as abnormal time-consuming data. In order to improve the efficiency of data analysis, the data within 5 minutes before the abnormal moment can be obtained first, and then the data belonging to the 90% confidence interval can be screened out from the above data.

[0144] Another possible implementation method is provided in the embodiment of the present application, and step S22 specifically includes:

[0145] Use normal distribution to count the call time of nodes in abnormal call chains;

[0146] According to the call logic of the abnormal call chain and the call time of the node, the pruning algorithm is used to search the nodes of the abnormal call chain layer by layer to obtain abnormal nodes and pending nodes. The abnormal nodes and pending nodes are the candidate root cause nodes of the microservice. Specifically, the node time consumption data belonging to the 5 minutes before the abnormal moment in the normal distribution statistics of the node call time consumption can be obtained. If the call time of the node is greater than 45% of the time consumption of its parent node, it is considered that the node meets the time consumption abnormality condition; if the call time of the node is not greater than 45% of the time consumption of its parent node, it is considered that the node does not meet the time consumption abnormality condition. The judgment logic of abnormal nodes and pending nodes can be:

[0147] If the node meets the time-consuming exception condition, or the node call fails, the node is an abnormal node;

[0148] If the node does not meet the time consumption exception condition, or the node call is successful, but the parent node of the node takes more than twice the total time consumption of all nodes in the same layer, the node is a pending node. Usually, when the search determines that the node is a pending node or the node has no child nodes, the search stops.

[0149] The following is combined with Figure 3The following is an example of a complete microservice call chain, which describes in detail the process of using the pruning algorithm to search the nodes of the abnormal call chain layer by layer:

[0150] Starting from the first node, we record A_001 as an abnormal node, and then record the node information {'A_001':[1,1,0],}(node ​​name:[number of abnormalities, depth, whether abnormal (abnormal 0, pending 1)]);

[0151] Continue to search down to the B_001 node: The B_001 node meets the time-consuming abnormal condition, so the node information is recorded {'A_001': [1,1,0], 'B_001': [1,2,0],};

[0152] Knowing that B_001 is an abnormal node, continue to search down to B_003 node: B_003 node does not meet the time consumption but is a node that failed to call, so record the node information {'A_001': [1,1,0], 'B_001': [1,2,0], 'B_003': [1,3,0],};

[0153] Knowing that B_003 is an abnormal node, continue to search down to C_015 and C_016 nodes: C_15 node does not meet the time-consuming abnormal condition but is a node that failed to call, so record the node information {'A_001':[1,1,0],'B_001':[1,2,0],'B_003':[1,3,0],'C_015':[1,4,0]};

[0154] The C_016 node does not meet the time-consuming abnormal condition and the call is successful, so C_016 is not an abnormal node; the time consumption of B_003 is 20s, and the time consumption of B_003 child nodes is self-call, C_015, and C_016 (4s, 3s, and 2s), which is more than twice the total time consumption of child node calls. Therefore, C_016 is a pending node, and the node information is recorded {'A_001': [1, 1, 0], 'B_001': [1, 2, 0], 'B_003': [1, 3, 0], 'C_015': [1, 4, 0] 'C_016': [1, 4, 1]};

[0155] Knowing that C_015 is an abnormal node, continue to search down to D_003 node: D_003 node does not meet the abnormal time consumption condition and the call is successful, so D_003 is not an abnormal node; the time consumption of C_015 is 3s and the time consumption of C_015 child node D_003 is 1s, which is more than twice the time consumption of the child node call, so D_003 is a pending node, then record the node information {'A_001': [1, 1, 0], 'B_001': [1, 2, 0], 'B_003': [1, 3, 0], 'C_015': [1, 4, 0], 'C_016': [1, 4, 1], 'D_003': [1, 5, 1]};

[0156] Knowing that C_016 is a pending node and D_003 has no child nodes, stop searching.

[0157] The call time and whether the call is successful can reflect the node failure information. If the bottom node takes a long time or fails to call, the upper node will also take a longer time or fail to call. Locating the root cause node based on the call logic of the above microservice call chain can improve the efficiency of microservice node location analysis.

[0158] Another possible implementation method is provided in the embodiment of the present application, and step S23 specifically includes:

[0159] Count the number of failures and failure rates of candidate root cause nodes during multiple calls;

[0160] According to the depth, number of failures and failure rate of the candidate root cause nodes, the candidate root cause nodes are analyzed layer by layer to determine the root cause node.

[0161] According to the call logic of the call chain, the root cause is transmitted from bottom to top, so the lower-level nodes are more likely to become the root cause nodes. The distribution status of abnormal nodes can be understood based on the statistical information of the candidate root cause nodes of the microservices. The specific call chain positioning algorithm is as follows:

[0162] According to the node time consumption data belonging to the 5 minutes before the abnormal moment in the normal distribution statistics of the node call consumption obtained above, the total number of call chains, the node call depth, the number of node calls and the number of abnormalities are counted, and the analysis starts from the deepest candidate root cause node:

[0163] If the candidate root cause node abnormality times meet:

[0164] m 异常 ≥0.8 (deep-2) ×0.05×n 调用链 ,

[0165] And the abnormal rate satisfies p 异常 =m 异常 ÷n节点 ≥50%, the node is determined as the pending root cause node.

[0166] If the candidate root cause node pending times meet:

[0167] l 待定 ≥0.8 (deep-2) ×0.1×n 调用链 ,

[0168] And the pending rate satisfies p 待定 = l 待定 ÷n 节点 ≥50%, the node is determined to be a possible root cause node.

[0169] Among them, m 异常 The number of times the node is identified as an abnormal node, l 待定 is the number of times the node is determined as a pending node, deep is the hierarchical depth of the node, n 调用链 is the total number of call chains, n 节点 The number of times the node is called.

[0170] If there is a single pending root cause node at a certain level of depth, the node is directly determined as the microservice root cause node; if there are multiple pending root cause nodes at a certain level of depth, the node with more exceptions is determined as the microservice root cause node; if there are multiple pending root cause nodes at a certain level of depth and the number of exceptions is the same, the node with a higher exception rate is returned as the microservice root cause node; if the number of exceptions is equal to the exception rate, the docker node is returned as the microservice root cause node.

[0171] In the root cause node location scenario, the call logic of the call chain is effectively utilized to traverse and analyze the status of the microservice nodes of the abnormal call chain, and a pruning strategy is adopted to speed up the search. The call chain location algorithm is then used to determine the root cause node through the number of failures and the failure rate. This solves the technical problems of time-consuming and inefficient root cause location in the existing technology, thereby achieving the technical effect of quickly and accurately locating the root cause node of the microservice call chain.

[0172] Another possible implementation is provided in the embodiment of the present application. After step S2, method 10 may further include the following steps:

[0173] S3, according to the microservice root cause node and abnormal time, obtain the platform indicators of the corresponding microservice to perform abnormality detection analysis and determine the root cause indicators.

[0174] In this embodiment, the platform indicators may include one or more of: CPU utilization, memory utilization, number of database connections, and network transceiver queue length.

[0175] Another possible implementation method is provided in the embodiment of the present application, and step S3 specifically includes:

[0176] S31, detecting the platform indicators of the components corresponding to the microservice root cause nodes within a preset time;

[0177] S32, based on the platform indicators and the abnormal time, a combination of multiple abnormal detection methods is used to perform abnormal detection analysis on the platform indicators to obtain root cause indicators.

[0178] Another possible implementation method is provided in the embodiment of the present application. Step S32 may specifically include the following steps:

[0179] If both the deviation analysis method and the fluctuation point analysis method are used to detect platform indicators, and the test results are all abnormal, the platform indicator is used as the root cause indicator; if both the deviation analysis method and the fluctuation point analysis method are used to detect platform indicators, and one of the test results is abnormal, if the test result obtained by using the absolute threshold method to detect platform indicators is abnormal, the platform indicator is used as the root cause indicator;

[0180] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the test results obtained are normal, the platform indicator is regarded as a normal indicator;

[0181] If both the deviation analysis method and the fluctuation point analysis method are used to detect platform indicators, and one of the detection results is abnormal, if the absolute threshold method is used to detect platform indicators and the result is normal, the platform indicator will be regarded as a normal indicator.

[0182] In actual operation, the three statistical methods of deviation analysis, fluctuation point analysis and absolute threshold method can be encapsulated and integrated into the function. The input of the function call is the detection time and the observed indicator column. The detection time is usually the abnormal time, and the observed indicator column is all platform indicator data obtained within the preset time. The output of the function call is whether the indicator has an abnormality within 5 minutes before the abnormal time.

[0183] Specifically, the specific process of using the deviation analysis method to detect platform indicators is as follows: according to all the input platform indicator data, the mean and maximum values ​​in the data column are obtained, and the deviation threshold is set according to the data input, and the deviation of the indicator data corresponding to a certain moment is calculated according to the following formula: Disp i =(series i -median series )÷max series ;

[0184] If Disp i >Thres disp, then the indicator data is abnormal at this moment;

[0185] If Disp i ≤Thres disp , then the indicator data is normal at this moment;

[0186] Among them, Disp i is the deviation of the indicator data corresponding to a certain moment, series i is the indicator data at that moment, median series is the mean value in the data column, max series is the maximum value in the data column, Thres disp is the deviation threshold.

[0187] The fluctuation point analysis method mainly uses the sliding window statistical method to capture the fluctuation characteristics of indicator data over time. The specific process of using the fluctuation point analysis method to detect platform indicators is: all input platform indicator data is time sliced ​​using a window length of 3 minutes, and the above time window slice data is statistically analyzed using normal distribution to obtain the mean and variance of each time window slice data:

[0188] If |newseries i -u newseries |≥3×σ newseries , then the indicator data is abnormal at this moment;

[0189] If |newseries i -u newseries |<3×σ newseries , then the indicator data is normal at this moment;

[0190] Among them, newseries i The time window slice of the input indicator column, u newseries is the mean of the time window slice sequence, σ newseries is the variance of the time window slice series.

[0191] The specific process of using the absolute threshold method to detect platform indicators is to first arrange the input platform indicator data in order from small to large, and then take the value in the middle order as the absolute threshold.

[0192] If series i ≥median normalseries , then the indicator data is abnormal at this moment;

[0193] If series i <median normalseries , then the indicator data is normal at this moment;

[0194] Among them, series i is the indicator data at that moment, median normalseries The input platform indicator data is arranged in order from small to large and then the value in the middle is taken.

[0195] In the root cause indicator detection scenario, the idea of ​​ensemble learning is used to combine multiple anomaly detection methods, which enhances the robustness of the indicator detection algorithm and the accuracy of the root cause indicator confirmation method. The business experience is integrated into the indicator detection algorithm through the absolute threshold method, which enhances the practicality of the indicator detection algorithm.

[0196] Based on the same inventive concept, the present application embodiment also provides a root cause location device for a microservice application system. Figure 4 As shown, the microservice application system root cause location device 20 may include: an anomaly detection module 201 and a root cause node location module 202, wherein:

[0197] The anomaly detection module 201 is used to perform anomaly detection on the microservice application system and obtain the abnormal microservice and the abnormal time corresponding to the abnormal microservice;

[0198] The root cause node location module 202 is used to detect the call chain of the abnormal microservice and the corresponding microservice node, and locate the fault according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node.

[0199] Another possible implementation method is provided in the embodiment of the present application. The microservice application system root cause location device 20 further includes:

[0200] The root cause indicator determination module is used to obtain the platform indicators of the corresponding microservices according to the microservice root cause nodes and abnormal time, perform abnormality detection and analysis, and determine the root cause indicators.

[0201] Another possible implementation is provided in the embodiment of the present application. The above-mentioned abnormality detection module 201 may include:

[0202] The anomaly detection unit is used to perform golden indicator anomaly detection on the microservice application system using an absolute threshold detection method to obtain the fault type;

[0203] The abnormality determination unit is used to determine the abnormal microservice and the abnormal time corresponding to the abnormal microservice according to the fault type.

[0204] Another possible implementation method is provided in the embodiment of the present application. The above-mentioned abnormality detection unit can be specifically used for:

[0205] Detect the average latency and success rate of microservice requests in microservice application systems;

[0206] If the average delay is not less than the preset average delay threshold and the success rate is not less than the preset success rate threshold, the fault type is time consumption abnormality;

[0207] If the average delay is less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, the fault type is abnormal success rate;

[0208] If the average delay is less than the average delay threshold and the success rate is zero, the fault type is database abnormality;

[0209] If the average delay is not less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, then the fault type is that both the time consumption and the success rate are abnormal;

[0210] If the average delay is not less than the average delay threshold and the success rate is zero, the fault type is that both the time consumption and the database are abnormal.

[0211] Another possible implementation is provided in the embodiment of the present application. The root cause node location module 202 may include:

[0212] A screening unit is used to screen the call chain of abnormal microservices to obtain the abnormal call chain;

[0213] A search unit is used to perform abnormal analysis on the nodes of the abnormal call chain using a search algorithm according to the call logic of the abnormal call chain to obtain abnormal nodes and pending nodes, which are candidate root cause nodes of the microservice;

[0214] The node locating unit is used to locate and analyze the candidate root cause nodes using a call chain locating algorithm to obtain the microservice root cause node.

[0215] Another possible implementation method is provided in the embodiment of the present application. The screening unit may include:

[0216] The statistical subunit is used to use normal distribution to count the call chain time consumption in each microservice request of abnormal microservices;

[0217] The filtering subunit is used to filter out abnormal time-consuming data in the call chain time-consuming statistics according to the abnormal time;

[0218] The call chain determination subunit is used to determine that the call chain corresponding to the abnormal time-consuming data is an abnormal call chain.

[0219] Another possible implementation method is provided in the embodiment of the present application, and the above screening subunit can be specifically used for:

[0220] Obtain the data in the preset time period before the abnormal moment in the call chain time consumption statistics as the data to be screened;

[0221] The data to be screened that are within the confidence interval under the preset confidence level in the normal distribution statistics are screened out as abnormal time-consuming data.

[0222] Another possible implementation method is provided in the embodiment of the present application, and the above-mentioned search unit can be specifically used for:

[0223] Use normal distribution to count the call time of nodes in abnormal call chains;

[0224] According to the calling logic of the abnormal call chain and the calling time of the node, the pruning algorithm is used to search the nodes of the abnormal call chain layer by layer to obtain the abnormal nodes and pending nodes. The abnormal nodes and pending nodes are the candidate root cause nodes of the microservice.

[0225] Another possible implementation method is provided in the embodiment of the present application. The node positioning unit can be specifically used for:

[0226] Count the number of failures and failure rates of candidate root cause nodes during multiple calls;

[0227] According to the depth, number of failures and failure rate of the candidate root cause nodes, the candidate root cause nodes are analyzed layer by layer to determine the root cause node.

[0228] Another possible implementation method is provided in an embodiment of the present application. The root cause indicator determination module may include:

[0229] A platform indicator detection unit, used to detect the platform indicator of the component corresponding to the microservice root cause node within a preset time;

[0230] The root cause indicator determination unit is used to perform anomaly detection and analysis on the platform indicators according to the platform indicators and the abnormal time by combining multiple anomaly detection methods to obtain the root cause indicators.

[0231] Another possible implementation method is provided in the embodiment of the present application. The root cause indicator determination unit can be specifically used for:

[0232] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the test results are all abnormal, the platform indicator is taken as the root cause indicator;

[0233] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the detection result obtained is that one of them is abnormal, if the detection result obtained by the absolute threshold method to detect the platform indicator is abnormal, the platform indicator is used as the root cause indicator;

[0234] If both the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator, and the test results obtained are normal, the platform indicator is regarded as a normal indicator;

[0235] If both the deviation analysis method and the fluctuation point analysis method are used to detect platform indicators, and one of the detection results is abnormal, if the absolute threshold method is used to detect platform indicators and the result is normal, the platform indicator will be regarded as a normal indicator.

[0236] Another possible implementation method is provided in an embodiment of the present application, and the above-mentioned platform indicators may include one or more of CPU utilization, memory utilization, number of database connections, and network transceiver queue length.

[0237] For the contents not described in detail in the root cause locating device for a microservice application system provided in the embodiment of the present application, reference may be made to the root cause locating method for a microservice application system provided in the above embodiment. The beneficial effects that can be achieved by the root cause locating device for a microservice application system provided in the embodiment of the present application are the same as those of the root cause locating method for a microservice application system provided in the above embodiment, and will not be repeated here.

[0238] The application of the embodiments of the present application has at least the following beneficial effects:

[0239] The present application performs multi-dimensional anomaly detection on the microservice application system, first discovers faults in the microservice system to obtain abnormal microservices and abnormal times, and then further detects the call chain and nodes of the abnormal microservices to obtain the root cause nodes, thereby achieving decoupling analysis of the microservice operation and maintenance scenarios, forming two independent modules accordingly, which helps to clarify the actual business of each scenario and reduce the complexity of the algorithms for each scenario, overcoming the technical problem in the prior art that requires a lot of manpower costs, and ensuring the efficiency and stability of the root cause positioning device of the microservice application system.

[0240] Based on the same inventive concept, the embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when it is run on a computer, the computer can execute the corresponding content of the aforementioned method embodiment. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0241] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0242] Based on the same inventive concept, the embodiment of the present application further provides an electronic device 30, such as Figure 5 As shown, the electronic device includes a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the program, the steps of the method in the embodiment are implemented.

[0243] Figure 5 is a schematic diagram of the hardware structure of an electronic device for executing the root cause location method of a microservice application system provided by an embodiment of the present invention, such as Figure 5 As shown, the electronic device includes one or more processors 301 and a memory 302. Figure 5 A processor 301 is taken as an example.

[0244] The electronic device executing the microservice application system root cause location method may further include: an input device 303 and an output device 304 .

[0245] The processor 301, the memory 302, the input device 303 and the output device 304 may be connected via a bus or other means. Figure 5 The connection via bus 305 is taken as an example.

[0246] The processor 301 may be a central processing unit (CPU). The processor 301 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0247] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A method for locating the root cause of a microservice application system, characterized in that: include: Perform anomaly detection on the microservice application system to obtain abnormal microservices and abnormal times corresponding to the abnormal microservices; Detect the call chain of the abnormal microservice and the corresponding microservice node, and locate the fault according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node; The detecting the call chain of the abnormal microservice and the corresponding microservice node, and locating the fault according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node includes: Screening the call chain of the abnormal microservice to obtain the abnormal call chain; According to the calling logic of the abnormal call chain, a search algorithm is used to perform abnormal analysis on the nodes of the abnormal call chain to obtain abnormal nodes and pending nodes, wherein the abnormal nodes and pending nodes are candidate root cause nodes of microservices; Counting the number of failures and failure rates of the candidate root cause nodes during multiple calls, and counting the total number of call chains during multiple calls; In order from deep to shallow levels, if there is a single undetermined root cause node at one level of depth, the undetermined root cause node is used as the microservice root cause node; if there are multiple undetermined root cause nodes at one level of depth and the multiple undetermined root cause nodes have the same number of failures but different abnormality rates, the undetermined root cause node with the highest abnormality rate is used as the microservice root cause node; Among them, the pending root cause node refers to an alternative root cause node that has become an abnormal node more than the first preset threshold number of times and whose failure rate is greater than the second preset threshold, and an alternative root cause node that has become a pending node more than the third preset threshold number of times and whose failure rate is greater than the fourth preset threshold.

2. The root cause location method of a microservice application system according to claim 1, characterized in that: The detecting the call chain of the abnormal microservice and the corresponding microservice node, and locating the fault according to the abnormal time and the call logic of the call chain, and obtaining the microservice root cause node, further includes: According to the microservice root cause node and the abnormal time, the platform indicators of the corresponding microservice are obtained to perform abnormality detection and analysis to determine the root cause indicators.

3. The root cause location method of a microservice application system according to claim 1 or 2, characterized in that: The performing anomaly detection on the microservice application system to obtain an abnormal microservice and an abnormal time corresponding to the abnormal microservice includes: An absolute threshold detection method is used to perform golden indicator anomaly detection on the microservice application system to obtain the fault type; Determine the abnormal microservice and the abnormal time corresponding to the abnormal microservice according to the fault type.

4. The root cause location method of the microservice application system according to claim 3 is characterized in that: The absolute threshold detection method is used to perform golden indicator anomaly detection on the microservice application system to obtain the fault type, including: Detecting the average delay and success rate of microservice requests of the microservice application system; If the average delay is not less than the preset average delay threshold and the success rate is not less than the preset success rate threshold, the fault type is time consumption abnormality; If the average delay is less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, then the fault type is abnormal success rate; If the average delay is less than the average delay threshold and the success rate is zero, the fault type is database abnormality; If the average delay is not less than the average delay threshold, and the success rate is less than the success rate threshold and is not zero, then the fault type is that both the time consumption and the success rate are abnormal; If the average delay is not less than the average delay threshold and the success rate is zero, the fault type is that both the time consumption and the database are abnormal.

5. The root cause location method of a microservice application system according to claim 1, characterized in that: The filtering of the call chain of the abnormal microservice to obtain the abnormal call chain includes: Use normal distribution to count the call chain time consumed in each microservice request of the abnormal microservice; According to the abnormal time, filter out the abnormal time-consuming data in the call chain time-consuming statistics; It is determined that the call chain corresponding to the abnormal time-consuming data is an abnormal call chain.

6. The root cause location method of a microservice application system according to claim 5, characterized in that: The filtering out the abnormal time-consuming data in the call chain time-consuming statistics according to the abnormal time includes: Obtaining data in the call chain time-consuming statistics that is within a preset time period before the abnormal moment as data to be screened; The data to be screened that are within the confidence interval at a preset confidence level in the normal distribution statistics are screened out as abnormal time-consuming data.

7. The root cause location method of a microservice application system according to claim 1, characterized in that: According to the calling logic of the abnormal calling chain, a search algorithm is used to perform abnormal analysis on the nodes of the abnormal calling chain to obtain abnormal nodes and pending nodes, wherein the abnormal nodes and pending nodes are candidate root cause nodes of microservices, including: The calling time consumption of the nodes of the abnormal calling chain is counted using normal distribution; According to the calling logic of the abnormal call chain and the calling time of the node, a pruning algorithm is used to search the nodes of the abnormal call chain layer by layer to obtain abnormal nodes and pending nodes, which are candidate root cause nodes of microservices.

8. The root cause location method of a microservice application system according to claim 2, characterized in that: The step of obtaining the platform indicator of the corresponding microservice for anomaly detection and analysis based on the microservice root cause node and the abnormal time to determine the root cause indicator includes: Detecting the platform indicators of the components corresponding to the microservice root cause nodes within a preset time; According to the platform indicators and the abnormal time, a combination of multiple abnormality detection methods is used to perform abnormality detection analysis on the platform indicators to obtain root cause indicators.

9. The root cause location method of a microservice application system according to claim 8, characterized in that: The method of combining multiple anomaly detection methods according to the platform indicators and the abnormal time is used to perform anomaly detection analysis on the platform indicators to obtain root cause indicators, including: If the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator at the same time, and the detection results obtained are all abnormal, the platform indicator is used as the root cause indicator; if the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator at the same time, and the detection results obtained are that one of them is abnormal, if the detection result obtained by using the absolute threshold method to detect the platform indicator is abnormal, the platform indicator is used as the root cause indicator; If the deviation analysis method and the fluctuation point analysis method are used to detect the platform indicator at the same time, and the test results obtained are both normal, the platform indicator is regarded as a normal indicator; If the deviation analysis method and the fluctuation point analysis method are used simultaneously to detect the platform indicator, and the detection result obtained is that one of the detection results is abnormal, if the absolute threshold method is used to detect the platform indicator and the detection result obtained is that there is no abnormality, then the platform indicator is regarded as a normal indicator.

10. The root cause location method of a microservice application system according to claim 9, characterized in that: The platform indicators include one or more of CPU utilization, memory utilization, number of database connections, and network transceiver queue length.

11. A root cause location device for a microservice application system, characterized in that: include: An anomaly detection module is used to perform anomaly detection on the microservice application system to obtain an abnormal microservice and an abnormal time corresponding to the abnormal microservice; A root cause node location module is used to detect the call chain of the abnormal microservice and the corresponding microservice node, and locate the fault according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node; The root cause node location module is used to detect the call chain of the abnormal microservice and the corresponding microservice node, and locate the fault according to the abnormal time and the call logic of the call chain to obtain the microservice root cause node, which is specifically used to: Screening the call chain of the abnormal microservice to obtain the abnormal call chain; According to the calling logic of the abnormal call chain, a search algorithm is used to perform abnormal analysis on the nodes of the abnormal call chain to obtain abnormal nodes and pending nodes, wherein the abnormal nodes and pending nodes are candidate root cause nodes of microservices; Counting the number of failures and failure rates of the candidate root cause nodes during multiple calls, and counting the total number of call chains during multiple calls; In order from deep to shallow levels, if there is a single undetermined root cause node at one level of depth, the undetermined root cause node is used as the microservice root cause node; If there are multiple pending root cause nodes at one level of depth and the multiple pending root cause nodes have the same number of failures but different abnormality rates, the pending root cause node with the highest abnormality rate is used as the microservice root cause node; Among them, the pending root cause node refers to an alternative root cause node that has become an abnormal node more than the first preset threshold number of times and whose failure rate is greater than the second preset threshold, and an alternative root cause node that has become a pending node more than the third preset threshold number of times and whose failure rate is greater than the fourth preset threshold.

12. The root cause location device for a microservice application system according to claim 11, characterized in that: The microservice application system root cause positioning device also includes: The root cause indicator determination module is used to obtain the platform indicator of the corresponding microservice according to the microservice root cause node and the abnormal time, perform abnormality detection analysis, and determine the root cause indicator.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the root cause location method for a microservice application system according to any one of claims 1 to 10.

14. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the root cause location method of the microservice application system according to any one of claims 1-10.