Micro-service system fault root cause positioning method, device and equipment and storage medium

Abstract

The method and device for locating the root cause of the fault of the micro-service system, the equipment and the storage medium. In at least one embodiment of the present disclosure, based on the data of the code repository and the data of the monitoring platform, a method fault propagation chain is introduced to represent the propagation track of the fault that may be introduced by a certain method modification, so as to improve the explainability of the fault root cause positioning result. On the other hand, by preprocessing the data of the monitoring platform, the micro-service system runtime abnormal method call chain can be obtained, and then based on the method fault propagation chain and the abnormal method call chain set, the fault root cause information including the suspicious method is determined, the fault root cause information is located to the method level, in addition, the fault root cause information can also include the suspicious submission record information, the suspicious method and the suspicious submission record information are associated, and the troubleshooting time under the code change scene is shortened.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, in particular to a micro-service system fault root cause positioning method and device, electronic equipment and non-transitory computer readable storage medium. BACKGROUND

[0002] The fault root cause positioning of the traditional monolithic system can be performed using methods based on incremental debugging, slicing, frequency spectrum, program state analysis and machine learning, which can locate the suspected methods and codes causing the system failure. However, the micro-service system has high dynamicity and complexity, and each micro-service can include multiple instances. According to scheduling requirements, multiple instances of each micro-service can be dynamically created and destroyed in different containers or virtual machines. In addition, the micro-services often include complex call chains, and each call chain can be called thousands of times, and some asynchronous calls are usually included in these call chains. The above factors make it difficult to effectively apply the fault root cause positioning methods of the traditional monolithic system to the micro-service system.

[0003] The fault types in the micro-service system mainly include system-level resource type faults (such as CPU, Memory occupancy, etc.) and application-level business type faults (such as null pointer, parameter passing error induced parsing exception, etc.).

[0004] For system-level resource type faults, researchers have proposed methods such as call chain comparison and machine learning to locate them. The former compares the abnormal call chain on the line with the historical abnormal call chain to obtain the suspected fault root cause micro-service. The latter analyzes the service-level call chain of the system when the fault occurs, trains a machine learning model, and predicts the suspected fault root cause position (service level) in the production system. These methods currently only target system-level resource type faults, and can only locate the fault root cause to the service level (i.e., locate which service has a fault), but cannot go deeper to the method level (i.e., locate which method called by which service has a fault, a service usually calls multiple methods (methods are essentially codes or functions) to provide services), and cannot be associated with commit record information (Commit information).

[0005] For application-level business type faults, there are many types of faults, and enumeration-based fault injection can only cover a small part of them. Existing fault root cause positioning solutions based on incremental debugging, slicing, frequency spectrum and program state analysis can only target a specific micro-service, and cross-service calls are very frequent in the micro-service system, so simply applying these solutions will lose important context information. The machine learning-based solution has relatively poor interpretability, and when the accuracy is not high, the results will make users difficult to accept.

[0006] The above description of the problem discovery process is only used to assist in understanding the technical solutions of the present disclosure and does not mean that the above content is prior art. SUMMARY

[0007] To solve at least one problem existing in the prior art, at least one embodiment of the present disclosure provides a microservice system fault root cause positioning method, device, electronic equipment and non-transitory computer readable storage medium.

[0008] In a first aspect, an embodiment of the present disclosure provides a microservice system fault root cause positioning method, which comprises:

[0009] Obtaining code data, a first commit record information set, application logs and a method call chain set;

[0010] Determining a method fault propagation chain set based on the code data, the first commit record information set, the application logs and the method call chain set;

[0011] Identifying an abnormal method call chain in the method call chain set;

[0012] Aggregating the identified abnormal method call chain based on the application logs and an abnormal link aggregation condition to obtain one or more abnormal method call chain sets;

[0013] Determining fault root cause information based on the method fault propagation chain set and the abnormal method call chain set, the fault root cause information including a suspicious method.

[0014] In some embodiments, the code data and the first commit record information set are derived from a code repository; the data of a monitoring platform includes: the application logs and the method call chain set are derived from the monitoring platform; and the microservice system includes the code repository and the monitoring platform.

[0015] In some embodiments, determining the method fault propagation chain set based on the code data, the first commit record information set, the application logs and the method call chain set comprises:

[0016] Performing static scanning on the code data to determine indirect dependency relationships between methods;

[0017] Determining a modified method set and a second commit record information set corresponding to each modified method based on the first commit record information set;

[0018] Determining the method fault propagation chain set based on the method call chain set, the indirect dependency relationships, the modified method set and the second commit record information set corresponding to each modified method.

[0019] In a second aspect, an embodiment of the present disclosure further provides a microservice system fault root cause positioning device, which comprises:

[0020] The acquisition module is configured to acquire code data, a first commit record information set, application logs, and a method call chain set.

[0021] The fault propagation chain determination module is configured to determine a method fault propagation chain set based on the code data, the first commit record information set, the application logs, and the method call chain set.

[0022] The abnormal call chain determination module is configured to identify an abnormal method call chain in the method call chain set, aggregate the identified abnormal method call chain based on the application logs and an abnormal chain link aggregation condition, and obtain one or more abnormal method call chain sets.

[0023] The fault root cause positioning module is configured to determine fault root cause information based on the method fault propagation chain set and the abnormal method call chain set, the fault root cause information including a suspicious method.

[0024] In a third aspect, the embodiments of the present disclosure further provide an electronic device, including a processor and a memory; the processor is configured to execute the steps of the microservice system fault root cause positioning method according to the first aspect by calling programs or instructions stored in the memory.

[0025] In a fourth aspect, the embodiments of the present disclosure further provide a non-transitory computer-readable storage medium for storing programs or instructions, which enable a computer to execute the steps of the microservice system fault root cause positioning method according to the first aspect.

[0026] As can be seen, in at least one embodiment of the present disclosure, based on the data of the code repository and the data of the monitoring platform, a method fault propagation chain is introduced to represent the propagation track of the fault possibly introduced by a method modification, so as to improve the explainability of the fault root cause positioning result. On the other hand, by preprocessing the data of the monitoring platform, a microservice system runtime abnormal method call chain can be obtained, and then based on the method fault propagation chain and the abnormal method call chain set, fault root cause information including a suspicious method is determined, the fault root cause information is positioned to the method level, in addition, the fault root cause information can further include suspicious commit record information, the suspicious method and the suspicious commit record information are associated, and the fault troubleshooting time in the code change scenario is shortened.

[0027] In at least one embodiment of the present disclosure, the method execution track of each business request is traced in further granularity by combining the method call chain of the microservice system runtime. By comparing the abnormal method in the abnormal method call chain of the microservice system runtime with the method fault propagation chain, a suspicious method set is obtained, and the suspicious rate of each suspicious method is calculated to obtain the suspicious fault root cause. Finally, by analyzing all the Commit information of the suspicious method, a part of the Commit information that may cause the fault is found to associate the suspicious method with the Commit information and shorten the troubleshooting time under the code change scenario. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0029] Figure 1 is an exemplary application scenario provided by an embodiment of the present disclosure;

[0030] Figure 2 is an exemplary block diagram of a fault root cause positioning device provided by an embodiment of the present disclosure;

[0031] Figure 3 is an exemplary block diagram of a fault root cause positioning module provided by an embodiment of the present disclosure;

[0032] Figure 4 is an exemplary block diagram of an electronic device provided by an embodiment of the present disclosure;

[0033] Figure 5 is an exemplary flowchart of a microservice system fault root cause positioning method provided by an embodiment of the present disclosure;

[0034] Figure 6 is an exemplary flowchart of another microservice system fault root cause positioning method provided by an embodiment of the present disclosure;

[0035] Figure 7 is an exemplary flowchart of determining an indirect fault propagation chain set provided by an embodiment of the present disclosure;

[0036] Figure 8 is an exemplary flowchart of determining a direct fault propagation chain set provided by an embodiment of the present disclosure;

[0037] Figure 9 is an exemplary flowchart of determining an abnormal method call chain set provided by an embodiment of the present disclosure;

[0038] Figure 10 is an exemplary flowchart of an output suspicious method provided by an embodiment of the present disclosure;

[0039] Figure 11 is an exemplary flowchart of a suspicious Commit set of an output suspicious method provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the present disclosure will be further described in detail below with reference to the drawings and embodiments. It can be understood that the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. The specific embodiments described herein are only used to explain the present disclosure, but not to limit the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present disclosure.

[0041] It should be noted that, in this paper, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations.

[0042] In order to facilitate the understanding of the scheme of the embodiments of the present disclosure, the terms related to the embodiments of the present disclosure are explained as follows:

[0043] Method dependency: describes the dependency relationship between methods, mainly including direct dependency and indirect dependency. Direct dependency refers to the relationship between different methods that exist between the calling and the called, and these methods are on a method call chain. Indirect dependency refers to the dependency formed by different methods accessing certain shared states (shared states include but are not limited to global variables, caches and databases), such as two methods accessing the same global variable, cache or database, and these methods are not on the same method call chain.

[0044] Fault propagation: most of the faults of the system are introduced by code (method) modification. When the method of a certain service in the system is modified, a fault may be introduced. The fault introduced by the modification may affect other methods that have direct dependency on the method, the method affected by the fault upstream is called fault backward propagation, and the method affected by the fault downstream is called fault forward propagation. In addition, the fault introduced by the modification may affect other methods that have indirect dependency on the method, which is called fault indirect propagation.

[0045] Method fault propagation chain: describes the propagation track of the fault introduced by the method modification, divided into direct dependency fault propagation chain and indirect dependency fault propagation chain, and the head of the chain is the modified method (i.e. the modified method). The direct dependency fault propagation chain includes two types: the propagation chain formed by the forward propagation of the fault and the propagation chain formed by the backward propagation of the fault. The indirect dependency fault propagation chain is mainly the propagation chain formed by the indirect propagation of the fault.

[0046] Method call chain: the traditional call chain is to take microservices as the statistical granularity to count the call relationship between microservices. The method-level distributed call chain further refines the statistical granularity to the method, for example, counts all the methods of different microservices passed through by a call chain (Trace), and reflects the call condition of each method, including whether the call is successful, the fault type when the call fails, etc.

[0047] Commit information: in the process of using the code version control system by the developer, a commit record information is generated by the system every time the code is submitted to the central code repository, which is called Commit information. A file can be modified multiple times, and each submission generates a Commit information. A Commit information can modify multiple files, and each file includes one or more methods.

[0048] Fault root cause: the source or root cause of the fault.

[0049] The following describes six fault root cause positioning schemes in the prior art.

[0050] Fault root cause positioning scheme one

[0051] For monolithic systems, a twice positioning strategy is used to locate the fault. The first positioning may contain the function (i.e. the method) of the fault, and the second positioning locates the fault code line (i.e. the fault statement) in the function. Since there is an essential difference between monolithic systems and microservice systems, the fault root cause positioning scheme of monolithic systems cannot be directly applied to microservice systems.

[0052] Fault root cause positioning scheme two

[0053] For microservice systems, the method is taken as the granularity, and the method call that causes the fault is located by analyzing the difference in execution track. However, this fault root cause positioning scheme only locates the method call that causes the fault, and the difference in the link track may not be the root cause of the fault, so it cannot really locate the fault root cause to the method level (i.e. locate which method called by which service appears fault).

[0054] Fault root cause positioning scheme three

[0055] For microservice system, the abnormal propagation-oriented microservice intelligent monitoring scheme is adopted, but the abnormal propagation chain node is microservice instead of method, so the fault root cause can only be located at service level (or service entry API level), that is, the fault root cause can only be located at a certain microservice, and the fault root cause cannot be really located at method level.

[0056] Fault root cause localization scheme four

[0057] For microservice system, the two parts of offline training and online prediction are adopted. In order to obtain training data, a large number of test cases need to be written to collect a large amount of microservice-level call chain data (i.e. execution trajectory log), and the fault root cause can only be located at service level, and the fault root cause cannot be really located at method level.

[0058] Fault root cause localization scheme five

[0059] For monolithic system, a scheme combining error prevention and resolution techniques is adopted, which is divided into two stages: the first stage includes establishing a metric-based model to assess the risk of Commit in the incoming Git library. The second stage compares the code blocks with suspected risks in the first stage with known historical error submissions and resolution methods. Since there are essential differences between monolithic system and microservice system, the fault root cause localization scheme of monolithic system cannot be directly applied to microservice system, for example, it is often difficult to establish general metric indicators and dynamic threshold range in microservice system.

[0060] Fault root cause localization scheme six

[0061] For microservice system, the root cause localization scheme MicroRCA (Root Cause Localization of Performance Issues in Microservices) for performance fault is adopted, which is mainly used to locate three kinds of abnormalities: response delay, CPU occupation and memory leakage, and can only be located at service level, and the fault root cause cannot be really located at method level.

[0062] The method, device, electronic device and non-transitory computer readable storage medium provided by the embodiments of the present disclosure can locate the fault root cause to the method for the abnormal method call link on the line in the microservice system, and associate the method with the commit information, and filter out the related commit information that is most likely to introduce defects and cause the failure of the microservice system. In at least one embodiment of the present disclosure, the method fault propagation chain is introduced to represent the propagation track of the fault that may be introduced by a method modification, so as to improve the explainability of the fault root cause locating result. In at least one embodiment of the present disclosure, the method execution track of each business request is further traced in a fine-grained manner by combining the method call chain of the microservice system runtime. In at least one embodiment of the present disclosure, the suspicious method set is obtained by comparing the abnormal method in the abnormal method call link of the microservice system runtime with the method fault propagation chain, and the suspicious rate of each suspicious method is calculated, so as to obtain the suspicious fault root cause. In at least one embodiment of the present disclosure, the part of commit information that may cause the fault is found by analyzing all commit information of the suspicious method, so as to associate the suspicious method with the commit information and shorten the troubleshooting time in the code change scenario.

[0063] Compared with the fault root cause locating scheme one and scheme four, a large number of test cases need to be written. The microservice system fault root cause locating scheme provided by the embodiments of the present disclosure avoids writing a large number of test cases by using log information, method call link across services in the microservice system runtime and code static scanning to obtain the method dependency relationship and execution across services.

[0064] Compared with the fault root cause locating scheme one and scheme five, the two schemes are mainly aimed at monolithic systems. Since the microservice system needs higher coverage test cases, and the fault of the microservice system may propagate across services in runtime, it is difficult for the two schemes to be well applied to the microservice system. The microservice system fault root cause locating scheme provided by the embodiments of the present disclosure has good support for the application of the microservice system by using the method call link across services in the microservice system runtime to describe the method dependency relationship and execution across services.

[0065] Compared with the fault root cause locating scheme three and scheme four, the two schemes are mainly aimed at the microservice system, but the fault root cause locating result is at the service level. The microservice system fault root cause locating scheme provided by the embodiments of the present disclosure can locate the fault root cause to the method level by using the method fault propagation chain and the method call chain of the microservice system runtime to describe the dependency relationship and execution of the method.

[0066] Compared with the second fault root cause positioning scheme, the second positioning result is the occurrence point of the fault, not the root cause point of the fault. For example, microservice A calls microservice B to obtain data c, and then calls microservice C based on data c. If the method of microservice B is modified, the obtained data c is incorrect, and then when microservice C is called based on data c, microservice C reports an error. The second scheme can only locate the occurrence point of the fault (microservice C), but cannot locate the fault root cause (microservice B). The microservice system fault root cause positioning scheme provided in the embodiments of the present disclosure uses the cross-service method fault propagation chain to describe the fault root cause and the fault propagation track, so that the located result is the fault root cause.

[0067] Compared with the second fault root cause positioning scheme and the fifth scheme, a large number of historical fault samples need to be accumulated to improve the accuracy of the scheme. The microservice system fault root cause positioning scheme provided in the embodiments of the present disclosure uses the method call chain of the microservice system runtime to represent the cross-service method dependency relationship and execution, and does not need to accumulate a large number of historical fault samples.

[0068] Compared with the fourth fault root cause positioning scheme using machine learning, the result of the fault root cause positioning has poor explainability. The microservice system fault root cause positioning scheme provided in the embodiments of the present disclosure uses the cross-service method fault propagation chain to describe the fault root cause and the fault propagation track, so that each positioning result has high explainability.

[0069] Compared with the first, second, third, fourth and sixth fault root cause positioning schemes, the fault root cause positioning result cannot be associated with the Commit information. The microservice system fault root cause positioning scheme provided in the embodiments of the present disclosure uses log information, a cross-service method fault propagation chain and a method call chain of a microservice system runtime to first locate the fault root cause to the method level, and then analyzes the Commit information of the method to filter out a part of Commit information that may cause the fault, to complete the association of the fault root cause with the Commit information.

[0070] Compared with the sixth fault root cause positioning scheme using a graph, the scheme mainly targets system resource type faults and cannot locate application-level business type faults. The microservice system fault root cause positioning scheme provided in the embodiments of the present disclosure uses a cross-service method fault propagation chain to describe the fault root cause and the fault propagation track, so that the business type method level fault root cause information can be located.

[0071] The embodiments of the present disclosure can be applied to any microservice system, wherein the microservice system can be configured according to different business types, and the configuration of the microservice system belongs to the mature technology in the art, which will not be described here. Since different microservice systems can be used for different businesses, the embodiments of the present disclosure can be applied to different business scenarios, such as an online shopping scenario. It should be understood that the application scenarios of the embodiments of the present disclosure are only some examples or embodiments of the present disclosure, and for those skilled in the art, the present disclosure can also be applied to other similar scenarios without creative labor.

[0072] Figure 1 An exemplary application scenario graph is provided for the embodiments of the present disclosure. As shown in Figure 1 , the application scenario includes a code repository 10, a monitoring platform 11, a fault root cause positioning device 12 and other devices related to the microservice system. Figure 1 It can be understood as a microservice system or a part of a microservice system.

[0073] In some embodiments, a user initiates a business request, for example, clicks the order button when shopping online, which initiates an order request. This business request will be processed by multiple microservices in the microservice system, and each microservice will use one or more methods (which can be understood as functions or codes).

[0074] For example, this order request will be processed by behavior judgment service, inventory service, order service and other microservices, that is, the user clicks the order button will call these microservices. Among them, the behavior judgment service is used to judge whether the user's click this time is legal, and give a prompt information when it is judged to be illegal, for example, the user clicks multiple times in a short time, the behavior judgment service will prompt the user not to order multiple times. The inventory service is used to find out whether the inventory quantity of the product meets the product quantity ordered by the user. The order service is used to generate order information, which includes user ID, product ID, product quantity, price, user address, merchant address, contact information, etc.

[0075] Each microservice will use one or more methods. For web architecture microservices, a layered design is usually adopted, for example, a three-layer design includes a presentation layer, a business logic layer and a data access layer, wherein the presentation layer corresponds to the front-end user interface (UI). Then each layer will correspond to at least one method, for example, the inventory service is a web architecture microservice, which will use the method of the presentation layer to receive the order request, and then use the method of the business logic layer to parse the order request to obtain various information (including product ID), and then use the method of the data access layer to access the product database to obtain the inventory quantity of the product ID.

[0076] In this way, the microservice system forms a microservice execution track (which can be understood as a service-level execution track) for the business request and forms a method execution track (which can be understood as a method-level execution track) for the business request. Accordingly, different business requests correspond to different microservice execution tracks or method execution tracks.

[0077] A code repository 10 is configured to store static data in the microservice system. The static data can include but is not limited to: code data and commit record information set. The code data can be understood as code data of methods involved in different microservices in the microservice system.

[0078] A monitoring platform 11 is configured to monitor dynamic data generated by the microservice system during runtime. The monitoring platform 11 can monitor the method execution track to form a method call chain. The dynamic data can include but is not limited to: a method call chain set and an application log. The method call chain set can be understood as a collection of cross-service method call chains during the runtime of the microservice system, and the method call chain set can be used to represent the direct dependency relationship between methods across services. The application log can include but is not limited to: exception type and stack information of an abnormal method call chain.

[0079] A fault root cause positioning device 12 is configured to locate a fault root cause to a method when a fault occurs in the method execution track to obtain fault root cause positioning information. The fault root cause positioning information includes: a suspicious method, suspicious commit record information, an exception location, and an exception type, thereby associating the suspicious method with the suspicious commit record information (suspicious commit information) and shortening the troubleshooting time under the code change scenario.

[0080] Figure 2 An example block diagram of a fault root cause positioning apparatus 20 provided by the embodiments of the present disclosure is shown. In some embodiments, the fault root cause positioning apparatus 20 can be implemented as the fault root cause positioning device 12 or a part of the fault root cause positioning device 12 in the microservice system 1, and is configured to locate a fault root cause. Figure 1

[0081] As shown in FIG. 2, the fault root cause positioning apparatus 20 can include but is not limited to: a fault propagation chain determination module 21, an abnormal call chain determination module 22, and a fault root cause positioning module 23. Figure 2 The fault propagation chain determination module 21 is configured to determine a fault propagation chain in the method execution track.

[0082]

[0083] ​​The fault propagation chain determination module 21 is configured to determine a method fault propagation chain set based on the data of the code repository and the data of the monitoring platform. In some embodiments, the fault propagation chain determination module 21 constructs the method fault propagation chain set by performing static scanning on the code data of the code repository, analyzing a first commit record information set (i.e., a first commit set) of the code repository, and obtaining a method call chain set of the microservice system during runtime across services, to construct the method fault propagation chain set, which belongs to the method level fault propagation chain, rather than the service level fault propagation chain. The first commit set is a commit set in the code repository, which is not processed and directly obtained from the code repository.

[0084] In some embodiments, the fault propagation chain determination module 21 can perform static scanning on the code data in the code repository to determine the indirect dependency relationship between different methods. The indirect dependency relationship can be understood as the indirect dependency relationship between the methods within the service (rather than across services). The methods with the indirect dependency relationship are usually not on the same method call chain.

[0085] In some embodiments, the fault propagation chain determination module 21 can scan the code repository to determine which methods use the same global variable, and the methods have the indirect dependency relationship.

[0086] In some embodiments, the fault propagation chain determination module 21 can scan the code repository to determine which methods use the same cache tool class to access the same cache, for example, scan which methods use the Jedis class to access Redis, and the methods have the indirect dependency relationship.

[0087] In some embodiments, the fault propagation chain determination module 21 can scan the code repository to determine which methods use the same database tool class to access the same database, for example, which methods under JPA use the same Repository class to access the same database or which methods under Mybatis use the same Mapper class to access the same database, and the methods have the indirect dependency relationship.

[0088] In some embodiments, the fault propagation chain determination module 21 determines the indirect dependency relationship between the methods in a manner similar to the Find Usage function in the IDE. When implemented, the JavaParser and Eclipse JDT tools are used to perform static analysis to construct an AST (abstract syntax tree), and then analyze the shared state (global variable, cache, and database) to determine the indirect dependency relationship between the methods.

[0089] In some embodiments, the fault propagation chain determination module 21 can extract the commit record information of the code repository within a certain time window to obtain a first commit record information set (i.e., a first Commit set). The fault propagation chain determination module 21 analyzes the first Commit set to obtain the method modified by each Commit, thereby determining a modification method set within the time window and a second commit record information set (i.e., a second Commit set) corresponding to each modification method. The modification method set can be understood as a set of modified methods. In some embodiments, the fault propagation chain determination module 21 can also determine the number of code lines of the method modified by each Commit in the second Commit set. Since one modification method corresponds to one second Commit set, each Commit in the second Commit set modifies the same method, and the number of modified code lines can be the same or different.

[0090] In some embodiments, the fault propagation chain determination module 21 can obtain a method call chain set within a certain time window from the monitoring platform, for example, by data extraction or other methods. Regardless of the method, it is a mature technology in the art and will not be described here. In some embodiments, a method call chain (trace) is composed of multiple method call spans.

[0091] In some embodiments, the fault propagation chain determination module 21 can determine a method fault propagation chain set, and the head method (headMethod) of each method fault propagation chain is a modification method. In some embodiments, the fault propagation chain determination module 21 can determine the method fault propagation chain set based on the method call chain set, the indirect dependency relationship between methods, the modification method set, and the second Commit set corresponding to each modification method. Since the method call chain set is a collection of cross-service method call chains, the method fault propagation chain is a collection of cross-service method fault propagation chains.

[0092] For example, a cross-service method call chain is: S1.M1->S2.M2->S3.M3->S3.M4. Where S1, S2, and S3 represent three different microservices, and M1, M2, M3, and M4 represent four methods. S1.M1 represents method 1 in microservice 1, S2.M2 represents method 2 in microservice 2, S3.M3 represents method 3 in microservice 3, and S3.M4 represents method 4 in microservice 3. This cross-service method call chain indicates that S1.M1 calls S2.M2, S2.M2 calls S3.M3, and S3.M3 calls S3.M4. Therefore, when the modification method is S2.M2, its forward fault propagation chain is: S2.M2->S3.M3->S3.M4, which is a cross-service method fault propagation chain; and its reverse fault propagation chain: S2.M2->S1.M1, which is a cross-service method fault propagation chain.

[0093] In some embodiments, the method fault propagation chain set comprises a direct fault propagation chain set, and the fault propagation chain determination module 21 can determine the direct fault propagation chain set. In some embodiments, the fault propagation chain determination module 21 can determine the direct dependency between different methods based on the method call chain set, where the direct dependency can be understood as the direct dependency between methods across services. In some embodiments, the fault propagation chain determination module 21 can determine the direct dependency between different methods based on the normal method call chain set in the method call chain set. In some embodiments, the fault propagation chain determination module 21 can determine the direct fault propagation chain set based on the direct dependency, the modified method set, and the second Commit set corresponding to each modified method. In this embodiment, the fault propagation chain determination module 21 constructs the direct fault propagation chain by looking at which method call chain each modified method appears on. In some embodiments, the fault propagation chain determination module 21 can divide the direct fault propagation chain set into a forward fault propagation chain set and a reverse fault propagation chain set according to the fault propagation direction.

[0094] For example, the method call chain across services is: S1.M1->S2.M2->S3.M3->S3.M4. Where S1, S2 and S3 represent three different microservices, and M1, M2, M3 and M4 represent four methods. S1.M1 represents method 1 in microservice 1, S2.M2 represents method 2 in microservice 2, S3.M3 represents method 3 in microservice 3, and S3.M4 represents method 4 in microservice 3. This method call chain indicates that S1.M1 calls S2.M2, S2.M2 calls S3.M3, and S3.M3 calls S3.M4, that is, this method call chain includes three method call pairs: S1.M1 calls S2.M2, S2.M2 calls S3.M3, and S3.M3 calls S3.M4.

[0095] If the modified method is S2.M2, its forward and reverse fault propagation chains are as follows:

[0096] The forward fault propagation chain is: S2.M2->S3.M3->S3.M4.

[0097] The reverse fault propagation chain is: S2.M2->S1.M1.

[0098] A direct fault propagation chain information can include but is not limited to: the direct fault propagation chain itself, identification information (the identification information is used to express whether the direct fault propagation chain is a forward fault propagation chain or a reverse fault propagation chain), and the ID information of each Commit in the second Commit set of the modified method in the direct fault propagation chain. The direct fault propagation chain set can be understood as a collection of multiple direct fault propagation chain information.

[0099] In some embodiments, the fault propagation chain determination module 21 can determine a set of indirect fault propagation chains. In some embodiments, the fault propagation chain determination module 21 can determine the shared state accessed by each method in the indirect fault propagation chain while determining the set of indirect fault propagation chains. In some embodiments, the fault propagation chain determination module 21 determines the set of indirect fault propagation chains based on the indirect dependency relationship between the methods within the service, the set of modified methods, and the second commit set corresponding to each modified method. In some embodiments, the fault propagation chain determination module 21 can determine the code line at which each method in the indirect fault propagation chain accesses the shared state based on the second commit set corresponding to each modified method.

[0100] In some embodiments, the fault propagation chain determination module 21 determines the set of indirect fault propagation chains in the following manner: for any modified method in the set of modified methods, find one method that has an indirect dependency relationship with the modified method, and form an indirect fault propagation chain with the modified method and the one method that has an indirect dependency relationship with the modified method. There can be multiple indirect fault propagation chains. The indirect fault propagation chain indicates that the fault introduced by the modified method can indirectly affect the method that has an indirect dependency relationship with the modified method.

[0101] For example, Si.SSj represents the shared state j in the microservice i. If there are three methods in the microservice 1, denoted as S1.M1, S1.M2, and S1.M3, and the three methods access a global variable S1.SS1, then the dependency relationship between the three methods is an indirect dependency relationship between the methods within the service.

[0102] If the modified method is S1.M1, there are two indirect fault propagation chains:

[0103] S1.M1->S1.M2, indicating that the head method of the chain is S1.M1, and the fault introduced by S1.M1 can affect S1.M2. At the same time, the shared state accessed by the methods in this indirect fault propagation chain is recorded as the global variable S1.SS1. In addition, the code line at which each method in this indirect fault propagation chain accesses the shared state can also be recorded.

[0104] S1.M1->S1.M3, indicating that the head method of the chain is S1.M1, and the fault introduced by S1.M1 can affect S1.M3. At the same time, the shared state accessed by the methods in this indirect fault propagation chain is recorded as the global variable S1.SS1. In addition, the code line at which each method in this indirect fault propagation chain accesses the shared state can also be recorded.

[0105] The indirect fault propagation chain information can include, but is not limited to, the indirect fault propagation chain itself, the shared state accessed by the methods in the indirect fault propagation chain, and the ID information of each commit in the second commit set of the head method of the chain. The indirect fault propagation chain set can be understood as a collection of multiple pieces of indirect fault propagation chain information.

[0106] The abnormal call chain determination module 22

[0107] The abnormal call chain determination module 22 is configured to preprocess the data of the monitoring platform and determine the abnormal method call chain set. In this embodiment, the data of the monitoring platform includes the method call chain set and the application log, and therefore, the abnormal call chain determination module 22 can preprocess these data, associate the application log with the method call chain set, and aggregate and classify the method call chains of the same type.

[0108] In some embodiments, the abnormal call chain determination module 22 can identify the abnormal method call chain in the method call chain set. In some embodiments, the abnormal call chain determination module 22 identifies the abnormal method call chain based on the call result of the method call chain, where the call result includes the response code and the exception type. In some embodiments, if there is a method with a failed call in a method call chain, the abnormal call chain determination module 22 identifies the method call chain as an abnormal method call chain.

[0109] In some embodiments, the abnormal call chain determination module 22 can aggregate the method call chains of the same type to obtain the number of normal method call chains and abnormal method call chains. In some embodiments, the abnormal call chain determination module 22 can aggregate based on the application log and the link aggregation condition, where the link aggregation condition includes the normal link aggregation condition and the abnormal link aggregation condition. The normal link aggregation condition includes, for example, the normal method call chains that have the same methods and order in the aggregated link. The abnormal link aggregation condition includes, for example, the abnormal method call chains that have the same methods and order in the aggregated link, the same method that throws an exception, and the same exception type, where the same method that throws an exception is, for example, the same method that throws a null pointer exception. In some embodiments, the normal method call chain set obtained by aggregation is used to construct the direct fault propagation chain.

[0110] In some embodiments, the abnormal call chain determination module 22 can associate the application logs with the method call chain set (including the normal method call chain set and the abnormal method call chain set after aggregation) to improve the information of the method call chain, such as the abnormal type and stack information of the abnormal call method chain. In some embodiments, the abnormal call chain determination module 22 can associate the application logs with the method call chain based on the link ID or the like in the application logs, for example, by associating the application logs and the method call chain with the same link identification. In some embodiments, the abnormal call chain determination module 22 can associate the application logs with the method call chain based on the timestamp and the method name, for example, by associating the application logs and the method call chain if the timestamp of the application logs and the timestamp of the method call chain are within a preset time range and the method name in the application logs and the method name in the method call chain are the same, to improve the abnormal information.

[0111] The fault root cause positioning module 23

[0112] The fault root cause positioning module 23 is configured to determine fault root cause information based on the method fault propagation chain set and the abnormal method call chain set, the fault root cause information including a suspicious method. In some embodiments, the fault root cause information further includes suspicious commit record information. In some embodiments, the fault root cause information further includes an abnormal position and an abnormal type.

[0113] In some embodiments, the fault root cause positioning module 23 can perform fault diagnosis on the abnormal method call chain set to obtain a suspicious method set. In some embodiments, the fault root cause positioning module 23 performs fault diagnosis on the abnormal method call chain set based on the method fault propagation chain set and the method call chain set. In some embodiments, the fault root cause positioning module 23 can output multiple suspicious methods as the fault root cause information.

[0114] In some embodiments, after outputting multiple suspicious methods, the fault root cause positioning module 23 performs suspicious analysis on the third commit set corresponding to the suspicious methods to find the most relevant suspicious commit set for the current abnormality. In some embodiments, the fault root cause information can include multiple suspicious methods, multiple suspicious commits, the abnormal position and the abnormal type of each suspicious method.

[0115] In some embodiments, the root cause localization module 23 can determine the location of the anomaly. For direct dependencies, the suspicious method must be on the abnormal method call chain. After obtaining the abnormal method call chain from the associated application logs, the root cause localization module 23 can determine the line of code where the suspicious method throws the exception and the line of code where the last abnormal method on the abnormal method call chain throws the exception, and output these two lines of code as the anomaly location. For indirect dependencies, it is impossible to find a direct abnormal line of code related to the suspicious method and the abnormal call chain. The root cause localization module 23 outputs the lines of code where the suspicious method accesses or modifies shared state variables as the anomaly locations.

[0116] In some embodiments, the root cause localization module 23 can determine the exception type. For direct dependencies, after obtaining the exception method call chain from the associated application logs, the root cause localization module 23 can determine the exception type thrown by the suspicious method and the exception type thrown by the last exception method in the exception method call chain containing the suspicious method, and output these two exception types. For indirect dependencies, the root cause localization module 23 outputs the exception type thrown by the exception method related to the suspicious method and the exception type thrown by the last exception method in the exception method call chain containing the suspicious method. The exception method related to the suspicious method is, for example, an exception method that has an indirect dependency on the suspicious method.

[0117] In some embodiments, the division of units in the fault root cause localization device 20 is only a logical functional division. In actual implementation, there may be other division methods. For example, at least two modules of the fault propagation chain determination module 21, the abnormal call chain determination module 22, and the fault root cause localization module 23 can be implemented as one module; the fault propagation chain determination module 21, the abnormal call chain determination module 22, or the fault root cause localization module 23 can also be divided into multiple units. It is understood that each module or unit can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application.

[0118] Figure 3 This is an exemplary block diagram of a fault root cause localization module 30 provided in an embodiment of the present disclosure. In some embodiments, the fault root cause localization module 30 can be implemented as follows: Figure 2 The fault root cause localization module 23 or a part of the fault root cause localization module 23.

[0119] like Figure 3 As shown, the fault root cause localization module 30 may include, but is not limited to, a fault diagnosis unit 31 and a code comparison unit 32.

[0120] fault diagnosis unit 31

[0121] The fault diagnosis unit 31 can be implemented as a fault diagnosis engine configured to diagnose the abnormal method invocation chain set to obtain a suspicious method set. In some embodiments, the fault diagnosis unit 31 diagnoses the abnormal method invocation chain set based on the method fault propagation chain set and the method invocation chain set.

[0122] In some embodiments, the fault diagnosis unit 31 matches the abnormal method invocation chain set with the method fault propagation chain set (including the direct fault propagation chain set and the indirect fault propagation chain set) to obtain a suspicious fault propagation chain set. For a certain type of abnormal method invocation chain set, one abnormal method invocation chain is selected and matched with the method fault propagation chain set. In some embodiments, the fault diagnosis unit 31 matches the abnormal method in the abnormal method invocation chain with the method fault propagation chain set, and if the abnormal method appears in a method fault propagation chain, the method fault propagation chain is determined as a suspicious fault propagation chain.

[0123] For example, in the abnormal method invocation chain "S1.M1->S2.M2->S3.M3->S3.M4", S2.M2, S3.M3, and S3.M4 throw an exception or have an abnormal response code or fail to be invoked, the abnormal methods in the abnormal method invocation chain are S2.M2, S3.M3, and S3.M4. The fault diagnosis unit 31 matches each abnormal method with the method fault propagation chain set.

[0124] In some embodiments, the fault diagnosis unit 31 filters the suspicious fault propagation chain set to filter out suspicious fault propagation chains irrelevant to the current invocation. The suspicious fault propagation chain set includes the direct fault propagation chain set and the indirect fault propagation chain set, and the fault diagnosis unit 31 retains the indirect fault propagation chain set and filters out the direct fault propagation chain set. The invocation irrelevant can be understood as follows: the suspicious fault propagation chain is a direct fault propagation chain, and the methods before and after the abnormal method in the suspicious fault propagation chain do not appear in the abnormal method invocation chain, and thus the suspicious fault propagation chain is irrelevant to the abnormal method invocation chain.

[0125] For example, in the abnormal method call chain "S1.M1->S2.M2->S3.M3", S2.M2 is the abnormal method, and S3.M3 is the modification method. Then the method fault propagation chain "S3.M3->S2.M2->S1.M1" is matched. At this time, if there is another method fault propagation chain "S4.M4->S2.M2->S5.M5", since S2.M2 is on it, this method fault propagation chain will also be matched out. This method fault propagation chain formed by the direct dependency relationship is needed to be filtered out, because the abnormal method call chain "S1.M1->S2.M2->S3.M3" and the method fault propagation chain "S4.M4->S2.M2->S5.M5" are irrelevant.

[0126] In some embodiments, the fault diagnosis unit 31 extracts the chain head method of each suspicious fault propagation chain from the filtered suspicious fault propagation chain set to obtain a suspicious method set. In some embodiments, if the number of suspicious methods is 0, that is, no modification method is related to the abnormal method call chain; if the number of suspicious methods is 1, that is, only one modification method is related to the abnormal method call chain; if the number of suspicious methods is greater than 1, that is, there are multiple modification methods related to the abnormal method call chain.

[0127] In some embodiments, the fault diagnosis unit 31 can determine the failure rate of each suspicious method in the suspicious method set. In some embodiments, for any suspicious method, the fault diagnosis unit 31 can determine the number of abnormal method call chains in all method call chains (including normal method call chains and abnormal method call chains) passing through the suspicious method, and then determine the number of abnormal method call chains and the number of all method call chains passing through the suspicious method, to determine the failure rate of the suspicious method.

[0128] In some embodiments, the fault diagnosis unit 31 determines one or more method call chains in which each suspicious method is located. In this embodiment, the fault diagnosis unit 31 can construct an inverted index based on the methods on each method call chain, and then determine which method call chains pass through the suspicious method. In some embodiments, the fault diagnosis unit 31 determines the failure rate of the suspicious method by the following formula:

[0129]

[0130] Wherein, S i represents the failure rate of the suspicious method i, T f represents the number of abnormal method call chains passing through the suspicious method i, T t represents the total number of method call chains passing through the suspicious method i.

[0131] For example, there are three suspicious methods: S1.M1, S2.M2, and S3.M3. The method call chains (including normal method call chains and abnormal method call chains, and the abnormal method call chains can be caused by direct propagation or indirect propagation of abnormal methods) related to the above three suspicious methods within the time window are as follows:

[0132] Link number Method of link passing Is abnormal 1 S1.M1->S2.M2->S3.M3 Yes 2 S2.M1->S1.M1->S3.M4->S3.M2 No 3 S3.M1->S2.M2->S3.M3 Yes 4 S4.M1->S2.M2->S3.M3 Yes 5 S5.M1->S5.M2->S4.M2->S3.M3 No 6 S3.M1->S1.M1->S3.M2 No

[0133] The failure rates of the three suspicious methods are as follows:

[0134]

[0135] In some embodiments, the fault diagnosis unit 31 can determine the failure rate of each method on the abnormal method call chain, and the calculation method is similar to that of the suspicious method failure rate, which will not be described again.

[0136] In some embodiments, the fault diagnosis unit 31 can determine the suspicious rate of the suspicious method. In some embodiments, the fault diagnosis unit 31 can determine the total number of code modifications and the number of commits of the suspicious method based on the third commit set corresponding to the suspicious method. The number of commits of the suspicious method can be understood as the number of commits included in the third commit set corresponding to the suspicious method. In some embodiments, the fault diagnosis unit 31 can determine the suspicious rate of the suspicious method based on the failure rate of the suspicious method, the total number of code modifications of the suspicious method, and the number of commits.

[0137] For example, the fault diagnosis unit 31 determines the suspicious rate of the suspicious method by the following formula:

[0138]

[0139] Wherein, F i represents the suspicious rate of the suspicious method i, S i represents the failure rate of the suspicious method i, L i represents the total number of code modifications of the suspicious method i, L t represents the total number of code modifications of the microservice system, C i represents the total number of modifications of the suspicious method i (that is, the number of commits corresponding to the suspicious method i), C t represents the total number of modifications of the microservice system; K l and K c respectively represent the influence factors of the modification area and the modification times of the suspicious method, and the value range is [0, 1]. In some embodiments, K l is 0.7, K c is 0.3, and it should be noted that for different microservice systems, K l and Kc K can be set to different values, and those skilled in the art can set K according to actual needs l and the value of K. c .

[0140] In some embodiments, if the number of suspicious methods is 0, the fault diagnosis unit 31 sorts the failure rates of each method on the abnormal method call chain, and outputs the Top k methods as the fault root cause information, where the Top k methods represent the first k methods sorted from large to small in the failure rate; if the number of suspicious methods is 1, the fault diagnosis unit 31 outputs the method as the fault root cause information; if the number of suspicious methods is greater than 1, the fault diagnosis unit 31 sorts the suspicious methods according to the suspicious rate from large to small, and outputs the Top k suspicious methods as the fault root cause information. i and S i , the fault diagnosis unit 31 sorts the failure rates of each method on the abnormal method call chain, and outputs the Top k methods as the fault root cause information, where the Top k methods represent the first k methods sorted from large to small in the failure rate; if the number of suspicious methods is 1, the fault diagnosis unit 31 outputs the method as the fault root cause information; if the number of suspicious methods is greater than 1, the fault diagnosis unit 31 sorts the suspicious methods according to the suspicious rate from large to small, and outputs the Top k suspicious methods as the fault root cause information.

[0141] The code comparison unit 32

[0142] The code comparison unit 32 can be implemented as a code analysis (Code diff) engine, which is used to perform suspicious analysis on the third Commit set corresponding to the Top k suspicious methods after outputting the Top k suspicious methods.

[0143] In some embodiments, if the number of suspicious methods is 0, the code comparison unit 32 outputs the Top k methods, and the Top k methods belong to the method in the abnormal method call chain set, the code comparison unit 32 performs suspicious analysis on the Commit set corresponding to the Top k methods, and the suspicious analysis is similar to the suspicious analysis of the third Commit set corresponding to the Top k suspicious methods.

[0144] In some embodiments, if the suspicious method is a chain head method directly dependent on the fault propagation chain, the code comparison unit 32 determines the code line where the suspicious method throws an exception based on the abnormal method call chain set associated with the application log; then performs comparison analysis on the third Commit set corresponding to the suspicious method based on the code line, to obtain a suspicious Commit set, wherein the suspicious Commit set includes a Commit that modifies the code corresponding to the code line, and if the code line is a method call, the suspicious Commit set also includes a Commit that modifies the parameters of the method call.

[0145] In some embodiments, if the suspicious method is a chain head method indirectly dependent on the fault propagation chain, the code comparison unit 32 determines the shared state accessed or modified by the suspicious method; then performs comparison analysis on the third Commit set corresponding to the suspicious method, to obtain a suspicious Commit set, wherein the suspicious Commit set includes a suspicious Commit that accesses or modifies the shared state.

[0146] In some embodiments, the code comparison unit 32 determines the number of code modification lines corresponding to each Commit in the suspicious Commit set, and then sorts based on the number of code modification lines and the modification time, and outputs the Top k suspicious Commit. In some embodiments, the more the number of code modification lines, the higher the suspicious Commit is sorted; when the number of code modification lines is the same, the earlier the modification time, the higher the suspicious Commit is sorted.

[0147] In some embodiments, the fault root cause information output by the code comparison unit 32 includes: suspicious methods, suspicious commit record information (i.e. suspicious Commit), abnormal position and abnormal type. Among them, the abnormal position is the code line position where the suspicious method occurs an exception or is related to an exception. In some embodiments, the fault root cause information includes Top k suspicious methods, Top k suspicious Commit, abnormal position and abnormal type of each suspicious method.

[0148] In some embodiments, the division of each unit in the fault root cause positioning module 30 is only a logical functional division, and there can be another division manner in actual implementation, for example, the fault diagnosis unit 31 and the code comparison unit 32 can be implemented as one unit; the fault diagnosis unit 31 or the code comparison unit 32 can also be divided into multiple sub-units. It can be understood that each unit or sub-unit can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software mode depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application.

[0149] Figure 4 is a structural schematic diagram of an electronic device provided by an embodiment of the disclosure. In some embodiments, the electronic device can be implemented as Figure 1 the fault root cause positioning device 12 or part of the fault root cause positioning device 12 in

[0150] As shown in Figure 4 , the electronic device includes at least one processor 41, at least one memory 42 and at least one communication interface 43. Each component in the electronic device is coupled together through a bus system 44. The communication interface 43 is used for information transmission between external devices. It can be understood that the bus system 44 is used to realize the connection communication between these components. In addition to including a data bus, the bus system 44 also includes a power bus, a control bus and a status signal bus. However, in order to clearly illustrate, all kinds of buses are marked as the bus system 44 in Figure 4 .

[0151] It can be understood that the memory 42 in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0152] In some embodiments, the memory 42 stores the following elements, executable units or data structures, or a subset of them, or an extended set of them: an operating system and an application program.

[0153] The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program includes various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The program for implementing the microservice system fault root cause positioning method provided in the embodiments of the present disclosure can be included in the application program.

[0154] In the embodiments of the present disclosure, the processor 41 processes the steps of each embodiment of the microservice system fault root cause positioning method provided in the embodiments of the present disclosure by invoking the programs or instructions stored in the memory 42, specifically, the programs or instructions stored in the application program. The processor 41 is configured to execute the steps of each embodiment of the microservice system fault root cause positioning method provided in the embodiments of the present disclosure.

[0155] The microservice system fault root cause positioning method provided in the embodiments of the present disclosure can be applied to the processor 41 or implemented by the processor 41. The processor 41 can be an integrated circuit chip with a signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 41 or an instruction in the form of software. The processor 41 described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general processor can be a microprocessor or the processor can also be any conventional processor.

[0156] The steps of the microservice system fault root cause positioning method provided in the embodiments of the present disclosure can be directly embodied as a hardware decoding processor for execution, or executed by a combination of hardware and software units in the decoding processor. The software unit can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage media in the art. The storage medium is located in the memory 42, and the processor 41 reads the information in the memory 42 and combines the hardware to complete the steps of the method.

[0157] Figure 5 An exemplary flowchart of a method for locating a root cause of a microservice system failure is provided for the embodiments of the present disclosure. The execution subject of the method is an electronic device. For ease of description, the flow of a method for locating a root cause of a microservice system failure is described below with the electronic device as the execution subject.

[0158] In step 501, the electronic device determines a set of method failure propagation chains based on data of the code repository and data of the monitoring platform. The data of the code repository includes code data and a set of commit record information. The data of the monitoring platform includes a set of method call chains and application logs, wherein the application logs at least include exception types and stack information of an abnormal method call chain. The set of method call chains can be understood as a collection of method call chains across services in the runtime of the microservice system, and this set of method call chains can be used to represent direct dependency relationships between methods across services.

[0159] In some embodiments, the electronic device can perform static scanning on the code data in the code repository to determine indirect dependency relationships between different methods. The indirect dependency relationship can be understood as an indirect dependency relationship between methods within a service (rather than across services). Methods with indirect dependency relationships are usually not on the same method call chain.

[0160] In some embodiments, the electronic device can scan the code repository to determine which methods use the same global variable, and these methods have indirect dependency relationships.

[0161] In some embodiments, the electronic device can scan the code repository to determine which methods use the same cache tool class to access the same cache, for example, scan which methods use the Jedis class to access Redis, and these methods have indirect dependency relationships.

[0162] In some embodiments, the electronic device can scan the code repository to determine which methods use the same database tool class to access the same database, for example, under JPA, which methods use the same Repository class to access the same database, or under Mybatis, which methods use the same Mapper class to access the same database, and these methods have indirect dependency relationships.

[0163] In some embodiments, the electronic device determines the indirect dependency relationships between methods in a manner similar to the Find Usage function in an IDE. When implemented, JavaParser and Eclipse JDT are used for static analysis to construct an AST (Abstract Syntax Tree), and then the shared state (global variables, caches, and databases) is analyzed to determine the indirect dependency relationships between methods.

[0164] In some embodiments, the electronic device can extract commit record information of the code repository within a certain time window to obtain a first commit record information set (i.e., a first Commit set). The electronic device analyzes the first Commit set to obtain the method modified by each Commit, thereby determining a modification method set within the time window and a second commit record information set (i.e., a second Commit set) corresponding to each modification method. The modification method set can be understood as a set of modified methods. In some embodiments, the electronic device can also determine the number of code lines of the method modified by each Commit in the second Commit set. Since one modification method corresponds to one second Commit set, each Commit in the second Commit set modifies the same method, and the number of modified code lines can be the same or different.

[0165] In some embodiments, the electronic device can obtain a method call chain set within a certain time window from the monitoring platform, for example, by data extraction or other methods. Regardless of the method, it is a mature technology in the art and will not be described here. In some embodiments, a method call chain (trace) is composed of multiple method call spans.

[0166] In some embodiments, the electronic device can determine a method fault propagation chain set, and the head method (headMethod) of each method fault propagation chain is a modification method. In some embodiments, the electronic device can determine the method fault propagation chain set based on the method call chain set, the indirect dependency relationship between methods, the modification method set, and the second Commit set corresponding to each modification method. Since the method call chain set is a set of cross-service method call chains, the method fault propagation chain is a set of cross-service method fault propagation chains.

[0167] In some embodiments, the method fault propagation chain set includes a direct fault propagation chain set, and the electronic device can determine the direct fault propagation chain set. In some embodiments, the electronic device can determine the direct dependency relationship between different methods based on the method call chain set, where the direct dependency relationship can be understood as the direct dependency relationship between cross-service methods. In some embodiments, the electronic device determines the direct dependency relationship between different methods based on the normal method call chain set in the method call chain set. In some embodiments, the electronic device can determine the direct fault propagation chain set based on the direct dependency relationship, the modification method set, and the second Commit set corresponding to each modification method. In this embodiment, the electronic device constructs the direct fault propagation chain by checking in which method call chains each modification method appears. In some embodiments, the electronic device can divide the direct fault propagation chain set into a forward fault propagation chain set and a reverse fault propagation chain set according to the fault propagation direction.

[0168] For example, the cross-service method call chain is: S1.M1 -> S2.M2 -> S3.M3 -> S3.M4. Wherein, S1, S2 and S3 represent three different microservices, and M1, M2, M3 and M4 represent four methods. S1.M1 represents method 1 in microservice 1, S2.M2 represents method 2 in microservice 2, S3.M3 represents method 3 in microservice 3, and S3.M4 represents method 4 in microservice 3. The method call chain indicates that S1.M1 calls S2.M2, S2.M2 calls S3.M3, and S3.M3 calls S3.M4, that is, the method call chain includes three method call pairs: S1.M1 calls S2.M2, S2.M2 calls S3.M3, and S3.M3 calls S3.M4.

[0169] If the modified method is S2.M2, the forward and reverse fault propagation chains are as follows:

[0170] The forward fault propagation chain is: S2.M2 -> S3.M3 -> S3.M4.

[0171] The reverse fault propagation chain is: S2.M2 -> S1.M1.

[0172] In some embodiments, the electronic device can determine a set of indirect fault propagation chains. In some embodiments, the electronic device can determine the shared states accessed by the methods in each indirect fault propagation chain while determining the set of indirect fault propagation chains. In some embodiments, the electronic device determines the set of indirect fault propagation chains based on the indirect dependency relationship between the methods within a service, the set of modified methods, and the second commit set corresponding to each modified method. In some embodiments, the electronic device can determine the code line (codeLine) at which each method accesses the shared state based on the second commit set corresponding to each modified method.

[0173] For example, Si.SSj represents a shared state j in microservice i. If there are three methods in microservice 1, denoted as S1.M1, S1.M2 and S1.M3, and the three methods access a global variable S1.SS1, the dependency relationship between the three methods is an indirect dependency relationship between methods within a service.

[0174] If the modified method is S1.M1, there are two indirect fault propagation chains:

[0175] S1.M1 -> S1.M2 indicates that the head method of the chain is S1.M1, and the fault introduced by S1.M1 will affect S1.M2. Meanwhile, it is recorded that the shared state accessed by the methods in this indirect fault propagation chain is the global variable S1.SS1. In addition, the code line at which each method accesses the shared state in this indirect fault propagation chain can also be recorded.

[0176] S1.M1->S1.M3, which means that the chain head method is S1.M1, and the fault introduced by S1.M1 affects method S1.M3. The shared state accessed by the methods in this indirect fault propagation chain is recorded as global variable S1.SS1. In addition, the code line where each method accesses the shared state in this indirect fault propagation chain can also be recorded.

[0177] In step 502, the electronic device can preprocess the data of the monitoring platform to determine a set of abnormal method invocation chains. In this embodiment, the data of the monitoring platform includes a set of method invocation chains and application logs, and thus the electronic device can preprocess these data, associate the application logs with the set of method invocation chains, and aggregate and classify method invocation chains of the same type.

[0178] In some embodiments, the electronic device can identify abnormal method invocation chains in the set of method invocation chains. In some embodiments, the electronic device identifies abnormal method invocation chains based on the invocation results of the method invocation chains, where the invocation results include response codes and exception types. In some embodiments, if a method invocation chain includes a method that fails to be invoked, the electronic device identifies this method invocation chain as an abnormal method invocation chain.

[0179] In some embodiments, the electronic device can aggregate method invocation chains of the same type to obtain the number of normal method invocation chains and abnormal method invocation chains. In some embodiments, the electronic device can aggregate based on application logs and link aggregation conditions, where the link aggregation conditions include normal link aggregation conditions and abnormal link aggregation conditions. The normal link aggregation conditions include, for example, normal method invocation chains that have the same methods and order in the aggregation link. The abnormal link aggregation conditions include, for example, abnormal method invocation chains that have the same methods and order in the aggregation link, the same method that throws an exception, and the same exception type, where the same method that throws an exception is, for example, a method that throws a null pointer exception. In some embodiments, the set of normal method invocation chains obtained by aggregation is used to construct direct fault propagation chains.

[0180] In some embodiments, the electronic device can associate the application log with the method call chain set (including the normal method call chain set and the abnormal method call chain set after aggregation) to improve the information of the method call chain, such as the exception type and stack information of the abnormal method call chain. In some embodiments, the electronic device can associate the application log with the method call chain based on the link ID and the like in the application log, for example, by associating the application log and the method call chain with the same link identification. In some embodiments, the electronic device can associate the application log with the method call chain based on the timestamp and the method name, for example, by associating the application log and the method call chain if the timestamp of the application log and the timestamp of the method call chain are within a preset time range and the method name in the application log is the same as the method name in the method call chain.

[0181] In step 503, the electronic device can determine the fault root cause information based on the method fault propagation chain set and the abnormal method call chain set, the fault root cause information including the suspicious method. In some embodiments, the fault root cause information further includes the suspicious commit record information. In some embodiments, the fault root cause information further includes the abnormal position and the abnormal type.

[0182] In some embodiments, the electronic device can perform fault diagnosis on the abnormal method call chain set to obtain a suspicious method set. In some embodiments, the electronic device performs fault diagnosis on the abnormal method call chain set based on the method fault propagation chain set and the method call chain set.

[0183] In some embodiments, the electronic device matches the abnormal method call chain set with the method fault propagation chain set (including the direct fault propagation chain set and the indirect fault propagation chain set) to obtain a suspicious fault propagation chain set. For a certain type of abnormal method call chain set, one abnormal method call chain is selected and matched with the method fault propagation chain set. In some embodiments, the electronic device matches the abnormal method on the abnormal method call chain with the method fault propagation chain set, and if the abnormal method appears on a certain method fault propagation chain, the method fault propagation chain is determined as a suspicious fault propagation chain.

[0184] For example, in the abnormal method call chain “S1.M1->S2.M2->S3.M3->S3.M4”, S2.M2, S3.M3, and S3.M4 throw an exception or have an abnormal response code or fail to call, and the abnormal methods on this abnormal method call chain are S2.M2, S3.M3, and S3.M4. The electronic device matches each abnormal method with the method fault propagation chain set.

[0185] In some embodiments, the electronic device filters the suspicious fault propagation chain set, and filters out suspicious fault propagation chains irrelevant to the current call. The suspicious fault propagation chain set includes the direct fault propagation chain set and the indirect fault propagation chain set. The electronic device retains the indirect fault propagation chain set and filters out the direct fault propagation chain set. The call can be understood as irrelevant in that the suspicious fault propagation chain is a direct fault propagation chain, and the methods before and after the abnormal method on the suspicious fault propagation chain are not present in the abnormal method call chain. Therefore, the suspicious fault propagation chain is irrelevant to the abnormal method call chain.

[0186] For example, in the abnormal method call chain "S1.M1->S2.M2->S3.M3", S2.M2 is an abnormal method, and S3.M3 is a modification method. The method fault propagation chain "S3.M3->S2.M2->S1.M1" is matched. At this time, if there is another method fault propagation chain "S4.M4->S2.M2->S5.M5", since S2.M2 is on it, this method fault propagation chain will also be matched out. This method fault propagation chain formed by the direct dependency relationship is needed to be filtered out, because the abnormal method call chain "S1.M1->S2.M2->S3.M3" of this time is irrelevant to the method fault propagation chain "S4.M4->S2.M2->S5.M5".

[0187] In some embodiments, the electronic device extracts the chain head method of each suspicious fault propagation chain from the filtered suspicious fault propagation chain set to obtain a suspicious method set. In some embodiments, if the number of suspicious methods is 0, that is, no modification method is related to the abnormal method call chain; if the number of suspicious methods is 1, that is, only one modification method is related to the abnormal method call chain; if the number of suspicious methods is greater than 1, that is, there are multiple modification methods related to the abnormal method call chain.

[0188] In some embodiments, the electronic device can determine the failure rate of each suspicious method in the suspicious method set. In some embodiments, for any suspicious method, the electronic device can determine the number of abnormal method call chains in all method call chains (including normal method call chains and abnormal method call chains) passing through the suspicious method, and then determine the number of abnormal method call chains and the number of all method call chains passing through the suspicious method, and determine the failure rate of the suspicious method.

[0189] In some embodiments, the electronic device determines one or more method call chains in which each suspicious method is located. In this embodiment, the electronic device can construct an inverted index based on the methods on each method call chain, and then determine which method call chains pass through the suspicious method. In some embodiments, the electronic device determines the failure rate of the suspicious method by the following formula:

[0190]

[0191] wherein S i denotes the failure rate of suspicious method i, T f denotes the number of abnormal method invocation chains passing through suspicious method i, T t denotes the total number of method invocation chains passing through suspicious method i.

[0192] For example, there are three suspicious methods: S1.M1, S2.M2, S3.M3. The method invocation chains (including normal method invocation chains and abnormal method invocation chains, and the abnormal method invocation chains can be caused by direct propagation or indirect propagation of abnormal methods) related to the above three suspicious methods within a time window are as follows:

[0193] Link number Method of link passing Is abnormal 1 S1.M1->S2.M2->S3.M3 Yes 2 S2.M1->S1.M1->S3.M4->S3.M2 No 3 S3.M1->S2.M2->S3.M3 Yes 4 S4.M1->S2.M2->S3.M3 Yes 5 S5.M1->S5.M2->S4.M2->S3.M3 No 6 S3.M1->S1.M1->S3.M2 No

[0194] The failure rates passing through the three suspicious methods are as follows:

[0195]

[0196] In some embodiments, the electronic device can determine the failure rate of each method on the abnormal method invocation chain, and the calculation method is similar to that of the suspicious method failure rate, which will not be described again.

[0197] In some embodiments, the electronic device can determine the suspicious rate of the suspicious method. In some embodiments, the electronic device can determine the total number of code modifications and the number of commits of the suspicious method based on the third commit set corresponding to the suspicious method. Wherein, the number of commits of the suspicious method can be understood as the number of commits included in the third commit set corresponding to the suspicious method. In some embodiments, the electronic device can determine the suspicious rate of the suspicious method based on the failure rate of the suspicious method, the total number of code modifications of the suspicious method, and the number of commits.

[0198] For example, the electronic device determines the suspicious rate of the suspicious method by the following formula:

[0199]

[0200] wherein F i denotes the suspicious rate of suspicious method i, S i denotes the failure rate of suspicious method i, L i denotes the total number of code modifications of suspicious method i, L t denotes the total number of code modifications of the microservice system, C i denotes the total number of modifications of suspicious method i (i.e., the number of commits corresponding to suspicious method i), C t denotes the total number of modifications of the microservice system; K l and Kc These represent the influence factors of the modification area and the number of modifications of the questionable method, respectively, with values ​​ranging from [0,1]. In some embodiments, K l K is 0.7 c The value is 0.3. It should be noted that K varies depending on the specific microservice system. l and K c Different values ​​can be set, and those skilled in the art can set K according to actual needs. l and K c The value of .

[0201] In some embodiments, if the number of suspicious methods is 0, then F i With S i Similarly, the electronic device sorts the failure rates of each method in the abnormal method call chain and outputs the Top k methods as the root cause information of the failure. The Top k methods represent the top k methods with the failure rates sorted from largest to smallest. If the number of suspicious methods is 1, the electronic device outputs that method as the root cause information of the failure. If the number of suspicious methods is greater than 1, the electronic device sorts the suspicious methods according to their suspicion rates from largest to smallest and outputs the Top k suspicious methods as the root cause information of the failure.

[0202] In some embodiments, after outputting the Top k suspicious methods, the electronic device performs a suspicious analysis on the third commit set corresponding to these suspicious methods. In some embodiments, if the number of suspicious methods is 0, and the electronic device outputs the Top k methods, which belong to the abnormal method call chain set, then the electronic device performs a suspicious analysis on the commit set corresponding to these Top k methods, and its suspicious analysis is similar to the suspicious analysis of the third commit set corresponding to the Top k suspicious methods.

[0203] In some embodiments, if the suspected method is the head method of a fault propagation chain that directly depends on the fault propagation chain, the electronic device determines the line of code in which the suspected method throws an exception based on the abnormal method call chain set associated with the application log; then, it performs a comparative analysis on the third commit set corresponding to the suspected method based on the line of code to obtain a suspected commit set, wherein the suspected commit set includes commits that modify the code corresponding to the line of code, and if the line of code is a method call, the suspected commit set also includes commits that modify the parameters of the method call.

[0204] In some embodiments, if the suspicious method is a chain head method of an indirect dependency fault propagation chain, the electronic device determines a shared state accessed or modified by the suspicious method; and then performs comparison analysis on a third Commit set corresponding to the suspicious method to obtain a suspicious Commit set, wherein the suspicious Commit set includes suspicious Commits that access or modify the shared state.

[0205] In some embodiments, the electronic device determines the number of code modification lines corresponding to each Commit in the suspicious Commit set, and then sorts the suspicious Commits based on the number of code modification lines and the modification time, and outputs Top k suspicious Commits. In some embodiments, the more the number of code modification lines, the higher the suspicious Commit is sorted; when the number of code modification lines is the same, the earlier the modification time, the higher the suspicious Commit is sorted.

[0206] In some embodiments, the fault root cause information output by the electronic device includes: a suspicious method, suspicious commit record information (i.e., a suspicious Commit), an abnormal position, and an abnormal type. The abnormal position is the code line position of the suspicious method where the exception occurs or is related to the exception. In some embodiments, the fault root cause information includes Top k suspicious methods, Top k suspicious Commits, the abnormal position, and the abnormal type of each suspicious method.

[0207] In some embodiments, the electronic device can determine the abnormal position. For a direct dependency relationship, the suspicious method must be on the abnormal method call chain, and after the electronic device obtains the abnormal method call chain of the associated application log, the electronic device can determine the code line where the suspicious method throws an exception and the code line where the last exception method on the abnormal method call chain throws an exception, and outputs the two abnormal code line positions as the abnormal position. For an indirect dependency relationship, the electronic device cannot find the direct abnormal code line related to the suspicious method and the abnormal call chain, and the electronic device outputs the code line position of the shared state variable accessed or modified by the suspicious method as the abnormal position.

[0208] In some embodiments, the electronic device can determine the abnormal type. For a direct dependency relationship, after the electronic device obtains the abnormal method call chain of the associated application log, the electronic device can determine the exception type thrown by the suspicious method and the exception type thrown by the last exception method on the abnormal method call chain, and outputs the two exception types. For an indirect dependency relationship, the electronic device outputs the exception type thrown by the abnormal method related to the suspicious method and the exception type thrown by the last exception method on the abnormal method call chain. The abnormal method related to the suspicious method is, for example, an abnormal method having an indirect dependency relationship with the suspicious method.

[0209] As can be seen, in at least one embodiment of the present disclosure, the direct dependency relationship between methods is described by using the method-level call chain of the microservice system runtime across services to construct a direct fault propagation chain, and the indirect dependency relationship between methods is described by using the shared state access situation obtained by code static scanning to construct an indirect fault propagation chain. In some embodiments, if real-time fault root cause positioning is required online, and the method-level call chain data of the microservice system runtime across services is not much, the direct dependency relationship between methods can also be obtained by code static scanning to construct a direct fault propagation chain, but a part of the method call chain obtained by code static scanning may never be passed through in the microservice system runtime.

[0210] Figure 6 FIG. 1 is an exemplary flowchart of a microservice system fault root cause positioning method provided by an embodiment of the present disclosure. The execution subject of the method is an electronic device. For ease of description, the execution subject is omitted. The method comprises the following steps 601 to 608:

[0211] In step 601, the code data in the code repository is statically scanned to determine the indirect dependency relationship between different methods.

[0212] In step 602, the methods modified by each Commit in the first Commit set in the code repository are analyzed to determine the modified method set and the second Commit set corresponding to each modified method.

[0213] In step 603, the application logs and the method call chain set are obtained by data extraction from the monitoring platform.

[0214] In step 604, the application logs and the method call chain set are preprocessed to obtain the abnormal method call chain set associated with the application logs.

[0215] In step 605, the method call chain set, the indirect dependency relationship, the modified method set, and the second Commit set corresponding to each modified method are integrated to obtain the method fault propagation chain set.

[0216] In step 606, the abnormal method call chain set associated with the application logs and the method fault propagation chain set are subjected to fault diagnosis to obtain the suspicious method set.

[0217] In step 607, the suspicious method set, the modified method set, and the second Commit set corresponding to each modified method are subjected to code comparison (i.e., Code diff) to obtain the suspicious Commit set.

[0218] In step 608, the abnormal method call chain set associated with the application log, the suspicious method set and the suspicious Commit set are data integrated to obtain fault root cause information, which can include: suspicious method, suspicious commit record information, abnormal position and abnormal type.

[0219] It should be noted that, Figure 6 The microservice system fault root cause positioning method shown is substantially the same as the method shown in Figure 5 There is only a difference in the wording, Figure 6 The detailed implementation of each step in the foregoing embodiment can refer to the related embodiments of the foregoing embodiment, and will not be described here. Figure 5

[0220] Figure 7 An example flowchart for determining an indirect fault propagation chain set is provided in the embodiment of the present disclosure, the execution subject of the method is an electronic device, and the execution subject is omitted for ease of description, and the method includes the following steps 701 to 702:

[0221] In step 701, the method matching is performed based on the indirect dependency relationship to determine a plurality of methods having an indirect dependency relationship.

[0222] In step 702, the approach of indirect propagation is determined based on the modification method set and the second Commit set corresponding to each modification method.

[0223] The indirect fault propagation chain set can be determined from the results of steps 701 and 702.

[0224] It should be noted that, Figure 7 The determination of the indirect fault propagation chain set shown is substantially the same as the determination of the indirect fault propagation chain set in Figure 5 The embodiments related to the indirect fault propagation chain set are substantially the same, only the wording is different, Figure 7 The detailed implementation of each step in the foregoing embodiment can refer to the related embodiments of the foregoing embodiment, and will not be described here. Figure 5

[0225] Figure 8 An example flowchart for determining a direct fault propagation chain set is provided in the embodiment of the present disclosure, the execution subject of the method is an electronic device, and the execution subject is omitted for ease of description, and the method includes the following steps 801 to 804:

[0226] In step 801, the method call chain set is data extracted to obtain a direct dependency relationship.

[0227] In step 802, the method matching is performed based on the direct dependency relationship, the modification method set and the second Commit set corresponding to each modification method.

[0228] ​​In step 803, a forward propagation track is determined to obtain a forward fault propagation chain set.

[0229] In step 804, a reverse propagation track is determined to obtain a reverse fault propagation chain set.

[0230] It should be noted that, Figure 8 the determination of the direct fault propagation chain set and Figure 5 the embodiments related to the direct fault propagation chain set are substantially the same, only different in the textual description, Figure 8 the detailed implementation of each step in the above embodiment can refer to the related embodiments of Figure 5 and will not be described here.

[0231] Figure 9 is an exemplary flowchart of a method for determining an abnormal method call chain set provided by an embodiment of the present disclosure. The execution subject of the method is an electronic device. For ease of description, the execution subject is omitted. The method includes the following steps 901 and 902.

[0232] In step 901, the method call chain set is identified for abnormal method call chains and the same type of method call chains is aggregated and classified to obtain how many normal method call chains and abnormal method call chains.

[0233] In step 902, the application log and the aggregated abnormal method call chain set are integrated to obtain an abnormal method call chain set associated with the application log.

[0234] It should be noted that, Figure 9 the determination of the abnormal method call chain set and Figure 5 the embodiments related to the abnormal method call chain set are substantially the same, only different in the textual description, Figure 9 the detailed implementation of each step in the above embodiment can refer to the related embodiments of Figure 5 and will not be described here.

[0235] Figure 10 is an exemplary flowchart of a method for outputting a suspicious method provided by an embodiment of the present disclosure. The execution subject of the method is an electronic device. For ease of description, the execution subject is omitted. The method includes the following steps 101 and 104.

[0236] In step 101, the abnormal method call chain set in the method call chain set is matched with the method fault propagation chain set to obtain a suspicious fault propagation chain set.

[0237] In step 102, the suspicious fault propagation chain set is filtered and the chain head method is extracted to obtain a suspicious method set.

[0238] In step 103, based on the normal method call chain set and the abnormal method call chain set in the method call chain set, the failure rate of each suspicious method in the suspicious method set and the failure rate of each method in the abnormal method call chain set are determined.

[0239] In step 104, based on the failure rate of each suspicious method in the suspicious method set and the third Commit set corresponding to each suspicious method in the suspicious method set, the suspicious rate of each suspicious method is calculated, and the Top k suspicious methods are output based on the suspicious rate.

[0240] It should be noted that, Figure 10 The output suspicious method shown in the Figure 5 The embodiments related to the output suspicious method are substantially the same as the embodiments related to the output suspicious method, only different in the wording, Figure 10 The detailed implementation of each step in the Figure 5 The related embodiments are not repeated here.

[0241] Figure 11 is an exemplary flowchart of the suspicious Commit set of the output suspicious method provided by the embodiments of the present disclosure. The execution subject of the method is an electronic device. For ease of description, the execution subject is omitted. The method includes the following steps 111 and 113:

[0242] In step 111, the abnormal method call chain set associated with the application log is matched with the suspicious method (the suspicious method is the chain head method directly dependent on the fault propagation chain), and the code line where the suspicious method throws an exception is obtained.

[0243] In step 112, the indirect fault propagation chain is matched with the suspicious method (the suspicious method is the chain head method indirectly dependent on the fault propagation chain), and the code line where the suspicious method accesses or modifies the shared state is obtained.

[0244] In step 113, based on the code line where the suspicious method throws an exception, the code line where the suspicious method accesses or modifies the shared state, and the third Commit set corresponding to the suspicious method, the modification related to the code line is matched, and the Top k suspicious Commit is output.

[0245] It should be noted that, Figure 11 The suspicious Commit set of the output suspicious method shown in the Figure 5 The embodiments related to the suspicious Commit set are substantially the same as the embodiments related to the suspicious Commit set, only different in the wording, Figure 11 The detailed implementation of each step in the Figure 5 The related embodiments are not repeated here.

[0246] After determining the fault root cause information based on the above embodiments, an alarm message is further displayed, and the alarm message includes the fault root cause information.

[0247] For example, the alarm message is an email. Specifically, after determining the fault root cause information, an alarm email can be sent to a preset mailbox of the developer, and the fault root cause information is added in the text or picture of the body or the attachment of the alarm email, or the alarm email carries a fault root cause link, and the fault root cause page is popped up after the fault root cause link is clicked, and the fault root cause information is displayed in the page.

[0248] For another example, the alarm message is an alarm interface. Specifically, after determining the fault root cause information, an alarm interface for chatting with the developer can be established through a social software, and the fault root cause information or the fault root cause link is displayed in the interface, and the fault root cause page is popped up after the fault root cause link is clicked, and the fault root cause information is displayed in the page.

[0249] When the fault root cause information is displayed in the page, the fault root cause information can be displayed in the form of a fault list, or in the form of a diagram, or in other forms, and a person skilled in the art can arrange the display manner of the fault root cause information in the page according to actual needs.

[0250] The displayed fault root cause information can include but is not limited to suspicious methods, suspicious submission record information corresponding to the suspicious methods, abnormal code line positions of the suspicious methods, and abnormal types of the suspicious methods. While the fault root cause information is displayed, a link of a specific fault can also be displayed, and the link of the specific fault is clicked to jump to the position of the specific fault (i.e., the suspicious method).

[0251] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but a person skilled in the art can understand that the disclosure embodiments are not limited by the action sequence described, because according to the disclosure embodiments, certain steps can be performed in other sequences or simultaneously. In addition, a person skilled in the art can understand that the embodiments described in the specification all belong to optional embodiments.

[0252] The disclosure embodiments also propose a non-transitory computer readable storage medium, which stores programs or instructions, and the programs or instructions make the computer execute the steps of the microservice system fault root cause positioning method embodiments. To avoid repeated description, it will not be described here.

[0253] It has to be noted that, as used herein, the terms "includes", "including", "to include", "includes" or "including" and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without further recitation, preclude the inclusion of additional elements of the same type or a different type.

[0254] Those skilled in the art will appreciate that the features of the various embodiments can be combined with each other, where appropriate, in accordance with the disclosure and form different embodiments.

[0255] Those skilled in the art will appreciate that the description of various embodiments have been with emphasis on the principles of the disclosure and that modifications and variations of the specific embodiments described can be effected therein by persons of ordinary skill in the art.

[0256] While the embodiments of the disclosure have been described in connection with the preferred embodiments thereof, it will occur to those skilled in the art that modifications and variations can be made by them in light of this disclosure and that other embodiments can be utilized that fall within the spirit and scope of the present disclosure.

Claims

1. A method for locating the root cause of failures in a microservice system, characterized in that, include: Retrieve code data, first commit record information set, application logs, and method call chain set; Based on the code data, the first commit record information set, and the method call chain set, determine the method fault propagation chain set; Identify abnormal method call chains in the method call chain set; Based on the application logs and the abnormal link aggregation conditions, the identified abnormal method call chains are aggregated to obtain one or more abnormal method call chain sets. The abnormal link aggregation conditions include: the abnormal method call chains that are aggregated in the same way as the methods and order in which they pass, the methods that throw the exceptions, and the exception types. Based on the fault propagation chain set and the abnormal method call chain set, the fault root cause information is determined, and the fault root cause information includes suspicious methods. Specifically, determining the method failure propagation chain set based on the code data, the first commit record information set, and the method call chain set includes: performing a static scan on the code data to determine the indirect dependencies between methods; determining the modified method set and the second commit record information set corresponding to each modified method based on the first commit record information set; and determining the method failure propagation chain set based on the method call chain set, the indirect dependencies, the modified method set, and the second commit record information set.

2. The method according to claim 1, characterized in that, The code data and the first commit record information set are derived from the code repository; The data from the monitoring platform includes: the application logs and the method call chain set, which originate from the monitoring platform; the microservice system includes the code repository and the monitoring platform.

3. The method according to claim 1, characterized in that, The static scanning of the code data to determine the indirect dependencies between methods includes: A static scan of the shared state of the code data access is performed to determine that different methods accessing the same shared state have the indirect dependency relationship.

4. The method according to claim 3, characterized in that, The shared state includes: global variables, cache, and database.

5. The method according to claim 1, characterized in that, The method's fault propagation chain set includes a direct fault propagation chain set and an indirect fault propagation chain set.

6. The method according to claim 5, characterized in that, The methods for determining the direct fault propagation chain set include: Based on the method call chain set, determine the direct dependencies between methods; Based on the direct dependencies, the set of modification methods, and the second set of commit record information, the set of direct failure propagation chains is determined.

7. The method according to claim 6, characterized in that, The direct fault propagation chain set includes: a forward fault propagation chain set and a reverse fault propagation chain set.

8. The method according to claim 5, characterized in that, The methods for determining the indirect fault propagation chain set include: Based on the indirect dependencies, the set of modification methods, and the second set of commit record information, the set of indirect fault propagation chains is determined.

9. The method according to claim 1, characterized in that, After obtaining one or more sets of exception method call chains, the method further includes: Associate the application logs with the exception method call chain.

10. The method according to claim 9, characterized in that, The identification of abnormal method call chains in the method call chain set includes: The method call chain that includes an exception identifier and / or exception type in the call result is identified as the exception method call chain; and / or, The method call chain that has failed is identified as the abnormal method call chain.

11. The method according to claim 9, characterized in that, The method further includes: aggregating normal method call chains based on normal link aggregation conditions to obtain one or more sets of normal method call chains; The normal link aggregation condition includes: the normal method call chain through which the aggregated link passes in the same method and in the same order.

12. The method according to claim 11, characterized in that, The normal method call chain is used at least to determine the direct dependencies between methods.

13. The method according to claim 9, characterized in that, Associating the application logs with the abnormal method call chain includes: Associate the application logs and the exception method call chains that have the same link identifier; And / or, The application logs are associated with the abnormal method call chain based on the timestamp and method name.

14. The method according to claim 5, characterized in that, The determination of root cause information based on the method fault propagation chain set and the abnormal method call chain set includes: Based on the fault propagation chain set and the abnormal method call chain set, a set of suspicious methods and a set of third commit record information corresponding to each suspicious method are determined. Based on the method call chain set, the suspicious method set, and the third submission record information set, at least one suspicious method in the suspicious method set is selected as the root cause of the failure.

15. The method according to claim 14, characterized in that, The set of suspicious methods and the third submission record information set corresponding to each suspicious method include: Based on the abnormal method call chain set, the method fault propagation chain set is matched to obtain the suspected fault propagation chain set; The suspicious fault propagation chain set is filtered out if it is not related to the abnormal method call chain set. For the set of suspicious fault propagation chains obtained after the filtering operation, the head method of each suspicious fault propagation chain is extracted as the suspicious method, thus obtaining the set of suspicious methods and the set of third submission record information.

16. The method according to claim 15, characterized in that, The step of matching the abnormal method call chain set with the method failure propagation chain set to obtain the suspected failure propagation chain set includes: If the abnormal method in the abnormal method call chain is a method in the method failure propagation chain, then the method failure propagation chain is determined to be the suspected failure propagation chain.

17. The method according to claim 15, characterized in that, The suspicious fault propagation chain that is not related to the abnormal method call chain set includes: The suspected fault propagation chain is a direct fault propagation chain, and the methods before and after the abnormal method on the suspected fault propagation chain do not appear on the abnormal method call chain. Therefore, the suspected fault propagation chain is unrelated to the abnormal method call chain.

18. The method according to claim 14, characterized in that, The step of filtering at least one suspicious method from the suspicious method set as the root cause of the failure, based on the method call chain set, the suspicious method set, and the third submission record information set, includes: Based on the method call chain set, determine the failure rate of each suspected method; Based on the third commit record information set, determine the total number of lines of code modification and the number of third commit messages for each suspicious method; The suspicion rate of each suspicious method is determined based on the failure rate of each suspicious method, the total number of lines of code modification for each suspicious method, and the number of third commit messages. Based on the suspicion rate of each suspicious method, at least one suspicious method in the suspicious method set is selected as the root cause of the failure.

19. The method according to claim 18, characterized in that, The determination of the failure rate of each suspicious method based on the method call chain set includes: For each suspicious method, determine the number of abnormal method call chains in all method call chains that pass through the suspicious method; The failure rate of the suspicious method is determined based on the number of abnormal method call chains and the total number of all method call chains that pass through the suspicious method.

20. The method according to claim 18, characterized in that, The step of screening at least one suspicious method from the set of suspicious methods as the root cause of the failure based on the suspicion rate of each suspicious method includes: If the number of suspicious methods is 0, then sort the failure rates of each method in the abnormal method call chain and output the Topk methods as the root cause information of the failure. If the number of suspicious methods is 1, then the suspicious method is used as the root cause information of the failure; If the number of suspicious methods is greater than 1, the suspicious methods are sorted from largest to smallest according to their suspicion rate, and the Topk suspicious methods are output as the root cause information of the fault.

21. The method according to claim 18, characterized in that, The root cause information of the failure also includes a set of suspicious submission records; Based on the indirect fault propagation chain set, the abnormal method call chain set, and the third commit record information set corresponding to each suspicious method that is the root cause of the fault, the suspicious commit record information set is determined.

22. The method according to claim 21, characterized in that, The determination of the suspicious submission record information set includes: If the suspected method that is the root cause of the failure is the head method of the failure propagation chain, then the line of code in which the suspected method throws the exception is determined based on the abnormal method call chain set in the associated application logs. Based on the code line, the third commit record information set is compared and analyzed to obtain the suspicious commit record information set, wherein the suspicious commit record information set includes commit record information that modified the code corresponding to the code line.

23. The method according to claim 22, characterized in that, The method further includes: if the line of code is a method call, the suspicious commit record information set also includes commit record information that modified the parameters of the method call.

24. The method according to claim 21, characterized in that, The determination of the suspicious submission record information set includes: If the suspected method that is the root cause of the failure is the head method of the failure propagation chain that is indirectly dependent on the failure propagation chain, then the shared state accessed or modified by the suspected method is determined. Based on the shared state, the third set of submission records is compared and analyzed to obtain the set of suspicious submission records, wherein the set of suspicious submission records includes the set of submission records that have modified the shared state.

25. The method according to claim 21, characterized in that, The method further includes: Determine the number of lines of code modification corresponding to each suspicious commit record in the suspicious commit record information set; Based on the number of lines modified and the modification time of the code, the top k suspicious commit records are output as the root cause information of the failure.

26. The method according to claim 21, characterized in that, The root cause information of the fault also includes: the location of the abnormality and the type of abnormality.

27. The method according to claim 26, characterized in that, The abnormal location includes at least one of the following: The location of the line of code where the suspicious method throws the exception; The location of the line of code in which the suspicious method throws the exception, which is the last exception method in the exception method call chain. Access or modify the line of code containing the shared state.

28. The method according to claim 26, characterized in that, The exception type includes at least one of the following: The type of exception thrown by the suspicious method; The exception type thrown by the last exception method in the exception method call chain containing the suspicious method; The type of exception thrown by the abnormal method associated with the suspected method; The exception type thrown by the last exception method in the exception method call chain containing the exception method.

29. The method according to claim 1, characterized in that, After determining the root cause information of the fault, the method further includes: Display an alarm message, which includes the root cause information of the fault.

30. A root cause localization device for microservice system failures, characterized in that, include: The acquisition module is used to acquire code data, the first commit record information set, application logs, and method call chain set; The fault propagation chain determination module is used to determine the method fault propagation chain set based on the code data, the first commit record information set, and the method call chain set. An abnormal call chain determination module is used to identify abnormal method call chains in the method call chain set; Based on the application logs and the abnormal link aggregation conditions, the identified abnormal method call chains are aggregated to obtain one or more abnormal method call chain sets. The abnormal link aggregation conditions include: the abnormal method call chains that are aggregated in the same way as the methods and order in which they pass, the methods that throw the exceptions, and the exception types. The fault root cause localization module is used to determine fault root cause information based on the method fault propagation chain set and the abnormal method call chain set, wherein the fault root cause information includes suspicious methods. The fault propagation chain determination module is further configured to perform the following steps: perform static scanning on the code data to determine the indirect dependencies between methods; determine the modified method set and the second commit record information set corresponding to each modified method based on the first commit record information set; and determine the method fault propagation chain set based on the method call chain set, the indirect dependencies, the modified method set, and the second commit record information set.

31. An electronic device, characterized in that, include: Processor and memory; The processor executes the steps of the method as described in any one of claims 1 to 29 by invoking programs or instructions stored in the memory.

32. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores a program or instructions that cause a computer to perform the steps of the method as described in any one of claims 1 to 29.

Images