A root cause model training method, analysis method and device in a microservice system

Abstract

The application provides a root cause model analysis method in a micro-service system, and belongs to the technical field of cloud computing. It solves the problems of low accuracy for developers and the like in the existing method. The root cause model analysis method in the micro-service system comprises the following steps: step S5: collecting distributed tracking log data of a target system, constructing through the distributed tracking data, and determining a potential abnormal node; step S6: collecting logs of the target system, processing the logs, extracting events and parameters, and determining an abnormal event; and step S7: performing fault root cause analysis processing based on the potential abnormal node and the abnormal event, and obtaining an analysis result. The application has the advantages of simplifying the operation and maintenance cost of positioning the root cause of faults in the micro-service system, enabling operation and maintenance personnel to more quickly find the root cause existing in the system and the like.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing technology, and specifically relates to a root cause model training method, analysis method, and apparatus in a microservice system. Background Technology

[0002] With the development of microservice technology, more and more applications are starting to use microservice architecture to provide services. A system may have hundreds of hosts and thousands of services providing system services. However, due to its own complexity and the complexity of the business, nodes in the system often experience anomalies. This not only generates a large number of alarm messages, but in severe cases, it can cause the system to stop providing services, which has a great negative impact on the system's performance and reliability.

[0003] Currently, related technologies still rely on the experience of maintenance engineers and development engineers, who use patterns accumulated from extensive experience to manually guess and test potential fault points.

[0004] Existing methods are less accurate for developers, require engineers with relevant experience, and are time-consuming to troubleshoot. Summary of the Invention

[0005] The first objective of this invention is to address the aforementioned problems in the prior art by providing a root cause model training method, analysis method, and apparatus for a microservice system. This method enables intelligent root cause analysis based on historical data, allowing for rapid identification of the root cause of an event when it occurs, reducing event processing time, and improving system stability and performance.

[0006] The first objective of this invention can be achieved through the following technical solution: a root cause model training method in a microservice system, characterized by comprising the following steps:

[0007] Step S1: Obtain the historical distributed tracing logs and historical running logs of the system during normal operation, as well as the problem feedback data within the corresponding events of the historical distributed tracing logs and historical running logs;

[0008] Step S2: Process the historical distributed tracing logs and historical running logs to obtain the characteristics of historical potential abnormal nodes and historical potential abnormal logs;

[0009] Step S3: Input the historical potential abnormal nodes and problem feedback data as a set of training data into the ranking sub-model of the fault root cause localization model, and train the ranking sub-model of the fault root cause localization model through the training data.

[0010] Step S4: Input the feature information and problem feedback data from the historical operation logs as a set of training data into the cause sub-model of the fault root cause localization model, and train the cause sub-model of the fault root cause localization model using the training data.

[0011] In the root cause model training method of the microservice system described above, online problem feedback data is generated by performing preset operations on the device running the target system.

[0012] In the root cause model training method of the microservice system described above, the input data in the training data and the input data in the usage process have the same or similar forms.

[0013] In the root cause model training method of the microservice system mentioned above, the historical operation logs are processed by the Drain method and the GPT method to obtain the feature information of the historical operation logs; the historical distributed tracing logs are analyzed to analyze the response time and thus obtain the historical potential abnormal nodes.

[0014] In the root cause model training method in the microservice system described above, in step S3, the aggregated historical potential abnormal nodes and aggregated problem feedback data within the same time window are used as a set of training data. A preset loss function is used to calculate the loss value between the predicted root cause of the fault and the problem feedback data. The model parameters of the ranking sub-model of the fault root cause localization model are adjusted according to the loss value.

[0015] In the root cause model training method in the microservice system described above, in step S4, the aggregated historical log feature information and aggregated problem feedback data within the same time window are used as a set of training data. A preset loss function is used to calculate the loss value between the predicted root cause of the fault and the problem feedback data. The model parameters of the cause sub-model of the fault root cause localization model are adjusted according to the loss value.

[0016] In the root cause model training method in the microservice system described above, the training samples are expanded and the data is extended.

[0017] The second objective of this invention can be achieved through the following technical solution: a root cause analysis method in a microservice system, characterized in that, based on the above-mentioned root cause model training method in a microservice system, it further includes the following steps:

[0018] Step S5: Collect distributed tracing log data of the target system, and construct a system based on the distributed tracing data to identify potential abnormal nodes;

[0019] Step S6: Collect the logs of the target system, process the logs, extract events and parameters, and identify abnormal events;

[0020] Step S7: Perform root cause analysis based on the potential abnormal nodes and abnormal events to obtain the analysis results.

[0021] In the root cause analysis method in the microservice system described above, step S5 includes step S5.1: collecting distributed tracing log data during the runtime of the target system, constructing a service call chain based on the tracing ID and span ID, analyzing the response time of each span, and identifying potential faulty nodes.

[0022] Step S6 includes step S6.1: obtaining the running log information corresponding to the target system, parsing the logs using Drain technology, and obtaining abnormal log information;

[0023] Step S7 includes the following steps:

[0024] Step S7.1: Extract features from the above abnormal log information to obtain feature information;

[0025] Step S7.2: Input the potential abnormal nodes into the ranking sub-model of the fault root cause localization model, analyze and process the potential abnormal nodes, and obtain the probability ranking of the fault nodes.

[0026] Step S7.3: Input the log feature information into the cause sub-model of the fault root cause localization model, analyze and process the true root cause of the fault node, and obtain the possible causes of the fault node.

[0027] Step S7.4: Input the possible causes of the fault node and the probability ranking of the fault node into the determination layer of the fault root cause localization model for analysis and processing to obtain the target fault root cause.

[0028] The third objective of the present invention can be achieved by the following technical solution: a root cause analysis apparatus in a microservice system, characterized in that it includes a communication component, a power component, an audio component, a display, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs include methods for performing root cause model analysis in a microservice system as described above.

[0029] Compared with existing technologies, this invention simplifies the operation and maintenance costs of locating the root cause of failures in microservice systems, enabling operation and maintenance personnel to discover the root cause of problems in the system more quickly. It also uses artificial intelligence technology, which can learn as the business system develops, reducing subsequent maintenance costs and making it easier to discover problems. It does not require operation and maintenance personnel to rely on experience to find the root cause of the system, thus improving the efficiency of personnel. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a root cause model training method in a microservice system according to the present invention.

[0031] Figure 2 This is a flowchart illustrating a root cause analysis method in a microservice system according to the present invention.

[0032] Figure 3 This is a schematic diagram of a fault root cause localization model for a root cause analysis method in a microservice system according to the present invention.

[0033] Figure 4 This is a training diagram of a root cause analysis method in a microservice system according to the present invention.

[0034] Figure 5 This is a schematic diagram of a fault root cause determination device according to the present invention.

[0035] Figure 6 This is a schematic diagram of the root cause analysis device in a microservice system according to the present invention. Detailed Implementation

[0036] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0037] like Figures 1-6 As shown, the first objective of the invention can be achieved through the following technical solution: a root cause model training method in a microservice system, comprising the following steps:

[0038] Step S1: Obtain the historical distributed tracing logs and historical running logs of the system during normal operation, as well as the problem feedback data within the corresponding events of the historical distributed tracing logs and historical running logs;

[0039] As the data foundation used for training the model, to distinguish it from the distributed tracing data and runtime logs described in S5 and S6, the distributed tracing data and runtime logs used for training the model are referred to as historical distributed tracing data and historical runtime logs. The historical distributed tracing logs and historical runtime logs were generated earlier than the distributed tracing data and runtime logs described in S5 and S6. The historical distributed tracing logs and historical runtime logs are basically the same in type and form as the distributed tracing logs and runtime logs, and will not be elaborated or restricted here.

[0040] Step S2: Process the historical distributed tracing logs and historical running logs to obtain the characteristics of historical potential abnormal nodes and historical potential abnormal logs;

[0041] Step S3: Input the historical potential abnormal nodes and problem feedback data as a set of training data into the ranking sub-model of the fault root cause localization model, and train the ranking sub-model of the fault root cause localization model through the training data.

[0042] Step S4: Input the feature information and problem feedback data from the historical operation logs as a set of training data into the cause sub-model of the fault root cause localization model, and train the cause sub-model of the fault root cause localization model using the training data.

[0043] To elaborate further, pre-defined operations are performed on devices running the target system to generate online problem feedback data.

[0044] In addition, it is necessary to obtain problem feedback data. In one example scenario, users perform some preset operations on the device running the target system, thereby generating online problem feedback data. For different target systems, the type and content of the problem feedback data may differ. Therefore, there are no restrictions on the type and content of the problem feedback data here. For ease of understanding, several examples are provided below.

[0045] Example 1: User A, by using the target system running on the device, pre-configured the target system device, causing the CPU utilization of the target system to be too high, resulting in operation response delays during use. The user A recorded the time when the problem was discovered and the cause of the problem.

[0046] Example 2: User B, by using the target system running on the device, pre-configured the target system device, causing excessive memory usage of the target system, resulting in operation response delays during use. The user recorded the time when the problem was discovered and the cause of the problem.

[0047] To elaborate further, the input data in the training data and the input data during the usage process have the same or similar forms.

[0048] In this embodiment of the application, in order to ensure the effectiveness of the trained anomaly detection model, the input data in the training data and the input data in the usage process are required to have the same or similar forms.

[0049] To elaborate further, historical operation logs are processed using the Drain and GPT methods to obtain characteristic information of the historical operation logs; historical distributed tracing logs are analyzed to analyze response time and thus identify potential historical abnormal nodes.

[0050] Based on the descriptions in embodiments S6.1 and S7.1, historical operation logs need to be processed using the Drain and GPT methods to obtain log characteristics. Based on the description in embodiment S5.1, historical distributed tracing logs need to be analyzed to determine their response times and thus identify potentially abnormal historical nodes.

[0051] To elaborate further, in step S3, the aggregated historical potential anomaly nodes and aggregated problem feedback data within the same time window are used as a set of training data. A preset loss function is used to calculate the loss value between the predicted root cause of the fault and the problem feedback data. The model parameters of the ranking sub-model of the fault root cause localization model are adjusted according to the loss value.

[0052] To ensure the time-series matching of model input and output and to accurately detect the root cause of the system in the time dimension, this embodiment of the application uses aggregated historical potential anomaly nodes and aggregated problem feedback data from the same time window as a set of training data when forming training data; thus, multiple sets of training data can be obtained based on different time windows.

[0053] In this embodiment, historical potential abnormal nodes are input into the ranking sub-model of the fault root cause localization model. Then, a preset loss function is used to calculate the loss value between the predicted fault root cause and the problem feedback data. The model parameters of the ranking sub-model of the fault root cause localization model are adjusted according to the loss value.

[0054] To elaborate further, in step S4, the aggregated historical log feature information and aggregated problem feedback data within the same time window are used as a set of training data. A preset loss function is used to calculate the loss value between the predicted root cause of the fault and the problem feedback data. The model parameters of the cause sub-model of the fault root cause localization model are adjusted according to the loss value.

[0055] To ensure the time-series matching of model input and output and to accurately detect the root cause of the system in the time dimension, this embodiment of the application uses aggregated historical log feature information and aggregated problem feedback data from the same time window as a set of training data when forming training data; thus, multiple sets of training data can be obtained based on different time windows.

[0056] In this embodiment, historical log feature information is input into the cause sub-model of the fault root cause localization model. Then, a preset loss function is used to calculate the loss value between the predicted fault root cause and the problem feedback data. The model parameters of the cause sub-model of the fault root cause localization model are adjusted according to the loss value.

[0057] Furthermore, the fault root cause localization model trained in the embodiments of this application can also be applied in cloud computing intelligent scenarios and scenarios for determining faults in new energy equipment.

[0058] To elaborate further, this involves expanding the training samples and extending the data.

[0059] In the embodiments of this application, by expanding the training samples and data, the robustness of the fault root cause localization model can be improved, thereby enabling the trained fault root cause localization model to accurately determine the root cause of the system failure, thus ensuring system quality.

[0060] The second objective of the invention can be achieved through the following technical solution: a root cause analysis method in a microservice system, based on the aforementioned root cause model training method in a microservice system, further comprising the following steps:

[0061] Step S5: Collect distributed tracing log data of the target system, and construct a system based on the distributed tracing data to identify potential abnormal nodes;

[0062] In this application embodiment, the target system refers to any software system to be detected, and the functionality of the target system is not limited here. The distributed tracing data of the target system includes, but is not limited to: TraceID, SpanId, parent spanId, service name, response time, etc. The TraceID is a unique ID of the distributed tracing in the entire response process, the SpanId is the ID of the distributed tracing performed in a certain service, the parent spanId is the ID of the distributed tracing performed in the previous service, the service name is the name of the service running in the target system, and the response time is the response time of this request in this service.

[0063] In one possible implementation, the service call chain graph can be generated by traversing the span ID and the parent span ID. In practical applications, the method of converting text into vectors is not limited to the traversal method of span ID and parent span ID. This solution does not limit the specific implementation method of S5.

[0064] Step S6: Collect the logs of the target system, process the logs, extract events and parameters, and identify abnormal events;

[0065] The target system's log data includes, but is not limited to: log ID, log time, log level, log generator, and log event; the log ID is the globally unique ID of the log in the target system, the log time is the time when the log was generated in the target system, the log level is the level to which the log belongs in the target system, the log generator is the information of the function or object to which the log belongs in the target system, and the log event is the information printed by the log in the target system.

[0066] An alternative implementation is to parse the runtime logs after obtaining them to obtain log events and log parameters.

[0067] Furthermore, after obtaining the runtime logs, the exception logs and the events that occurred within them are identified. These exception logs constitute at least a portion of the runtime logs.

[0068] Step S7: Perform root cause analysis based on the potential abnormal nodes and abnormal events to obtain the analysis results.

[0069] Among them, the fault root cause localization model is pre-trained and can accurately predict the target fault root cause based on the input information.

[0070] In this embodiment, the fault root cause localization model can combine multiple sub-models, each employing a corresponding algorithm model. By combining multiple sub-models, accurate prediction of the target fault root cause can be achieved. Alternatively, the fault root cause localization model can employ personalized PageRank algorithms, GPT (a natural language processing model), XGBoost (a machine learning model), etc. These algorithms can be pre-trained to predict the target fault root cause individually, or they can be integrated for prediction.

[0071] Furthermore, the root cause of a target failure refers to the condition or the initial cause of the causal chain that leads to the corresponding failure, specifically the fundamental, potential, deepest, or initial cause.

[0072] In this embodiment of the application, sending the target fault root cause to the terminal device enables maintenance personnel to obtain the target fault root cause in a timely manner, thereby repairing the system fault.

[0073] This method constructs a call chain from the distributed tracing data of the target system, analyzes it to obtain potential abnormal nodes, analyzes the target system's runtime logs to obtain log events and log parameters, and obtains abnormal logs by analyzing log events and log parameters. Then, it inputs the potential abnormal nodes and potential abnormal logs into the root cause localization model for root cause analysis to obtain the potential root cause of the failure. This application uses a pre-trained root cause analysis model to assist operation and maintenance personnel in completing the root cause analysis of the target system, reducing the professional requirements of operation and maintenance personnel and eliminating the need for complex maintenance. Therefore, it can improve the efficiency of system root cause analysis while reducing the implementation difficulty and cost.

[0074] To elaborate further, step S5 includes step S5.1: collecting distributed tracing log data during the runtime of the target system, constructing a service call chain based on the tracing ID and span ID, analyzing the response time of each span, and identifying potential faulty nodes;

[0075] Based on the response time of each span, if it exceeds the set threshold time, it indicates that the service may be faulty. To reduce false alarms, a normal distribution is used for judgment. If the response time exceeds 3σ, it indicates that the node may be faulty and belongs to the category of potential faulty nodes. Then, the upstream and downstream nodes of the node are traversed until all potential faulty nodes are found.

[0076] The aforementioned faulty nodes will be reconstructed into a new fault subgraph for subsequent processing.

[0077] Step S6 includes step S6.1: obtaining the running log information corresponding to the target system, and using Drain technology to parse the logs to obtain abnormal log information.

[0078] The Drain (An Online Log Parsing Approach with Fixed Depth Tree) technology can parse log samples to obtain log events and log parameters.

[0079] Search for log information during the fault period of the above-mentioned faulty nodes. Since some hidden anomalies may not be displayed before the fault occurs, the fault period refers to the time period before and after the fault occurs. The fault period can be a preset time period, such as logs that occurred 1 hour before the fault occurred. The specific fault period can be determined as needed.

[0080] Step S7 includes the following steps:

[0081] Step S7.1: Extract features from the above abnormal log information to obtain feature information;

[0082] The aforementioned abnormal log information contains a large amount of data, which needs to be processed to extract important data for analysis and reduce interference from other information. Specifically, in this embodiment, GPT (a natural language processing model) is used to extract features from the abnormal log information to obtain potential information that may exist in the log.

[0083] Step S7.2: Input the potential abnormal nodes into the ranking sub-model of the fault root cause localization model, analyze and process the potential abnormal nodes, and obtain the probability ranking of the fault nodes.

[0084] The root cause localization model can be an ensemble model that integrates one or more sub-models. Among them, the ranking sub-model will determine the root cause node of the failure. The ranking sub-model adopts the PageRank algorithm, and the ranking sub-model using the PageRank algorithm serves as the basis for the root cause analysis model, which has excellent performance.

[0085] Furthermore, the ranking sub-model is pre-trained and can analyze and process potential faulty nodes to obtain a probability ranking of the faulty nodes. The higher the ranking, the more likely it is to be the root cause of the fault. In the embodiments of this application, the first-ranked node can be multiple, such as multiple faults caused by insufficient memory resources and network failures.

[0086] Step S7.3: Input the log feature information into the cause sub-model of the fault root cause localization model, analyze and process the true root cause of the fault node, and obtain the possible causes of the fault node.

[0087] This can be achieved by using Granger causality techniques and the PageRank algorithm. Granger causality techniques can be used to infer the causal relationship between error signals emitted by services and logs. It is assumed that the abnormal behavior of a faulty component is likely to cause neighboring components (microservices) to emit error signals. These components are those that interact directly or indirectly with the faulty component. However, nodes unrelated to the error are likely to have higher causal scores. To avoid these false alarms, the cause of the error is considered as a candidate cause. Using a personalized PageRank algorithm to rank the error causes can assign higher weights to the error causes that caused the failure.

[0088] Step S7.4: Input the possible causes of the fault node and the probability ranking of the fault node into the determination layer of the fault root cause localization model for analysis and processing to obtain the target fault root cause.

[0089] The determination layer ranks various pre-set weights based on the possible causes and probabilities of the faulty node to identify the target root cause of the fault.

[0090] Among them, the two sub-models mentioned above are used in two ways to determine the possible causes of the fault nodes and the probability ranking of the fault nodes; a determination layer is used to determine the final target root cause of the fault; in this embodiment of the application, the pre-set weights corresponding to the determination layer can be pre-trained.

[0091] In one optional embodiment, the possible causes of the fault node and the probability ranking of the fault node can also be sent directly to the terminal device. The terminal device is used to display the possible causes of the fault node and the probability ranking of the fault node. The system also receives the operation and maintenance personnel's fault location information sent by the terminal device. The operation and maintenance personnel determine the fault location information based on the possible causes of the fault node and the probability ranking of the fault node. Finally, the system determines the target fault root cause based on the operation and maintenance personnel's fault location information, the possible causes of the fault node, and the probability ranking of the fault node.

[0092] In this process, maintenance personnel can determine potential causes of the fault based on the displayed possible causes and the probability ranking of the fault nodes, and then use their experience to locate the root cause of the fault. The maintenance personnel input the root cause of the fault into the terminal device, which then sends it to the server. The server can determine the final target root cause of the fault based on the root cause of the fault, the possible causes of the fault nodes, and the probability ranking of the fault nodes, thereby improving the accuracy of the target root cause of the fault.

[0093] Send the target fault root cause to the terminal device so that the terminal device can display the target fault root cause to the maintenance personnel.

[0094] In the embodiments of this application, the fault root cause localization model integrates multiple sub-models and makes full use of the fault cause-effect graph; the sub-models complement each other, and finally obtain the accurate target fault root cause.

[0095] The third objective of the invention can be achieved through the following technical solution: a fault root cause determination device 500, the fault root cause determination device 500 comprising:

[0096] The acquisition module 501 is used to acquire multiple operational log information and distributed tracing information during the system failure period. Each distributed tracing log includes: tracing ID, span ID, parent span ID, service name, and response time. Each operational log includes: log ID, log time, log level, log generator, and log event.

[0097] The data processing module 502 is used to process distributed tracing information and operation log information to obtain potential abnormal nodes and operation log feature information.

[0098] The input module 503 is used to input potential abnormal nodes and operation log information into the fault root cause localization model for fault root cause analysis and processing to obtain the target fault root cause of the fault.

[0099] The sending module 504 is used to send the target fault root cause to the terminal device so that the terminal device can display the target fault root cause to the maintenance personnel.

[0100] The fault root cause determination device provided in this application extends the operation log information and distributed tracing information to obtain potential fault nodes and operation log feature information, which are used as inputs to the fault root cause localization model, and can accurately determine the target fault root cause of the fault.

[0101] The fourth objective of the invention can be achieved by the following technical solution: a root cause analysis apparatus in a microservice system, comprising an input terminal, an output terminal, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising methods for performing root cause model analysis in a microservice system as described above.

[0102] Memory 601 is used to store programs. In addition to the programs described above, memory 601 can also be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phonebook data, messages, pictures, videos, etc.

[0103] The memory 601 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0104] Processor 602 is not limited to a processor (CPU), but may also be a graphics processing unit (GPU), a field-programmable gate array (FPGA), an embedded neural network processor (NPU), or an artificial intelligence (AI) chip. Processor 602 is coupled to memory 601 and executes the program stored in memory 601. When the program runs, it executes the root cause analysis method in the aforementioned microservice system.

[0105] Communication component 603 is configured to facilitate wired or wireless communication between electronic devices and other devices. The electronic devices can access wireless networks based on communication standards, such as WiFi, 3G, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 603 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 603 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0106] Power supply component 604 provides power to various components of an electronic device. Power supply component 604 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0107] Audio component 605 is configured to output and / or input audio signals. For example, audio component 605 includes a microphone (MIC) configured to receive external audio signals when the electronic device is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 501 or transmitted via communication component 603. In some embodiments, audio component 605 also includes a speaker for outputting audio signals.

[0108] Display 606 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.

[0109] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0110] Although this document uses a large number of technical terms, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of the invention; interpreting them as any additional limitation would be contrary to the spirit of the invention.

Claims

1. A root cause model training method in a microservice system, characterized in that, Includes the following steps: Step S1: Obtain the historical distributed tracing logs and historical running logs of the system during normal operation, as well as the problem feedback data within the corresponding events of the historical distributed tracing logs and historical running logs; Step S2: Process the historical distributed tracing logs and historical running logs to obtain the characteristics of historical potential abnormal nodes and historical potential abnormal logs; Step S3: Input the historical potential abnormal nodes and problem feedback data as a set of training data into the ranking sub-model of the fault root cause localization model, and train the ranking sub-model of the fault root cause localization model through the training data. Step S4: Input the feature information and problem feedback data from the historical operation logs as a set of training data into the cause sub-model of the fault root cause localization model, and train the cause sub-model of the fault root cause localization model using the training data. By executing preset operations on the device running the target system, online problem feedback data is generated. The input data in the training data and the input data during use have the same or similar forms; The historical running logs are processed using the Drain and GPT methods to obtain the characteristic information of the historical running logs; Analyze historical distributed tracing logs and response times to identify potentially abnormal nodes. In step S3, the aggregated historical potential anomaly nodes and aggregated problem feedback data within the same time window are used as a set of training data. A preset loss function is used to calculate the loss value between the predicted root cause of the fault and the problem feedback data. The model parameters of the ranking sub-model of the fault root cause localization model are adjusted according to the loss value. In step S4, the aggregated historical log feature information and aggregated problem feedback data within the same time window are used as a set of training data. A preset loss function is used to calculate the loss value between the predicted root cause of the fault and the problem feedback data. The model parameters of the cause sub-model of the fault root cause localization model are adjusted according to the loss value.

2. The root cause model training method in a microservice system according to claim 1, characterized in that, Expanding the training samples and data.

3. A root cause analysis method for a microservice system, characterized in that, The root cause model training method in a microservice system according to any one of claims 1-2 further includes the following steps: Step S5: Collect distributed tracing log data of the target system, and construct a system based on the distributed tracing log data to identify potential abnormal nodes; Step S6: Collect the logs of the target system, process the logs, extract events and parameters, and identify abnormal events; Step S7: Perform root cause analysis based on the potential abnormal nodes and abnormal events to obtain the analysis results; Step S5 includes step S5.1: collecting distributed tracing log data during the runtime of the target system, constructing a service call chain based on the tracing ID and span ID, analyzing the response time of each span, and identifying potential faulty nodes; Step S6 includes step S6.1: obtaining the running log information corresponding to the target system, parsing the logs using Drain technology, and obtaining abnormal log information; Step S7 includes the following steps: Step S7.1: Extract features from the above abnormal log information to obtain feature information; Step S7.2: Input the potential abnormal nodes into the ranking sub-model of the fault root cause localization model, analyze and process the potential abnormal nodes, and obtain the probability ranking of the fault nodes. Step S7.3: Input the log feature information into the cause sub-model of the fault root cause localization model, analyze and process the true root cause of the fault node, and obtain the possible causes of the fault node. Step S7.4: Input the possible causes of the fault node and the probability ranking of the fault node into the determination layer of the fault root cause localization model for analysis and processing to obtain the target fault root cause.

4. A root cause analysis device in a microservice system, characterized in that, The system includes communication components, power components, audio components, a display, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs include methods for performing root cause analysis in a microservice system as described in claim 3.

Images