Fault analysis method and device, electronic equipment, storage medium and product

By using horizontal and vertical dependencies in the operation and maintenance monitoring system to analyze faults, automatically locate fault nodes and analyze the cause of faults, the problem of low fault positioning and analysis efficiency in the operation and maintenance monitoring system is solved, reducing labor costs and improving accuracy .

CN119094330APending Publication Date: 2024-12-06ANHUI IFLYTEK INTELLIGENT SYST
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202411294077.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-14
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing operation and maintenance monitoring system cannot effectively locate problem points and causes when a failure occurs, resulting in high cost of manual analysis, low efficiency and accuracy.

Method used

Dependency application nodes are determined based on the initial abnormal application node when a fault occurs and the pre-built horizontal dependency relationship, combined with vertical dependencies and full-stack data to analyze the cause of failure, and use network requests and system monitoring data to automatically locate and analyze the cause of failure.

Benefits of technology

It reduces labor costs, improves the efficiency and accuracy of fault positioning analysis, and realizes automatic fault positioning and fault cause analysis without manual analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094330A_ABST
    Figure CN119094330A_ABST
Patent Text Reader

Abstract

The invention provides a fault analysis method and device, electronic equipment, a storage medium and a product, and the method comprises the steps: determining a dependent application node of an initial abnormal application node based on an address and a transverse dependency relationship of the initial abnormal application node when a fault occurs; the transverse dependency relationship comprises dependency relationships among the application nodes and application processing durations corresponding to the application nodes; determining a target abnormal application node corresponding to the fault according to the application processing duration corresponding to the dependent application node; and based on the longitudinal dependency relationship, obtaining target monitoring data corresponding to the target abnormal application node from the full-stack data, and analyzing a fault reason of the target abnormal application node by using the target monitoring data. According to the scheme, the fault node can be automatically positioned through the transverse dependency relationship and the network request data, the fault reason is automatically analyzed by utilizing the longitudinal dependency relationship and the full-stack data, manual analysis and processing are not needed, the labor cost is reduced, and the efficiency and the accuracy of fault positioning analysis are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of fault analysis, and in particular to a fault analysis method, device, electronic device, storage medium and product. Background Art

[0002] The operation and maintenance monitoring system refers to an integrated platform for monitoring, managing and maintaining IT infrastructure. Currently, most of the mainstream operation and maintenance monitoring systems only have functions such as data collection and alarm. When a fault occurs, they cannot locate the problem point and the actual cause of the problem. It still requires personnel who are familiar with the business and have professional operation and maintenance skills to analyze and handle the problem in order to solve it, which increases labor costs and reduces the efficiency and accuracy of fault location analysis. Summary of the invention

[0003] Based on the above needs, the present application proposes a fault analysis method, device, electronic device, storage medium and product, which can reduce labor costs and improve the efficiency and accuracy of fault location analysis.

[0004] To achieve the above objectives, this application proposes the following technical solutions:

[0005] According to a first aspect of an embodiment of the present application, a fault analysis method is provided, including:

[0006] Determine the dependent application nodes of the initial abnormal application node based on the address of the initial abnormal application node when the fault occurs and the pre-built horizontal dependency relationship; wherein the horizontal dependency relationship includes the dependency relationship between each application node and the application processing time corresponding to each application node;

[0007] Determine the target abnormal application node corresponding to the fault according to the application processing time corresponding to the dependent application node;

[0008] Based on the pre-built vertical dependency, the target monitoring data corresponding to the target abnormal application node is obtained from the full-stack data, and the target monitoring data is used to analyze the failure cause of the target abnormal application node; the vertical dependency includes the dependency between the operating environment of the application node and the application node, and the full-stack data includes network request data and system monitoring data.

[0009] Optionally, the process of building the horizontal dependency relationship includes:

[0010] Determine network request data based on call link data in the network request;

[0011] Based on the network request data, the dependency relationship between each application node in the network request data and the application processing time of each application node are determined, and based on the dependency relationship between each application node and the application processing time of each application node, a horizontal dependency relationship is constructed.

[0012] Optionally, based on the call link data in the network request, the network request data is determined, including:

[0013] Using an APM probe to collect first call link data in a network request, and / or using an eBPF probe to collect second call link data in a network request; wherein the first call link data and the second call link data are both data of an application node call request in a network request;

[0014] The network request data is determined based on the first call link data and / or the second call link data.

[0015] Optionally, determining a target abnormal application node corresponding to the fault according to the application processing duration corresponding to the dependent application node includes:

[0016] If the initial abnormal application node returns an exception, the initial abnormal application node is used as the target abnormal application node corresponding to the fault;

[0017] If the initial abnormal application node does not return an exception, the dependent application nodes whose application processing time ratio reaches a preset threshold are sorted in descending order according to the application processing time and serve as the target abnormal application nodes corresponding to the fault.

[0018] Optionally, based on the pre-built vertical dependency relationship, target monitoring data corresponding to the target abnormal application node is obtained from the full-stack data, including:

[0019] Based on the pre-built vertical dependency relationship, query the target operating environment on which the target abnormal application node depends;

[0020] Data corresponding to the target operating environment is obtained from the full-stack data as target monitoring data corresponding to the target abnormal application node.

[0021] Optionally, using the target monitoring data to analyze the cause of the failure of the target abnormal application node includes:

[0022] Based on the target monitoring data, determine the intermediate information corresponding to the target abnormal application node; wherein the intermediate information includes: current interface response time, average interface response time, interface response time during the same period, current business volume, business volume during the same period, predicted response time and process usage data;

[0023] Based on a preset fault type judgment rule, the fault type is identified for the intermediate information corresponding to the target abnormal application node, and the fault cause of the target abnormal application node is determined.

[0024] Optionally, based on a preset fault type judgment rule, identifying the fault type of the intermediate information corresponding to the target abnormal application node and determining the fault cause of the target abnormal application node include:

[0025] Inputting the intermediate information corresponding to the target abnormal application node into a pre-trained fault type recognition model to obtain the fault cause of the target abnormal application node;

[0026] Among them, the fault type identification model is based on fault type identification of sample intermediate information corresponding to sample abnormal nodes, and is trained with the goal of minimizing the difference between the predicted fault cause obtained by fault type identification and the actual fault cause of the sample abnormal node; the sample intermediate information corresponding to the sample abnormal node is determined based on sample monitoring data of the sample abnormal node collected in advance.

[0027] Optionally, the fault analysis method further includes:

[0028] When the fault is a host abnormality, host monitoring data corresponding to the host is obtained from the full-stack data, and the fault cause corresponding to the fault is analyzed using the host monitoring data.

[0029] According to a second aspect of an embodiment of the present application, a fault analysis device is provided, including:

[0030] A node query module, used to determine the dependent application nodes of the initial abnormal application node based on the address of the initial abnormal application node when the fault occurs and the pre-built horizontal dependency relationship; wherein the horizontal dependency relationship includes the dependency relationship between each application node and the application processing time corresponding to each application node;

[0031] An abnormal node determination module, used to determine a target abnormal application node corresponding to the fault according to the application processing time corresponding to the dependent application node;

[0032] A fault analysis module is used to obtain target monitoring data corresponding to the target abnormal application node from the full-stack data based on a pre-built vertical dependency relationship, and use the target monitoring data to analyze the cause of the failure of the target abnormal application node; the vertical dependency relationship includes the dependency between the operating environment of the application node and the application node, and the full-stack data includes network request data and system monitoring data.

[0033] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including: a memory and a processor;

[0034] The memory is connected to the processor and is used to store programs;

[0035] The processor is used to implement the above fault analysis method by running the program in the memory.

[0036] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned fault analysis method is implemented.

[0037] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to implement the above-mentioned fault analysis method.

[0038] The fault analysis method proposed in the present application determines the dependent application nodes of the initial abnormal application node based on the address of the initial abnormal application node when the fault occurs and the pre-built horizontal dependency relationship; wherein the horizontal dependency relationship includes the dependency relationship between each application node and the application processing time corresponding to each application node; according to the application processing time corresponding to the dependent application node, the target abnormal application node corresponding to the fault is determined; based on the pre-built vertical dependency relationship, the target monitoring data corresponding to the target abnormal application node is obtained from the full stack data, and the target monitoring data is used to analyze the cause of the fault of the target abnormal application node; the vertical dependency relationship includes the dependency relationship between the operating environment of the application node and the application node, and the full stack data includes network request data and system monitoring data. The technical solution of the present application can automatically locate the fault node through the horizontal dependency relationship and the network request data, and automatically analyze the cause of the fault using the vertical dependency relationship, the system monitoring data and the network request data, without the need for manual analysis and processing, thereby reducing labor costs and improving the efficiency and accuracy of fault location analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0040] Figure 1 A schematic diagram of a flow chart of a fault analysis method provided in an embodiment of the present application;

[0041] Figure 2A schematic diagram of a process flow for constructing a horizontal dependency relationship provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of the structure of an application node call request provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the structure of the horizontal dependency relationship provided in the embodiment of the present application;

[0044] Figure 5 A schematic diagram of a process flow for analyzing the cause of a fault provided in an embodiment of the present application;

[0045] Figure 6 A flowchart of another fault analysis method provided in an embodiment of the present application;

[0046] Figure 7 A schematic diagram of the structure of a fault analysis device provided in an embodiment of the present application;

[0047] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solution of the embodiment of the present application is applicable to the application scenario of fault analysis, and is specifically used in the application scenario of fault location and fault cause analysis. The technical solution of the embodiment of the present application can reduce labor costs and improve the efficiency and accuracy of fault location analysis.

[0049] The operation and maintenance monitoring system refers to an integrated platform for monitoring, managing and maintaining IT infrastructure. It collects various performance data and alarm information through real-time monitoring of servers, network devices, storage devices, applications, etc. to ensure the stable operation of IT systems. At present, most of the mainstream operation and maintenance monitoring systems only have functions such as data collection and alarm. When a fault occurs, it is impossible to locate the problem point and the actual cause of the problem. It still requires personnel who are familiar with the business and have professional operation and maintenance skills to analyze and handle the problem in order to solve it, which increases labor costs and reduces the efficiency and accuracy of fault location analysis.

[0050] Based on this, the present application proposes a fault analysis method. This technical solution can automatically locate the fault node through horizontal dependencies and network request data, and automatically analyze the cause of the fault using vertical dependencies, system monitoring data and network request data without the need for manual analysis and processing, thereby solving the problems of high labor costs and low efficiency and accuracy of fault location analysis in the prior art.

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] Exemplary Methods

[0053] See also Figure 1 As shown, the embodiment of the present application proposes a fault analysis method. The method includes:

[0054] S101 . Determine dependent application nodes of the initial abnormal application node based on the address of the initial abnormal application node when a fault occurs and a pre-built horizontal dependency relationship.

[0055] In this embodiment, the system for fault analysis will monitor the server, network equipment, storage equipment, application programs and other facilities in real time, so as to collect full-stack data in real time, wherein the full-stack tools include network request data and system monitoring data, and the network request data is the network message information monitored after the network request is initiated, including the relevant data of the mutual calls between the various application nodes in the network request, etc. The system monitoring data includes infrastructure data, application software data, middleware monitoring data, database monitoring data, etc. The infrastructure data includes the CPU, memory, storage, network, Cache and other data of the virtual machine and container; the middleware monitoring data mainly collects middleware data such as cache (hit rate, number of connections, process memory, CPU), message queue (queue data, consumption data, number of connections, process CPU, process memory); the database monitoring data mainly includes the number of database connections, the number of business submissions per second, the SQL data executed per second, the process CPU usage data, the process memory usage data, etc.; the application software data includes log data, process operation data, operation and maintenance work order data, etc.

[0056] The network request data packet in the full-stack data contains the calling relationship between each application node and the application processing time of each application node (for example, the time difference between the time when application node A sends a calling request to application node B and the time when the call returns can be used as the application processing time of application node A calling application node B). The dependency relationship between each application node in the network request can be constructed. For example, if application node A calls application node B, it means that application node A depends on application node B, and application node A has a dependency relationship with application node B, and the application processing time of application node A is the total time of the internal processing time of A and the application processing time of all other application nodes called by application node A. Using the dependency relationship between each application node in the network request and the corresponding application processing time of each application node, the corresponding horizontal dependency relationship can be constructed.

[0057] In this embodiment, when a fault occurs, it is first necessary to detect the application node where the initial abnormality occurs as the initial abnormal application node, and then obtain the address (i.e., URL) of the initial abnormal application node. Among them, the existing operation and maintenance monitoring system can detect the application node where the initial abnormality occurs, and can also obtain the address of the node, so this embodiment does not specifically describe the steps of obtaining the address of the initial abnormal application node.

[0058] When a fault occurs, the application node that initially experiences an abnormality is not necessarily the node where the fault occurs. It is possible that an abnormality occurs in other nodes called by the application node. Therefore, this embodiment needs to determine the dependent application nodes of the initial abnormal application node based on the pre-constructed horizontal dependency relationship. The dependent application nodes include the application nodes that the initial abnormal application node depends on in the horizontal dependency relationship. The application node can serve as the first-level dependent application node of the initial abnormal application node. The dependent application nodes also include the application nodes that the first-level dependent application nodes of the initial abnormal application node depend on. The application node can serve as the second-level dependent application node of the initial abnormal application node, and so on, until the dependent application node has no application nodes that it depends on, that is, the dependent application nodes of the initial application node include multiple levels of dependent application nodes.

[0059] Furthermore, based on the address of the initial abnormal application node when the fault occurs and the pre-built horizontal dependency relationship, the dependent application nodes of the initial abnormal application node are determined. In this embodiment, the fault is preferably an interface error corresponding to the application node or an interface response time timeout corresponding to the application node. In addition, the triggering condition of this step can be a manual trigger, that is, after the staff finds the fault, the fault analysis is manually triggered, or it can be an automatic trigger, that is, the fault analysis is automatically triggered after the abnormal alarm or timeout alarm is detected.

[0060] S102: Determine a target abnormal application node corresponding to the fault according to the application processing duration corresponding to the dependent application node.

[0061] After the dependent application nodes of the initial abnormal application node are determined through the above steps, the application processing time corresponding to each dependent application node is queried from the pre-built horizontal dependency relationship. This embodiment needs to determine the target abnormal application node corresponding to the fault according to the application processing time corresponding to each dependent application node of the initial abnormal application node. First, it is necessary to check whether the initial abnormal application node itself has an abnormality. It can be judged whether the node has an abnormality based on the relevant data of the initial abnormal application node in the pre-collected network request data. If it is judged that the initial abnormal application node has an abnormality, it is determined that the initial abnormal application node returns an abnormality. If it is judged that the initial abnormal application node does not have an abnormality, it is determined that the initial abnormal application node does not return an abnormality.

[0062] When the initial abnormal application node returns an abnormality, the initial abnormal application node is directly used as the target abnormal application node corresponding to the fault, that is, the fault location finally located is the target abnormal application node. Then, the next step of fault cause analysis is performed on the target abnormal application node.

[0063] When the initial abnormal application node does not return an exception, it means that the application node that caused the fault is among the dependent application nodes of the initial abnormal application node. At this time, it is necessary to determine the application processing time ratio of each dependent application node of the initial abnormal application node, and use the dependent application node whose application processing time ratio reaches the preset threshold as the target abnormal application node corresponding to the fault. At this time, some of the target abnormal application nodes may be fault locations, and some application nodes may not be fault locations. Therefore, each target abnormal application node is subjected to the next step of fault cause analysis. If the fault cause is analyzed, the target abnormal application node is the final fault location. If the fault cause cannot be analyzed, the target abnormal application node is regarded as an application node where no abnormality occurs, not a fault location. Among them, the larger the application processing time ratio, the greater the probability of determining it as a fault location. Therefore, this embodiment needs to sort the dependent application nodes whose application processing time ratio reaches the preset threshold according to the descending rule, obtain the sorted target abnormal application nodes, and then perform the next step of fault cause analysis on the target abnormal application nodes in the order of arrangement.

[0064] In this embodiment, the application processing time ratio of the dependent application node is the proportion of the application processing time of the dependent application node in the application processing time of the initial abnormal application node, and is calculated as follows: application processing time ratio of the dependent application node = application processing time of the dependent application node / application processing time of the initial abnormal application node × 100%.

[0065] S103: Based on the pre-built vertical dependency relationship, obtain target monitoring data corresponding to the target abnormal application node from the full-stack data, and use the target monitoring data to analyze the failure cause of the target abnormal application node.

[0066] This embodiment needs to pre-build a vertical dependency relationship based on the full-stack data collected in real time, and the vertical dependency relationship includes the dependency relationship between each application node and the operating environment of each application node. Specifically, the system monitoring data in the full-stack data also includes host resource monitoring data, container resource monitoring data, process operation status data, process log data, process business concurrent data, upgrade maintenance work order data, etc. In this embodiment, each application node has a specific operating environment, and the operating environment data of the application node is established with the application node. First, the corresponding relationship between the application node and the running process must be determined according to the relevant data (such as port, IP address, etc.) of the application node contained in the network request data in the full-stack data, and then the corresponding relationship between the host and the running process is determined according to the host resource monitoring data and the data related to the process on the host, and then the corresponding relationship between the host and the application node can be determined. Based on the corresponding relationship between the host, the running process and the application node, the dependency relationship between the operating environment of the application node and the application node can be determined. Among them, the host / container resource monitoring data and the running process of the application node are bidirectionally affected, and the process resource usage data directly affects the host / container resources, and the host / container resources will restrict the process resource allocation and cause the process to be abnormal.

[0067] After determining the target abnormal application node, this embodiment needs to obtain the target monitoring data corresponding to the target abnormal application node from the full stack data according to the pre-built vertical dependency relationship. Specifically, the vertical dependency relationship includes the dependency relationship between each application node and the operating environment of the application node. Therefore, it is first necessary to query the operating environment on which the target abnormal application node depends as the target operating environment according to the vertical dependency relationship, and then retrieve the data corresponding to the target operating environment from the full stack data, and use these data as the target monitoring data corresponding to the target abnormal application node. For example, according to the vertical dependency relationship, determine the process, host, etc. on which the target abnormal application node depends, and then query the relevant data of the process and host on which the target abnormal application node depends from the full stack data, such as host resource monitoring data, process running status data, process log data, CPU usage data in the host, memory usage data in the host, storage usage data in the host, IO, network, and hard disk resource occupancy data in the host, etc.

[0068] After this embodiment obtains the target monitoring data corresponding to the target abnormal application node, it is necessary to determine the cause of the failure of the target abnormal application node through data analysis of the target monitoring data. When performing data analysis on the target monitoring data, the cause of the failure cannot be directly analyzed through the target monitoring data. It is necessary to use the target monitoring data to calculate the intermediate information, and then perform a failure analysis on the target abnormal application node through the intermediate information. Among them, the intermediate information includes: current interface response time, interface response time average, interface response time in the same period, current business volume, business volume in the same period, predicted response time and process usage data, etc.

[0069] Further, using the target monitoring data, analyzing the failure cause of the target abnormal application node specifically includes the following steps:

[0070] First, based on the target monitoring data, determine the intermediate information corresponding to the target abnormal application node.

[0071] In this embodiment, by analyzing the target monitoring data, the intermediate information corresponding to the target abnormal application node determined includes the current interface response time, the average interface response time, the interface response time of the same period, the current business volume, the business volume of the same period, the predicted response time and the process usage data. The specific determination method of each intermediate information is as follows:

[0072] The current interface response time is the current response time of the interface corresponding to the target abnormal application node, and the current interface response time is directly obtained from the target monitoring data.

[0073] The interface response time average includes the current time average and the interface average response time. The current time average is the average of the current interface response time and several interface response times before the current interface response time. The several interface response times are all recorded in the target monitoring data. For example, the current time average is the average of 10 interface response times. The calculation rule of the current time average is: X n10 =(X t +X t-1 +X t-2 +X t-3 +X t-4 +X t-5 +...+X t-9 ) / 10,X n10 Indicates the current time average, X t Indicates the current interface response time, X t-1 Indicates the last collected interface response time of the current interface response time, X t-2 Represents X t-1 The interface response time of the last collection, X t-3 Represents X t-2The interface response time of the last collection, X t-4 Represents X t-3 The interface response time of the last collection, X t-5 Represents X t-4 The interface response time of the last collection, X t-9 Represents X t-8 The interface response time of the last collection. The average interface response time is the average time of the interface response time within a fixed period (for example, 1 month), that is, the average value of all interface response times corresponding to the target abnormal application nodes collected within a fixed period is calculated as the average interface response time. Among them, the current time average value and the average interface response time are mainly different in time span. The span of the average interface response time is relatively large, which can eliminate short-term fluctuations. In addition, this embodiment can also calculate the standard deviation based on the response time of each interface within a fixed period (for example, 1 month) and the average interface response time, and then judge whether the current time average value is out of time through the current time average value, the average interface response time and the standard deviation. Specifically, the standard deviation calculation rule is: Among them, x1~x n represents the response time of each interface within a fixed period (for example, 1 month), n represents the number of interface response times collected within a fixed period (for example, 1 month), and x p Indicates the average response time of the interface. n10- X p ||<2σ is used to determine whether the current time average value is within the normal distribution range. If it is true, it means that the current time average value is within the normal distribution range and belongs to the normal range. If it is not true, it means that the current time average value is out of time. Among them, X n10 represents the average value of the previous time, x p represents the average response time of the interface, and σ represents the standard deviation.

[0074] The interface response time of the same period is the average interface response time of the same period as the average interface response time, where the same period refers to the same time point in history, for example, today's 9:00-9:30 and yesterday's 9:00-9:30 are the same period. The calculation method of the interface response time of the same period is the same as that of the average interface response time, that is, the average time of the interface response time in the fixed period before the fixed period for calculating the average interface response time and in the same period as the fixed period for calculating the average interface response time. For example, if the fixed period for the average interface response time is June 4 to June 20, then the fixed period for calculating the average interface response time can be May 4 to May 20.

[0075] The current business volume is the current TPS, that is, the current number of interface requests per second. The calculation rule is the total number of currently counted interface requests / the time length of the statistical data. For example, if there are 200 interface requests within 5 seconds, then the current business volume is 200 / 5=40 times / s.

[0076] The business volume of the same period refers to the business volume in the same time period as the current business volume. For example, if the current business volume is the business volume from 9:00 to 9:30 today, then the business volume of the same period can be the business volume from 9:00 to 9:30 yesterday. The calculation method of the business volume of the same period is the same as that of the current business volume.

[0077] The predicted response time is mainly affected by business volume, hardware resources, network resources, etc. When the business does not change, the response time conforms to the linear change law. Therefore, the linear regression method can be used to predict the interface response time based on the business volume, and the predicted interface response time is used as the predicted response time. The calculation formula for the predicted response time is: t =b0+w1×x1+w2×x2+w3×x3+w4×x4+w5×x5+w6×x6, where r t represents the predicted response time, x1 represents the current business volume, x2 represents the current process CPU usage, x3 represents the memory usage, x4 represents the number of log bytes per second, x5 represents the IO rate of the host where the process corresponding to the target abnormal application node is located, and x6 represents the network rate of the host where the process corresponding to the target abnormal application node is located. The coefficients b0, w1, w2, w3, w4, w5, and w6 need to be updated regularly through data machine learning. The trigger condition for data machine learning is preferably set to valid operating data after each system upgrade, with 10,000 data items and a running time of 2 weeks as the training set. Valid operating data refers to data with an error rate of <1% per second.

[0078] Process usage data includes host resource usage, processes with high CPU and memory usage and usage data, resource usage data of high-resource processes in the same period, and the growth rate of process resource usage.

[0079] Second, based on the pre-set fault type judgment rules, the fault type is identified for the intermediate information corresponding to the target abnormal application node, and the fault cause of the target abnormal application node is determined.

[0080] After analyzing the target monitoring data and determining the above-mentioned intermediate information corresponding to the target abnormal application node, this embodiment needs to identify the fault type of the intermediate information corresponding to the target abnormal application node based on a pre-set fault type judgment rule, so as to determine the fault cause of the target abnormal application node.

[0081] Specifically, first, if the fault is an interface error fault, log data is read from the target monitoring data to determine whether the system has been upgraded. If it is determined that the system has been upgraded, it is determined that the cause of the fault of the target abnormal application node may be caused by the upgrade.

[0082] If the fault is an interface response timeout, etc., first determine whether the current interface response time is consistent with the interface response time mean. If the current interface response time is consistent with the interface response time mean, determine that the fault cause of the target abnormal application node includes a long-term slow interface response. Among them, the current interface response time can be compared with the current time average value in the interface response time mean, and can be compared with the interface average response time in the interface response time mean. Preferably, the current interface response time can be compared with the interface average response time in the interface response time mean. In addition, judging whether the current interface response time is consistent with the interface response time mean is by judging whether the time difference between the current interface response time and the interface response time mean is within a preset time difference range. If it is within the preset time difference range, then judge that the current interface response time is consistent with the interface response time mean. If it is not within the preset time difference range, then judge that the current interface response time is inconsistent with the interface response time mean.

[0083] If it is determined that the current interface response time is inconsistent with the interface response time mean, then continue to determine whether the interface response time of the same period is consistent with the interface average response time in the interface response time mean. If the interface response time of the same period is consistent with the interface average response time, then determine that the interface response time of the same period is the same as the interface average response time. Wherein, judging whether the interface response time of the same period is consistent with the interface average response time is by judging whether the time difference between the interface response time of the same period and the interface average response time is within a preset time difference range. If it is within the preset time difference range, then judge that the interface response time of the same period is consistent with the interface average response time. If it is not within the preset time difference range, then the interface response time of the same period is inconsistent with the interface average response time.

[0084] After judging whether the interface response time of the same period is consistent with the average interface response time, it is then judged whether the current business volume is consistent with the business volume of the same period. The judgment of whether the current business volume is consistent with the business volume of the same period is made by judging whether the difference between the current business volume and the business volume of the same period is within the preset business volume difference range. If it is within the preset business volume difference range, it is judged that the current business volume is consistent with the business volume of the same period. If it is not within the preset business volume difference range, it is judged that the current business volume is inconsistent with the business volume of the same period. If it is judged that the current business volume is consistent with the business volume of the same period, it means that the business has not increased. If it is judged that the current business volume is inconsistent with the business volume of the same period, it is judged whether the current interface response time is consistent with the predicted response time. If it is judged that the current interface response time is consistent with the predicted response time, it is determined that the cause of the failure of the target abnormal application node includes slow response due to business growth. If it is judged that the current interface response time is inconsistent with the predicted response time, it means that the failure is affected by other factors. Among them, judging whether the current interface response time is consistent with the predicted response time is by judging whether the time difference between the current interface response time and the predicted response time is within a preset time difference range. If it is within the preset time difference range, then the current interface response time is judged to be consistent with the predicted response time. If it is not within the preset time difference range, then the current interface response time is judged to be inconsistent with the predicted response time.

[0085] After determining that the failure is caused by other factors, use the target monitoring data of the target abnormal application node to perform overall resource estimation. The overall resource estimation directly finds the basic environment monitoring data of the target abnormal application node to check whether there is a high resource usage. The CPU and memory are calculated based on the resource idle rate. The idle rate is lower than 5%, which affects business use. Determine whether the overall resource estimation result is normal. If it is normal, it means that the host resources are normal and the failure is not a business problem and has nothing to do with the host environment. At this time, determine whether the system is upgraded. If it is upgraded, determine that the cause of the failure may be caused by the upgrade. If the overall resource estimation result is abnormal, it means that the fault is affected by the host resources. At this time, it is necessary to first use the CPU usage data and memory usage data in the target monitoring data of the target abnormal application node to determine whether the CPU or memory occupancy is high. The determination of whether the CPU or memory occupancy is high is determined by determining whether the CPU or memory usage reaches a preset threshold. If the CPU usage reaches the first preset threshold, it means that the CPU occupancy is high. If the memory usage reaches the second preset threshold, it means that the memory occupancy is high. If the CPU usage does not reach the first preset threshold, it means that the CPU occupancy is not high. If the memory usage does not reach the second preset threshold, it means that the memory occupancy is not high. In this embodiment, the first preset threshold is preferably set to 40%, and the second preset threshold is preferably set to 30%.

[0086] If it is determined that the CPU and memory usage are not high, the cause of the failure of the target abnormal application node is determined to include exhaustion of overall host resources, high IO, network, and hard disk resource usage. If it is determined that the CPU or memory usage is high, the target monitoring data of the target abnormal application node is used to determine the processes with high CPU and memory usage, and perform differential analysis on the CPU usage and memory usage corresponding to the processes with high CPU and memory usage. If the differential analysis result is >0, the cause of the failure of the target abnormal application node is determined to include high process usage and increased process resource usage. If the differential analysis result is ≤0, the current process usage is determined to be consistent with the process usage during the same period based on the intermediate information of the target abnormal application node. If they are consistent, the cause of the failure of the target abnormal application node is determined to include high process usage but no continuous increase in memory usage.

[0087] Through the above fault analysis process, the fault cause of the target abnormal application node can be determined, so as to locate the fault and determine the root cause of the fault. Based on the above content, it can be determined that the fault causes of the final output are roughly divided into four categories: first, the interface has a long-term slow response; second, the response is slow due to business growth; third, the overall resources of the host are exhausted; fourth, the process occupancy is high. The fault cause of the output target abnormal application node can also carry some corresponding data. For example, for the fault cause of long-term slow response of the interface, the average interface response time of the interface corresponding to the target abnormal application node can be carried; for the fault cause of slow response due to business growth, the current business volume and the current interface response time can be carried; for the fault cause of exhaustion of the overall resources of the host, the utilization rate of the host IO, network, and hard disk resources, memory utilization, and CPU utilization can be carried; for the fault cause of high process occupancy, the host process CPU utilization rate can be carried, etc.

[0088] Through the above introduction, it can be seen that the fault analysis method proposed in the embodiment of the present application determines the dependent application nodes of the initial abnormal application node based on the address of the initial abnormal application node when the fault occurs and the pre-built horizontal dependency relationship; wherein, the horizontal dependency relationship includes the dependency relationship between each application node and the application processing time corresponding to each application node; according to the application processing time corresponding to the dependent application node, the target abnormal application node corresponding to the fault is determined; based on the pre-built vertical dependency relationship, the target monitoring data corresponding to the target abnormal application node is obtained from the full stack data, and the target monitoring data is used to analyze the cause of the fault of the target abnormal application node; the vertical dependency relationship includes the dependency relationship between the operating environment of the application node and the application node, and the full stack data includes network request data and system monitoring data. The technical solution of this embodiment can automatically locate the fault node through the horizontal dependency relationship and the network request data, and automatically analyze the cause of the fault using the vertical dependency relationship, the system monitoring data and the network request data, without the need for manual analysis and processing, thereby reducing labor costs and improving the efficiency and accuracy of fault location analysis.

[0089] As an optional implementation, see Figure 2 As shown, in another embodiment of the present application, a process of constructing a horizontal dependency relationship is disclosed, which specifically includes the following steps:

[0090] S201. Determine network request data based on call link data in the network request.

[0091] In the process of sending a network request by the device, this embodiment collects call link data in the network request. The call link data includes information about the incoming request, information about the outgoing request, message content of the network request, information about the process and host, etc. when two application nodes make a call. This embodiment integrates and standardizes the call link data in the network request, so as to obtain the network request data. Among them, integrating and standardizing the call link data means integrating data from different sources in the call link data, and converting the call link data into a standard format according to a pre-set standard format to achieve data standardization.

[0092] Furthermore, this step specifically includes:

[0093] First, use an APM probe to collect first call link data in a network request, and / or use an eBPF probe to collect second call link data in a network request.

[0094] In this embodiment, the APM probe can be used to collect call link data in the network request, and the eBPF probe can also be used to collect call link data in the network request. Therefore, this embodiment can only use one of the probe methods to collect call link data, or both probes can be used to collect call link data, and finally the call link data collected by the two probes are integrated. Among them, the call link data collected by the APM probe is the first call link data, and the call link data collected by the eBPF probe is the second call link data. The first call link data and the second call link data are both data of the application node call request in the network request.

[0095] Specifically, the specific steps of using the APM probe to collect the first call link data in the network request include: first, using the Linux preload mechanism to intercept and monitor the process startup parameters to determine whether the program is started by Java. If the program is not started by Java, the APM probe will not be added, and the program will not be started by Java, and the APM probe cannot collect the call link data. If it is determined that the program is started by Java, the APM probe will be automatically added to the Java process startup, and then the program will be started.

[0096] The Java Agent then uses a bytecode manipulation library (such as ASM, Javassist, or Byte Buddy) to modify the bytecode of the Java class, enabling the APM probe to dynamically add additional functionality to the application at runtime, such as method call interception, exception capture, performance measurement, etc.

[0097] Then, after receiving the network request, the network request interceptor is used to intercept the network request, generate a trace information, and associate the trace information with the intercepted network request. This trace information will run through the entire network request processing process, including any internal service calls.

[0098] Then, at the entry point of the function (or method), the APM probe can automatically create a new Span information and associate it with the current Trace information. Span represents a specific operation or call in the Trace, which records the start time, end time, duration, and possible metadata (such as errors, exceptions, etc.) of the operation. When the application initiates an external HTTP call, the APM probe can automatically add the identifier of the current Span (such as Span ID and Trace ID) to the HTTP request header. In this way, the called service can recognize that this request is part of a certain Trace and create or associate its own Span accordingly. When the external call returns, the APM probe records the end time of this Span and calculates its duration. Before the function (or method) returns, the APM probe records the end time of the Span and associates it with the Trace. This helps to build a complete call chain and understand the dependencies between different operations.

[0099] Finally, before the network request is processed and is ready to be returned to the client, the APM probe will record the end time of the entire Trace and collect all Span information related to the request. All Trace information and Span information are used as the first call link data collected by the APM probe. Among them, the first call link data includes: process ID, process binding port information, host IP, inbound request IP, inbound request port, inbound request processing time, inbound request remote port, inbound request URL, outbound request IP, outbound request port, outbound request processing time, outbound request remote port, outbound request URL, message content, etc.

[0100] The specific steps of using the eBPF probe to collect the second call link data in the network request include: first, the eBPF probe monitors the network message by setting the kernel filter event. Then, when receiving the network request event, the network layer sender message information and the receiver message information are recorded, and when sending the network request event, the network layer sender message information and the receiver message information are recorded. Finally, the thread information is recorded, and the recorded host information, thread information, sender message information and receiver message information are all used as the second call link data collected by the eBPF probe. Among them, the second call link data includes: process ID, host IP, process binding port information, inbound request IP, inbound request time, inbound request remote port, inbound request URL, outbound request IP, outbound request remote port, outbound request local port, outbound request time, outbound request URL, message content, etc.

[0101] Among them, the content of the second call link data collected by the eBPF probe is more than the content of the first call link data collected by the APM probe. The APM probe captures call link data by modifying the business code, and the eBPF probe obtains call link data by analyzing and acquiring call link data from data packets through network packet capture. The APM probe is invasive and requires code integration. For some application developers who are unwilling to cooperate with the integration, it cannot be implemented. The eBPF probe works independently in the system kernel and the user is unaware. Therefore, if only one of the APM probe and the eBPF probe is used to collect call link data, it is preferred to use the eBPF probe.

[0102] like Figure 3 As shown, the first call link data and the second call link data both include: Figure 3 When application node A calls application node B, application node A sends a request and application node B receives the request. The sent request includes the outgoing request IP, outgoing port, outgoing local port, etc., and the received request includes the incoming request IP, inbound port, inbound local port, etc. When application node B calls application node A, application node B becomes the sender of the request and application node A becomes the receiver of the request.

[0103] Second, based on the first call link data and / or the second call link data, determine the network request data.

[0104] If this embodiment only uses the APM probe to collect the first call link data in the network request, the format of the first call link data is directly converted into a preset standard format to obtain the network request data; if this embodiment only uses the eBPF probe to collect the second call link data in the network request, the format of the second call link data is directly converted into a preset standard format to obtain the network request data; if this embodiment uses both the APM probe to collect the first call link data in the network request and the eBPF probe to collect the second call link data in the network request, the same content in the first call link data and the second call link data is extracted and converted into a preset standard format as the network request data. In this embodiment, the preset standard format is preferably set to json format.

[0105] S202: Based on the network request data, determine the dependency between each application node in the network request data and the application processing time of each application node, and build a horizontal dependency based on the dependency between each application node and the application processing time of each application node.

[0106] First, by determining the network request data, traverse the network request data and calculate the internal processing time of each application node, where the internal processing time of the application node refers to the time it takes for the application node to return data to the request end minus the time it takes for the application node to receive the network request. Then, analyze the message content in the network request data to parse out the URL, Trace ID, Span ID and other contents of the network request. Finally, determine the dependency relationship between the application nodes based on the calling relationship between the application nodes. For example, if application node A calls application node B, it means that application node A depends on application node B.

[0107] like Figure 4 As shown, application node A depends on application node B and application node C, application node B depends on application node D, application node D depends on application node E, and application node C depends on application node f. For http requests, the application node and the URL being accessed are used as the application node, and for non-http requests, the node IP and port are used as the node. For example Figure 4 In the figure, application nodes B, C, and D are application nodes in http requests, that is, B.urlB, C.urlC, and D.urlD. Application nodes E and F are application nodes in non-http requests, that is, xxxx:3306 and xxxx:8889, where xxxx represents the node IP, and 3306 and 8889 represent ports. Figure 4 TimeA represents the application processing time of application node A, TimeB represents the application processing time of application node B, TimeC represents the application processing time of application node C, TimeD represents the application processing time of application node D, TimeDB represents the application processing time of application node E, and TimeF represents the application processing time of application node F. The calculation formula for the application processing time of each application node is as follows:

[0108] TimeA=TimeB+TimeC+InterA

[0109] TimeB=TimeD+InerB

[0110] TimeD=TimeDB+InerD

[0111] TimeDB=InterE

[0112] TimeC=Time f+InterC

[0113] TimeF=InerF

[0114] Among them, InerA represents the internal processing time of application node A, InerB represents the internal processing time of application node B, InerD represents the internal processing time of application node D, InerE represents the internal processing time of application node E, InerC represents the internal processing time of application node C, and InerF represents the internal processing time of application node F.

[0115] The horizontal dependency relationship constructed in this embodiment will be updated after new data is collected.

[0116] In this embodiment, a dependency quick-lookup table can be created according to the constructed horizontal dependency, as shown in Table 1 below:

[0117]

[0118] Table 1

[0119] As shown in Table 1, if the main dependent application node depends on the dependent application node, then the corresponding table is the internal processing time of the dependent application node. If the main dependent application node does not depend on the dependent application node, then the corresponding value in the table is 0. For example, application node A.urlA depends on application node B.urlB, and 50ms is the internal processing time of application node B.urlB. Application node B.urlB does not depend on application node A.urlA, so the corresponding position in the table is 0. The application processing time of application node A.urlA is 50+50+250+50+15+10=425ms. If the application node call is abnormal, the data in the table is stored as -1. The single-row table with the highest percentage of numbers is the one that takes the longest time in the current dependency. For example, the internal processing time of application node C in A.urlA takes more than 50% of the time, which is the most time-consuming application node. When determining the dependent application node of the initial abnormal application node based on the horizontal dependency relationship, it can also be directly determined based on the above-mentioned dependency quick lookup table.

[0120] As an optional implementation, see Figure 5 As shown, in another embodiment of the present application, based on a preset fault type judgment rule, the fault type of the intermediate information corresponding to the target abnormal application node is identified to determine the fault cause of the target abnormal application node, including the following steps:

[0121] S501: Input the intermediate information corresponding to the target abnormal application node into a pre-trained fault type recognition model to obtain the fault cause of the target abnormal application node.

[0122] This embodiment pre-trains a fault type identification model, which can analyze and output the fault cause of the application node based on the intermediate information of the application node. Among them, the output fault types include: slow application node performance, business growth, insufficient host resources, etc. Specifically, the intermediate information corresponding to the target abnormal application node is input into the pre-trained fault type identification model to obtain the fault cause of the target abnormal application node.

[0123] For the training of the fault type identification model, first, the sample abnormal node, the real fault cause corresponding to the sample abnormal node, and the sample monitoring data of the sample abnormal node are collected; then, the sample monitoring data of the sample abnormal node is used to calculate the sample intermediate information corresponding to the sample abnormal node, wherein the sample monitoring data of the sample abnormal node and the target monitoring data corresponding to the target abnormal application node in this embodiment have the same collection method and data content, and the method of determining the sample intermediate information corresponding to the sample abnormal node using the sample monitoring data of the sample abnormal node is the same as the method of determining the intermediate information corresponding to the target abnormal application node using the target monitoring data corresponding to the target abnormal application node, which will not be repeated in this embodiment. Finally, the sample intermediate information corresponding to the sample abnormal node is input into the fault type identification model to obtain the predicted fault cause of the sample abnormal node, and then based on the difference between the predicted fault cause of the sample abnormal node and the real fault cause corresponding to the sample abnormal node, the model parameters of the fault type identification model are adjusted, and the training of the fault type identification model is aimed at minimizing the difference between the predicted fault cause of the sample abnormal node and the real fault cause corresponding to the sample abnormal node.

[0124] As an optional implementation, see Figure 6 As shown, the embodiment of the present application also proposes a fault analysis method, which includes:

[0125] S601: Determine dependent application nodes of the initial abnormal application node based on the address of the initial abnormal application node when a fault occurs and a pre-built horizontal dependency relationship.

[0126] S602: Determine a target abnormal application node corresponding to the fault according to the application processing duration corresponding to the dependent application node.

[0127] S603: Based on the pre-built vertical dependency relationship, obtain target monitoring data corresponding to the target abnormal application node from the full-stack data, and use the target monitoring data to analyze the failure cause of the target abnormal application node.

[0128] S604: When the fault is a host abnormality, obtain host monitoring data corresponding to the host from the full-stack data, and use the host monitoring data to analyze the fault cause corresponding to the fault.

[0129] When a fault occurs, if the fault is a host abnormality fault, this embodiment does not need to determine the target abnormal application node corresponding to the fault according to the application processing time corresponding to the dependent application node of the initial abnormal application node, but directly determines the operating environment of the host according to the pre-built vertical dependency relationship, and then obtains the host monitoring data that matches the host and the host's operating environment from the full stack data. This embodiment can determine the cause of the fault by analyzing the host monitoring data.

[0130] Specifically, this embodiment first analyzes the host CPU usage and the host memory usage in the host monitoring data to determine whether the host CPU usage reaches a first preset threshold and whether the host memory usage reaches a second preset threshold. If the host CPU usage does not reach the first preset threshold or the host memory usage does not reach the second preset threshold, it is determined that the fault cause includes high IO, network, and hard disk resource usage. If the host CPU usage reaches the first preset threshold and the host memory usage reaches the second preset threshold, the process that occupies the most is found according to the process usage in the host monitoring data, and it is determined whether the CPU memory of the process that occupies the most is consistent with the mean, wherein the mean is an average value of the CPU memory of the process collected multiple times within a period of time before. Whether the CPU memory of the process that occupies the most is consistent with the mean is determined by determining whether the difference between the CPU memory of the process that occupies the most and the mean is within a preset difference range. If the difference between the CPU memory of the process that occupies the most and the mean is within the preset difference range, it means that the CPU memory of the process that occupies the most is consistent with the mean. If the difference between the CPU memory of the process that occupies the most and the mean is not within the preset difference range, it means that the CPU memory of the process that occupies the most is inconsistent with the mean.

[0131] If it is determined that the CPU memory of the process that occupies the most is consistent with the average, it is determined that the cause of the fault includes that the CPU memory usage of the process is high but not continuously increasing; if it is determined that the CPU memory of the process that occupies the most is inconsistent with the average, a differential analysis is performed on the CPU memory of the process that occupies the most. If the differential analysis result is ≤0, it is determined that the cause of the fault includes that the CPU memory usage of the process is high but not continuously increasing. If the differential analysis result is >0, it means that the process uses more resources and it is necessary to take the average value of the same period for differential analysis. If the analysis result of the differential analysis of the average value of the same period is ≤0, it is determined that the cause of the fault includes the periodic increase in the use of process resources. If the analysis result of the differential analysis with the average value of the same period is >0, a differential analysis is performed on the business volume of the process in the same period. If the analysis result of the differential analysis of the business volume of the process in the same period is ≤0, it is determined that the cause of the fault includes the periodic increase in the use of process resources. If the analysis result of the differential analysis of the business volume of the process in the same period is >0, it is determined that the cause of the fault includes the increase in resource usage due to business changes.

[0132] The specific implementation manner of steps S601 to S603 in this embodiment is the same as the specific implementation manner of steps S101 to S103 in the above embodiment, and this embodiment will not further elaborate on steps S601 to S603.

[0133] Exemplary Devices

[0134] Accordingly, the present application also provides a fault analysis device, see Figure 7 As shown, the device comprises:

[0135] The node query module 100 is used to determine the dependent application nodes of the initial abnormal application node based on the address of the initial abnormal application node when the fault occurs and the pre-built horizontal dependency relationship; wherein the horizontal dependency relationship includes the dependency relationship between each application node and the application processing time corresponding to each application node;

[0136] The abnormal node determination module 110 is used to determine the target abnormal application node corresponding to the fault according to the application processing time corresponding to the dependent application node;

[0137] The fault analysis module 120 is used to obtain the target monitoring data corresponding to the target abnormal application node from the full-stack data based on the pre-built vertical dependency relationship, and use the target monitoring data to analyze the cause of the failure of the target abnormal application node; the vertical dependency relationship includes the dependency between the operating environment of the application node and the application node, and the full-stack data includes network request data and system monitoring data.

[0138] From the above introduction, it can be seen that the fault analysis device proposed in the embodiment of the present application can automatically locate the fault node through horizontal dependencies and network request data, and automatically analyze the cause of the fault by using vertical dependencies, system monitoring data and network request data. There is no need for manual analysis and processing, which reduces labor costs and improves the efficiency and accuracy of fault location analysis.

[0139] As an optional implementation, another embodiment of the present application discloses that the fault analysis device further includes: a data determination module and a horizontal dependency building module.

[0140] A data determination module, used to determine network request data based on call link data in the network request;

[0141] The horizontal dependency building module is used to determine the dependency between each application node in the network request data and the application processing time of each application node based on the network request data, and to build the horizontal dependency based on the dependency between each application node and the application processing time of each application node.

[0142] As an optional implementation, another embodiment of the present application discloses a data determination module, which is specifically used to:

[0143] Using an APM probe to collect first call link data in a network request, and / or using an eBPF probe to collect second call link data in a network request; wherein the first call link data and the second call link data are both data of an application node call request in a network request;

[0144] Based on the first call link data and / or the second call link data, network request data is determined.

[0145] As an optional implementation, another embodiment of the present application discloses that the abnormal node determination module 110 is specifically used to:

[0146] If the initial abnormal application node returns an exception, the initial abnormal application node is used as the target abnormal application node corresponding to the fault;

[0147] If the initial abnormal application node does not return an exception, the dependent application nodes whose application processing time ratio reaches the preset threshold are sorted in descending order according to the application processing time and used as the target abnormal application nodes corresponding to the fault.

[0148] As an optional implementation, another embodiment of the present application discloses that the fault analysis module 120 includes: a query unit and a data acquisition unit.

[0149] A query unit, used for querying a target operating environment on which a target abnormal application node depends based on a pre-built vertical dependency relationship;

[0150] The data acquisition unit is used to acquire data corresponding to the target operating environment from the full-stack data as target monitoring data corresponding to the target abnormal application node.

[0151] As an optional implementation, another embodiment of the present application discloses that the fault analysis module 120 further includes: an intermediate information determination unit and a fault identification unit.

[0152] An intermediate information determination unit is used to determine the intermediate information corresponding to the target abnormal application node based on the target monitoring data; wherein the intermediate information includes: current interface response time, average interface response time, interface response time during the same period, current business volume, business volume during the same period, predicted response time and process usage data;

[0153] The fault identification unit is used to identify the fault type of the intermediate information corresponding to the target abnormal application node based on a preset fault type judgment rule, and determine the fault cause of the target abnormal application node.

[0154] As an optional implementation manner, another embodiment of the present application discloses a fault identification unit, which is specifically used to:

[0155] The intermediate information corresponding to the target abnormal application node is input into the pre-trained fault type recognition model to obtain the fault cause of the target abnormal application node;

[0156] Among them, the fault type recognition model is based on fault type recognition of sample intermediate information corresponding to sample abnormal nodes, and is trained with the goal of minimizing the difference between the predicted fault cause obtained by fault type recognition and the actual fault cause of the sample abnormal node; the sample intermediate information corresponding to the sample abnormal node is determined based on the sample monitoring data of the sample abnormal node collected in advance.

[0157] As an optional implementation manner, another embodiment of the present application discloses that the fault analysis device further includes: a host fault analysis module.

[0158] The host fault analysis module is used to obtain the host monitoring data corresponding to the host from the full-stack data when the fault is a host abnormality, and use the host monitoring data to analyze the fault cause corresponding to the fault.

[0159] The fault analysis device provided in this embodiment belongs to the same application concept as the fault analysis method provided in the above embodiments of this application, can execute the fault analysis method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the fault analysis method. For technical details not fully described in this embodiment, please refer to the specific processing content of the fault analysis method provided in the above embodiments of this application, and will not be repeated here.

[0160] Exemplary Electronic Devices

[0161] Another embodiment of the present application also provides an electronic device, see Figure 8 As shown, the device includes:

[0162] Memory 200 and processor 210;

[0163] The memory 200 is connected to the processor 210 and is used to store programs;

[0164] The processor 210 is used to implement the fault analysis method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0165] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0166] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0167] A bus may include a pathway that transfers information between components of a computer system.

[0168] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0169] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0170] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0171] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0172] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0173] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0174] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement each step of any fault analysis method provided in the above embodiments of the present application.

[0175] Exemplary computer program products and storage media

[0176] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the fault analysis method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0177] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0178] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the fault analysis method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0179] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0180] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0181] The steps in the methods of each embodiment of the present application can be adjusted in sequence, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0182] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0183] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0184] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0186] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0187] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0188] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0189] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fault analysis method, It is characterized in that include: Determine the dependent application nodes of the initial abnormal application node based on the address of the initial abnormal application node when the fault occurs and the pre-built horizontal dependency relationship; wherein the horizontal dependency relationship includes the dependency relationship between each application node and the application processing time corresponding to each application node; Determine the target abnormal application node corresponding to the fault according to the application processing time corresponding to the dependent application node; Based on the pre-built vertical dependency, the target monitoring data corresponding to the target abnormal application node is obtained from the full-stack data, and the target monitoring data is used to analyze the failure cause of the target abnormal application node; the vertical dependency includes the dependency between the operating environment of the application node and the application node, and the full-stack data includes network request data and system monitoring data.

2. The method according to claim 1, It is characterized in that The process of building the horizontal dependency relationship includes: Determine network request data based on call link data in the network request; Based on the network request data, the dependency relationship between each application node in the network request data and the application processing time of each application node are determined, and based on the dependency relationship between each application node and the application processing time of each application node, a horizontal dependency relationship is constructed.

3. The method according to claim 2, It is characterized in that Based on the call link data in the network request, determine the network request data, including: Using an APM probe to collect first call link data in a network request, and / or using an eBPF probe to collect second call link data in a network request; wherein the first call link data and the second call link data are both data of an application node call request in a network request; The network request data is determined based on the first call link data and / or the second call link data.

4. The method according to claim 1, It is characterized in that Determining a target abnormal application node corresponding to the fault according to the application processing duration corresponding to the dependent application node includes: If the initial abnormal application node returns an exception, the initial abnormal application node is used as the target abnormal application node corresponding to the fault; If the initial abnormal application node does not return an exception, the dependent application nodes whose application processing time ratio reaches a preset threshold are sorted in descending order according to the application processing time and used as the target abnormal application nodes corresponding to the fault.

5. The method according to claim 1, It is characterized in that Based on the pre-built vertical dependency relationship, the target monitoring data corresponding to the target abnormal application node is obtained from the full stack data, including: Based on the pre-built vertical dependency relationship, query the target operating environment on which the target abnormal application node depends; Data corresponding to the target operating environment is obtained from the full-stack data as target monitoring data corresponding to the target abnormal application node.

6. The method according to claim 1, It is characterized in that Analyzing the cause of the failure of the target abnormal application node by using the target monitoring data includes: Based on the target monitoring data, determine the intermediate information corresponding to the target abnormal application node; wherein the intermediate information includes: current interface response time, average interface response time, interface response time during the same period, current business volume, business volume during the same period, predicted response time and process usage data; Based on a preset fault type judgment rule, the fault type is identified for the intermediate information corresponding to the target abnormal application node, and the fault cause of the target abnormal application node is determined.

7. The method according to claim 6, It is characterized in that Based on a preset fault type judgment rule, the fault type of the intermediate information corresponding to the target abnormal application node is identified to determine the fault cause of the target abnormal application node, including: Inputting the intermediate information corresponding to the target abnormal application node into a pre-trained fault type recognition model to obtain the fault cause of the target abnormal application node; Among them, the fault type identification model is based on fault type identification of sample intermediate information corresponding to sample abnormal nodes, and is trained with the goal of minimizing the difference between the predicted fault cause obtained by fault type identification and the actual fault cause of the sample abnormal node; the sample intermediate information corresponding to the sample abnormal node is determined based on sample monitoring data of the sample abnormal node collected in advance.

8. The method according to any one of claims 1 to 7, It is characterized in that Also includes: When the fault is a host abnormality, host monitoring data corresponding to the host is obtained from the full-stack data, and the fault cause corresponding to the fault is analyzed using the host monitoring data.

9. A fault analysis device, It is characterized in that include: A node query module, used to determine the dependent application nodes of the initial abnormal application node based on the address of the initial abnormal application node when the fault occurs and the pre-built horizontal dependency relationship; wherein the horizontal dependency relationship includes the dependency relationship between each application node and the application processing time corresponding to each application node; An abnormal node determination module, used to determine a target abnormal application node corresponding to the fault according to the application processing time corresponding to the dependent application node; A fault analysis module is used to obtain target monitoring data corresponding to the target abnormal application node from the full-stack data based on a pre-built vertical dependency relationship, and use the target monitoring data to analyze the cause of the failure of the target abnormal application node; the vertical dependency relationship includes the dependency between the operating environment of the application node and the application node, and the full-stack data includes network request data and system monitoring data.

10. An electronic device, It is characterized in that include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the fault analysis method according to any one of claims 1 to 8 by running the program in the memory.

11. A storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the fault analysis method according to any one of claims 1 to 8 is implemented.

12. A computer program product, It is characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to implement the fault analysis method according to any one of claims 1 to 8.

Citation Information

Cited By

  • Computer information abnormity identification and early warning method and system

    CN119847878A

  • A method and system for identifying and warning of computer information anomalies

    CN119847878B

  • Fault detection method, computing device, storage medium and program product

    CN121233440A

  • Fault detection method, computing device, storage medium, and program product

    CN121233440B

  • Open source component risk assessment method and apparatus, and electronic device

    CN121615144A