Distributed fault delimiting method, device, equipment, medium and product

Abstract

The invention provides a distributed fault delimiting method which can be applied to the field of financial science and technology. The method comprises the steps that a dynamic full-link topological graph with distributed application as the view angle is constructed according to distributed application data obtained in real time, the full-link topological graph comprises nodes and edges, rationality data of attribute information is obtained according to a preset anomaly detection index, and the nodes are represented to be abnormal in response to the rationality data. Taking the nodes as abnormal nodes; acquiring preset-order adjacent nodes of the abnormal nodes in the full-link topological graph to obtain an abnormal topological graph; and performing fault delimitation on the abnormal topological graph through a graph attention network model generated based on semi-supervised learning training to obtain a root cause node, and performing abnormal matching based on the root cause node to obtain a fault delimitation result. The invention further provides a distributed fault delimiting device, equipment, a medium and a product.

Description

Technical Field

[0001] This application relates to the field of financial technology, specifically to a data processing method for a distributed data system in financial technology, and more specifically to a distributed fault delimitation method, apparatus, device, medium, and product. Background Technology

[0002] With the rapid development of financial technology, and given the complex nature of financial data, financial institutions often rely on large-scale distributed systems for financial data processing. As the transformation to distributed architecture progresses, fault prevention in distributed systems becomes particularly critical.

[0003] While existing operation and maintenance monitoring tools make system operation status more transparent, when a fault actually occurs, operation and maintenance personnel still need to find a breakthrough from the overwhelming amount of monitoring data and alarm information, making it difficult to accurately identify the fault location. This results in low accuracy and inefficiency in distributed fault diagnosis. If existing knowledge graphs are used directly for fault diagnosis, the nodes of existing knowledge graphs are usually updated through the aggregation features of their neighborhoods. However, when aggregating neighbor information, uniform weights are usually used. This method is insufficient to capture the complex relationships in the graph structure, especially the association relationships. Moreover, the nodes and edges of distributed applications are dynamically changing during operation and maintenance. Existing knowledge graph tracking methods cannot capture the changing features in a timely and accurate manner, let alone the changing association relationships between nodes. This leads to untimely fault capture and inaccurate fault location. Furthermore, existing graph convolutional networks mostly use supervised training, which suffers from insufficient training data, resulting in low generalization ability of graph convolutional networks and thus inaccurate fault diagnosis. Summary of the Invention

[0004] In view of the above problems, this application provides a method, apparatus, device, medium and product for distributed fault delimitation.

[0005] According to the first aspect of this application, a distributed fault delimitation method is provided, comprising: constructing a dynamic full-link topology graph from the perspective of the distributed application based on real-time acquired distributed application data using an attention mechanism; the full-link topology graph includes nodes representing attribute information of operation and maintenance components in the distributed application and edges representing the relationships between various operation and maintenance components; the operation and maintenance components include at least one of application groups, data models, minimum deployable units, containers, application middleware, and application resources; obtaining reasonableness data of attribute information based on preset anomaly detection indicators; in response to reasonableness data indicating that a node has an abnormal condition, identifying the node as an abnormal node; obtaining preset-order adjacent nodes of the abnormal nodes in the full-link topology graph to obtain an abnormal topology graph; and performing fault delimitation on the abnormal topology graph using a graph attention network model trained based on semi-supervised learning to obtain root cause nodes, and performing anomaly matching based on the root cause nodes to obtain fault delimitation results.

[0006] According to embodiments of this application, a graph construction model in a graph attention network model is used to construct a dynamic full-link topology graph from the perspective of distributed applications based on real-time acquired distributed application data using an attention mechanism. This includes: acquiring attribute information and association information of distributed application data in real time; extracting features from the attribute information and association information to obtain attribute features and association features; constructing a graph based on the attribute features and association features to obtain a distributed operation and maintenance map; and dynamically assigning adjacency weights to nodes on the distributed operation and maintenance map based on the attribute features and association features using a multi-head attention mechanism to generate a dynamically changing full-link topology graph.

[0007] According to an embodiment of this application, reasonableness data of attribute information is obtained based on preset anomaly detection indicators. In response to the reasonableness data indicating an abnormal state of a node, the node is designated as an abnormal node. This includes: extracting features from the entire link topology graph to obtain a node time-series sequence of relevant attribute information; obtaining the node type of a node in the entire link topology graph; calling the anomaly detection indicator corresponding to the node type; calculating the indicator difference based on the anomaly detection indicator and the node time-series sequence to obtain reasonableness data; wherein the anomaly detection indicator includes at least one of the following: a golden indicator corresponding to an application group; a life-or-death indicator corresponding to a data model and a minimum deployable unit; a container indicator corresponding to a container; and a resource indicator corresponding to application resources; in response to the reasonableness data being greater than the node loss threshold corresponding to the node type, the reasonableness data indicates an anomaly of the node, and the node is designated as an abnormal node.

[0008] According to embodiments of this application, a multi-task learning model in a graph attention network model is used to perform fault delimitation on an abnormal topology graph to obtain root cause nodes, and anomaly matching is performed based on the root cause nodes to obtain fault delimitation results. This includes: extracting features from the abnormal topology graph to obtain abnormal node features containing adjacency weights; aggregating the abnormal node features based on adjacency weights using a semi-supervised graph encoder generated by semi-supervised training in the multi-task learning model to obtain aggregated features, and performing graph encoding based on the aggregated features to generate a fault dependency graph; extracting node dependency information in the fault dependency graph based on adjacency weights using a semi-supervised graph decoder corresponding to the semi-supervised graph encoder, tracing the nodes based on the dependency information to obtain root cause nodes; and classifying the faults based on the root cause nodes and fault behavior information to obtain fault delimitation results containing fault types and corresponding recovery methods.

[0009] According to an embodiment of this application, node tracing based on dependency information to obtain root cause nodes includes: obtaining a multimodal representation of node dependency information based on a preset global attention mechanism; performing anomaly detection on fault nodes in the fault dependency graph sequentially based on upstream and downstream information in the multimodal representation to obtain a fault binary classification result; and performing causal detection based on the fault binary classification result to obtain root cause nodes.

[0010] According to embodiments of this application, anomaly detection is performed sequentially on fault nodes in the fault dependency graph based on upstream and downstream information in the multimodal representation to obtain a fault binary classification result. Causal detection is then performed based on the fault binary classification result to obtain the root cause node. This includes: performing anomaly detection on the access layer of the application group nodes in the fault dependency graph to obtain an access layer fault binary classification result; in response to a negative access layer fault binary classification result, performing anomaly detection on the component layer of the application group nodes to obtain a component layer fault binary classification result; and in response to a negative component layer fault binary classification result, performing anomaly detection on the application middleware to obtain a middleware fault binary classification result. Wherein, if the middleware fault binary classification result is positive, the root cause node is the middleware node; if the middleware fault binary classification result is negative, the access layer fault binary classification result is positive, or the component layer fault binary classification result is positive, the root cause node is the application middleware node. Anomaly detection is performed on the smallest deployable unit in the data model contained in the group nodes to obtain a unit fault binary classification result. If the unit fault binary classification result is positive, causal detection is performed on the smallest deployable unit node to obtain a unit causal result. If the unit causal result representation passes, the unit node type of the smallest deployable unit is obtained. If the unit node type is an application type, the root cause node is the smallest deployable unit node of the application type; if the unit node type is a database type, the root cause node is the smallest deployable unit node of the database type. If the unit fault binary classification result is negative, anomaly detection is performed on the application resource nodes related to the application group nodes to obtain a resource fault binary classification result. If the resource fault binary classification result is positive, causal detection is performed on the resource nodes to obtain a resource causal result. If the resource causal result representation passes, the root cause node is the application resource node.

[0011] According to the embodiments of this application, if the root cause node is an application resource node and the fault behavior information indicates that there is a resource behavior fault, then the fault type is resource layer anomaly; if the root cause node is the smallest deployable unit node of the application type and the fault behavior information indicates that there is an application behavior fault, then the fault type is application anomaly; if the root cause node is the smallest deployable unit node of the database type and the fault behavior information indicates that there is a database behavior fault, then the fault type is database anomaly; if the root cause node is a middleware node and the fault behavior information indicates that there is a middleware behavior fault, then the fault type is middleware anomaly.

[0012] According to embodiments of this application, the basic architecture of the graph construction model and the multi-task learning model is a graph attention network architecture based on an attention mechanism. The pre-training of the multi-task learning model includes: pre-training the graph attention network architecture using pre-acquired normal distributed data in a self-supervised manner, so that the graph attention network architecture learns the local information and neighborhood topology information of the normal distributed data to obtain a graph attention learning architecture; supervising the graph attention learning architecture using pre-acquired faulty distributed data to obtain a weighted loss function for the graph attention learning architecture; and using the optimal graph attention learning architecture as the multi-task learning model until the weighted loss function is lower than a preset weighting threshold, and using the graph encoder in the optimal graph attention learning architecture as a semi-supervised graph encoder.

[0013] The second aspect of this application provides a distributed fault delimitation device, comprising: a link construction module, used to construct a full-link topology graph from the perspective of the distributed application based on real-time acquired distributed application data, the full-link topology graph including nodes representing attribute information of operation and maintenance components in the distributed application and edges representing the relationships between various operation and maintenance components, the operation and maintenance components including at least one of application groups, data models, minimum deployable units, containers, application middleware, and application resources; an anomaly identification module, used to obtain reasonableness data of attribute information based on preset anomaly detection indicators, and in response to reasonableness data indicating that a node has an abnormal condition, the node is identified as an anomaly node; an anomaly topology module, used to obtain preset-order adjacent nodes of the anomaly nodes in the full-link topology graph to obtain an anomaly topology graph; and a root cause detection module, used to perform fault delimitation on the anomaly topology graph through a graph attention network model trained based on semi-supervised learning to obtain root cause nodes, and perform anomaly matching based on the root cause nodes to obtain fault delimitation results.

[0014] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0015] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0016] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0017] The above one or more embodiments have the following beneficial effects: Based on the attention mechanism, a dynamic end-to-end topology map from the perspective of the distributed application is constructed using real-time distributed application data, solving the problem that existing technologies cannot flexibly and effectively capture data information in distributed applications. Based on this end-to-end topology map, the status data of the distributed application can be acquired and displayed dynamically and in a timely manner, improving the real-time monitoring and visualization of the distributed application. First, an anomaly topology map is obtained through preliminary detection based on anomaly detection indicators, and then fault delimitation is performed, thereby reducing the amount of input data for the subsequent graph attention network model. This allows the graph attention network model to quickly and accurately identify the fault location of the distributed application with minimal computational load, facilitating timely fault resolution and improving the fault resolution efficiency of the distributed application. Furthermore, a graph attention network model trained using semi-supervised learning is employed to obtain root cause nodes from the abnormal topology graph in the full-link topology graph generated based on the attention mechanism. Since it can take into account the weights related to each node in the abnormal topology graph, it can accurately obtain the degree of correlation between each node, thus facilitating the efficient and accurate acquisition of root cause nodes. Consequently, it can accurately identify the fault type caused by the root cause node, thereby improving the pertinence and effectiveness of fault resolution and increasing the efficiency and effectiveness of fault resolution. Moreover, semi-supervised learning overcomes the problem of poor accuracy caused by insufficient training samples in supervised learning, ensuring that the generalization ability of the graph attention model is improved with limited training samples, thereby improving the accuracy of obtaining root cause nodes and fault delimitation results in the application stage. Attached Figure Description

[0018] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 The illustrations depict application scenarios of the distributed fault delimitation method, apparatus, device, medium, and program products according to embodiments of this application.

[0020] Figure 2 A flowchart illustrating a distributed fault delimitation method according to an embodiment of this application is shown schematically.

[0021] Figure 3 This illustration shows a schematic diagram of the data flow involved in the distributed fault delimitation method according to an embodiment of this application;

[0022] Figure 4 The illustration shows a schematic diagram of obtaining root cause nodes through sequential anomaly detection in the distributed fault delimitation method according to an embodiment of this application;

[0023] Figure 5 The diagram illustrates a training graph attention network model in a distributed fault delimitation method according to an embodiment of this application.

[0024] Figure 6 A schematic diagram illustrating the structure of a distributed fault delimitation device according to an embodiment of this application is shown; and

[0025] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a distributed fault delimitation method according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0029] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0030] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0031] When a fault actually occurs, either maintenance personnel still need to search for a breakthrough from the overwhelming amount of monitoring data and alarm information, which cannot accurately identify the fault location, resulting in low accuracy and efficiency in distributed fault diagnosis; or they can directly use existing knowledge graphs for fault diagnosis, which is insufficient to capture the complex relationships in the graph structure, especially the association relationships. Moreover, in the operation and maintenance of distributed applications, the nodes and edges are constantly changing. Existing knowledge graph tracking methods cannot capture the changing features in a timely and accurate manner, let alone the changing association relationships between various nodes. This leads to untimely fault capture and inaccurate fault location. Furthermore, most existing graph convolutional networks use supervised training, which suffers from insufficient training data, resulting in low generalization ability of graph convolutional networks and thus inaccurate fault diagnosis.

[0032] This application provides a distributed fault delimitation method, apparatus, device, medium, and product. The distributed fault delimitation method includes: constructing a dynamic end-to-end topology graph from the perspective of the distributed application based on real-time acquired distributed application data using an attention mechanism. The end-to-end topology graph includes nodes representing attribute information of operation and maintenance components in the distributed application and edges representing the relationships between these components. The operation and maintenance components include at least one of application groups, data models, minimum deployable units, containers, application middleware, and application resources; obtaining reasonableness data of attribute information based on preset anomaly detection indicators; identifying nodes as anomalous nodes in response to reasonableness data indicating anomalies; obtaining preset-order adjacent nodes of the anomalous nodes in the end-to-end topology graph to obtain an anomaly topology graph; and performing fault delimitation on the anomaly topology graph using a graph attention network model trained based on semi-supervised learning to obtain root cause nodes, and performing anomaly matching based on the root cause nodes to obtain fault delimitation results.

[0033] Figure 1 The diagram illustrates an application scenario of the distributed fault delimitation method according to an embodiment of this application.

[0034] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105.

[0035] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0037] Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. Network 104 serves as the medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0038] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0039] It should be noted that the distributed fault delimitation method provided in this application embodiment can generally be executed by server 105. Correspondingly, the distributed fault delimitation device provided in this application embodiment can generally be located in server 105. The distributed fault delimitation method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the distributed fault delimitation device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0040] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0041] The following will be based on Figure 1 The described scene, through Figures 2-4 The distributed fault delimitation method according to the embodiments of this application will be described in detail.

[0042] Figure 2 A flowchart illustrating a distributed fault delimitation method according to an embodiment of this application is shown schematically.

[0043] like Figure 2 As shown, the distributed fault delimitation method of this embodiment includes S210~S250, as detailed below:

[0044] Operation S210 constructs a dynamic end-to-end topology graph from the perspective of the distributed application based on real-time acquired distributed application data using an attention mechanism. The end-to-end topology graph includes nodes representing the attribute information of operation and maintenance components in the distributed application and edges representing the relationships between various operation and maintenance components. Operation and maintenance components include at least one of application groups, data models, minimum deployable units, containers, application middleware, and application resources. Operation S220 obtains reasonableness data of attribute information based on preset anomaly detection indicators. In response to reasonableness data indicating that a node has an abnormal condition, the node is designated as an abnormal node. Operation S230 obtains the preset order adjacent nodes of the abnormal nodes in the end-to-end topology graph to obtain an abnormal topology graph. Operation S240 performs fault delimitation on the abnormal topology graph using a graph attention network model trained based on semi-supervised learning to obtain root cause nodes, and performs anomaly matching based on the root cause nodes to obtain fault delimitation results.

[0045] As an example, the first step is to acquire distributed application data in real time. Then, a dynamic end-to-end topology map from the perspective of the distributed application is constructed based on this data. This end-to-end topology map is built from the distributed application data, which includes attribute information and other related data of various operation and maintenance components in the distributed application system, such as application group data, data model data, minimum deployable unit data, container data, application middleware data, and application resource data. This establishes an end-to-end topology map containing attribute information and relationships of at least one of the operation and maintenance components, including application groups, data models, minimum deployable units, containers, application middleware, and application resources. In the end-to-end topology map of the distributed application system, nodes that generate anomalies often exhibit clustering. This is because, in reality, the same event cluster may be caused by a single root cause event. The ultimate goal of this example is to find the root cause node corresponding to the root cause event. This root cause node is actually a node or a node of a certain type within the operation and maintenance components.

[0046] In a more specific example, this end-to-end topology diagram, or simply topology diagram, is a topology diagram in which nodes are delimitable fault domains. These are objects that may fail and have monitoring indicators or alarms. These include group nodes, i.e., nodes corresponding to application groups; SET nodes, i.e., nodes corresponding to data models; middleware nodes, i.e., nodes corresponding to application middleware; container nodes, i.e., nodes corresponding to containers; and nodes corresponding to application resources. The nodes corresponding to application resources can be virtual machine nodes, physical machine nodes, host group nodes, etc. The nodes in the topology graph represent a fault domain in the end-to-end three-dimensional topology. The data structure of a node includes, but is not limited to, the following: Node ID, which is a unique identifier in the graph database corresponding to the end-to-end topology graph and is automatically generated by the graph database during storage; Node Index, a unique identifier under a given set of nodes in the graph database, which can be used to associate corresponding indicator data, alarm data, etc., and is assigned during storage, not automatically generated by the database, such as according to the key generation rules determined by actual conditions; The distributed campus where the node is located; Node Application, which may be shared by multiple applications for application resource nodes; Node Name, with a unified naming convention; Node IP, which is not required, and is empty for group-type nodes; Node Type, such as application group nodes, container nodes, etc.; Node Subtype, such as the smallest deployable unit for database types, which is distinguished by the parameters of the smallest deployable unit, and this parameter is the node subtype; Topology Layer Description, which is used for front-end layer display, as well as node description, etc.

[0047] The edges in the topology graph represent unidirectional relationships between the aforementioned nodes, including: depend_on (dependency relationship); including (inclusion relationship); runs_on (running relationship); and calls_on (calling relationship). Edge attributes include relationship type, weight, and other attributes. For example, the topology graph includes dependency relationships between the application's access layer, composition layer, and component layer; running relationships between containers, virtual machines, and compute nodes / bare metal; inclusion relationships between access groups and Sets (data models) and Sets and Pods (minimum deployable units); and access and calling relationships between Pods.

[0048] In this example, after establishing the full-link topology graph, preliminary anomaly detection can be performed. This involves obtaining reasonable data on attribute information based on preset anomaly detection indicators. In response to reasonable data indicating that a node has an abnormal condition, the node is designated as an anomaly node. This allows the acquisition of preset-order adjacent nodes of the anomaly node in the full-link topology graph, resulting in an anomaly topology graph. Subsequently, a graph attention network model trained based on semi-supervised learning is used to perform fault delimitation on this small amount of anomaly topology graph to obtain the root cause node. Anomaly matching is then performed based on the root cause node to obtain the fault delimitation result.

[0049] Therefore, a real-time full-link topology map of distributed applications is generated, solving the problem that existing technologies cannot flexibly and effectively capture data information in distributed applications. Based on this full-link topology map, the status data of distributed applications can be obtained dynamically and in a timely manner, improving the real-time monitoring and visualization of distributed applications. First, an anomaly topology map is obtained by preliminary detection based on anomaly detection indicators, thereby reducing the amount of input data for the fault delimitation model. This allows the subsequent model to quickly and accurately identify the fault location of the distributed application with minimal computation, so as to resolve the fault in a timely manner and improve the fault resolution efficiency of distributed applications. The graph attention network model trained by semi-supervised learning obtains the root cause node based on the attention mechanism. Since the attention mechanism can take into account the related weights between each node, it can accurately obtain the degree of correlation between each node, thus facilitating the efficient and accurate acquisition of the root cause node, and then obtaining the fault type caused by the root cause node with high precision. This improves the pertinence and effectiveness of fault resolution, and improves the efficiency and effectiveness of fault resolution. Furthermore, semi-supervised learning overcomes the problem of poor accuracy caused by insufficient training samples in supervised learning, ensuring that the limited training samples improve the generalization ability of the graph attention model.

[0050] Figure 3 The diagram illustrates the data flow involved in the distributed fault delimitation method according to an embodiment of this application.

[0051] exist Figure 3 In the illustrated embodiment, a dynamic end-to-end topology graph from the perspective of distributed applications is constructed based on the attention mechanism of the graph construction model in the graph attention network model, using real-time acquired distributed application data. This includes: acquiring attribute information and association information of distributed application data in real time; extracting features from the attribute information and association information to obtain attribute features and association features; constructing a graph based on the attribute features and association features to obtain a distributed operation and maintenance map; and dynamically assigning adjacency weights to nodes on the distributed operation and maintenance map based on the attribute features and association features using a multi-head attention mechanism to generate a dynamically changing end-to-end topology graph.

[0052] As an example, the graph construction model in the graph attention network model constructs a dynamic full-link topology graph from the perspective of distributed applications based on the attention mechanism and the real-time acquired distributed application data. In this example, the process of constructing the full-link graph and obtaining the root cause node are both completed by the graph attention network model based on semi-supervised training, thereby ensuring the scientific nature of the node tracing in the root cause node search. Furthermore, the graph attention network model operates based on the attention mechanism whether constructing the full-link topology graph or fault delimitation, thereby increasing the attention to the degree of correlation between nodes and thus improving the accuracy of fault delimitation.

[0053] In a specific example, this attribute information includes attributes related to at least one of the following dimensions: application group, data model, minimum deployable unit, container, application middleware, and application resource. Examples include the names of the application group, data model, minimum deployable unit, container, application middleware, and application resource, which correspond to the node IDs within the node; the index information of the aforementioned operation and maintenance components, which corresponds to the node indexes within the node and can be used to indicate metric data, alarm data, etc.; distributed campus information, corresponding to the distributed campus of the node; application information, used to describe the application on the corresponding application node; topology layer description information; and operation and maintenance information including operation and maintenance time and address. This association information refers to the dependencies, inclusion relationships, calling relationships, and runtime relationships that exist between the aforementioned application group, data model, minimum deployable unit, container, application middleware, and application resource.

[0054] After obtaining attribute and association information, feature extraction is performed to obtain attribute features and association features. Then, a graph is built based on the attribute and association features to obtain a distributed operation and maintenance map. The weights between different nodes are dynamically adjusted through an attention mechanism to obtain the attention coefficient. Based on the attention coefficient, different adjacency weights are assigned to neighboring nodes to aggregate the information of neighboring nodes and generate a dynamically changing full-link topology map. This allows the full-link topology map to intuitively reflect the closeness of the direct association between each node.

[0055] In this way, by obtaining the full-link graph based on the attention mechanism, adjacency weights are dynamically assigned to nodes, thereby objectively reflecting the correlation between nodes. This allows the subsequent graph attention network model to better capture the relationships between nodes and improve the accuracy of fault location.

[0056] In this embodiment, reasonableness data of attribute information is obtained according to preset anomaly detection indicators. In response to reasonableness data indicating that a node has an abnormal condition, the node is designated as an abnormal node. This includes: extracting features from the full-link topology graph to obtain the node time-series sequence of relevant attribute information; obtaining the node type of the node in the full-link topology graph, calling the anomaly detection indicator corresponding to the node type, and calculating the indicator difference based on the anomaly detection indicator and the node time-series sequence to obtain reasonableness data. The anomaly detection indicators include at least one of the following: the golden indicator corresponding to the application group, the life and death indicator corresponding to the data model and the minimum deployable unit, the container indicator corresponding to the container, and the resource indicator corresponding to the application resources. In response to reasonableness data being greater than the node loss threshold corresponding to the node type, the reasonableness data indicates that the node has an anomaly, and the node is designated as an abnormal node.

[0057] As an example, this describes the process of preliminary anomaly detection. In this process, the "life-or-death indicator" refers to a pre-set indicator that directly relates to whether the data model and the smallest deployable unit are abnormal. The "golden indicator" refers to a pre-set indicator that directly reflects whether the application group is abnormal. The "container indicator" refers to an indicator that directly reflects whether the container is abnormal. The "resource indicator" refers to an indicator that directly reflects whether the application resources are abnormal. The category status data, such as the golden indicator of the application and the life-or-death indicator of the application group, are all time-series data. Taking the golden indicator of the application group as an example, it can be that the difference between the data collection frequency of the application group and the standard collection frequency cannot exceed 1 minute. In this case, the node loss threshold is 1 minute. The standard golden indicator, i.e., the anomaly detection indicator, is assumed to be 5 seconds. If the difference in the indicator data of the application group is 70 seconds, then 70 seconds minus 5 seconds equals 65 seconds, which is greater than the loss threshold of 1 minute. Therefore, the node of the application group is an abnormal node. And so on. Based on the setting of anomaly detection indicators, nodes with anomalies can be quickly and initially identified, shortening the time for anomaly node determination.

[0058] In this embodiment, a multi-task learning model within a graph attention network model is used to perform fault delimitation on an anomaly topology graph to obtain root cause nodes. Anomaly matching is then performed based on these root cause nodes to obtain fault delimitation results. This includes: extracting features from the anomaly topology graph to obtain anomaly node features containing adjacency weights; a semi-supervised graph encoder generated through semi-supervised training in the multi-task learning model aggregating the anomaly node features based on adjacency weights to obtain aggregated features, and then performing graph encoding based on these aggregated features to generate a fault dependency graph; a semi-supervised graph decoder corresponding to the semi-supervised graph encoder extracting node dependency information from the fault dependency graph based on adjacency weights, tracing the nodes based on the dependency information to obtain root cause nodes; and classifying the faults based on the root cause nodes and fault behavior information to obtain fault delimitation results containing fault types and corresponding recovery methods.

[0059] As an example, the process of obtaining root cause nodes and performing anomaly matching based on root cause nodes to obtain fault delimitation results is executed by the multi-task learning model in the graph attention network model. Specifically, firstly, feature extraction is performed on the anomaly topology graph to obtain anomaly node features. Then, the semi-supervised graph encoder generated by the semi-supervised training in the multi-task learning model aggregates the anomaly node features based on the aforementioned adjacency weights to obtain aggregated features. Graph encoding is then performed based on these aggregated features to generate a fault dependency graph. This process of aggregating anomaly node features based on adjacency weights is essentially an application of the attention mechanism. By capturing the different weights between nodes, the degree of dependency between nodes is obtained, thereby generating a fault dependency graph with adjacency weights that represent the degree of dependency. Then, the semi-supervised graph decoder corresponding to the semi-supervised graph encoder extracts node dependency information from the fault dependency graph based on adjacency weights. Node tracing is performed based on the dependency information to obtain root cause nodes. Finally, the root cause nodes and fault behavior information are classified to obtain fault delimitation results that include fault types and corresponding recovery methods.

[0060] Based on this, the encoder generated by semi-supervised training performs feature aggregation and node dependency information extraction based on adjacency weights, thereby objectively and scientifically capturing the degree of dependency and correlation between each node, thus improving the basis for node tracing and improving the accuracy and precision of root cause node localization.

[0061] Figure 4 This diagram illustrates the process of sequentially detecting anomalies to obtain root cause nodes in a distributed fault delimitation method according to an embodiment of this application.

[0062] exist Figure 4 In the embodiment shown, node tracing based on dependency information to obtain root cause nodes includes: obtaining a multimodal representation of node dependency information based on a preset global attention mechanism; performing anomaly detection on fault nodes in the fault dependency graph sequentially based on upstream and downstream information in the multimodal representation to obtain fault binary classification results; and performing causal detection based on the fault binary classification results to obtain root cause nodes.

[0063] As an example, a multimodal representation is obtained based on a global attention mechanism, which is also a type of attention mechanism. By dynamically allocating weights, it can more effectively focus on the degree of dependency between nodes in the node dependency information. The resulting multimodal representation can reflect the different degrees of dependency between nodes. Based on this, sequential detection is performed according to the fault binary classification results to improve the orderliness and comprehensiveness of anomaly detection, as well as the objectivity of root cause tracing.

[0064] Figure 4 This demonstrates a detailed operational procedure for sequentially performing anomaly detection to obtain root cause nodes. Figure 4In the illustrated embodiment, anomaly detection is performed sequentially on fault nodes in the fault dependency graph based on upstream and downstream information in the multimodal representation to obtain a fault binary classification result. Causal detection is then performed based on the fault binary classification result to obtain the root cause node. This includes: performing anomaly detection on the access layer of the application group nodes in the fault dependency graph to obtain an access layer fault binary classification result; responding to a negative access layer fault binary classification result, performing anomaly detection on the component layer of the application group nodes to obtain a component layer fault binary classification result; responding to a negative component layer fault binary classification result, performing anomaly detection on the application middleware to obtain a middleware fault binary classification result; wherein, if the middleware fault binary classification result is positive, the root cause node is the middleware node; if the middleware fault binary classification result is negative, the access layer fault binary classification result is positive, or the component layer fault binary classification result is positive, the root cause node is the application group node. Anomaly detection is performed on the smallest deployable unit in the data model contained in the group node to obtain a binary classification result of unit faults. If the binary classification result is positive, causal detection is performed on the smallest deployable unit node to obtain a unit causal result. If the unit causal result representation passes, the unit node type of the smallest deployable unit is obtained. If the unit node type is an application type, the root cause node is the smallest deployable unit node of the application type; if the unit node type is a database type, the root cause node is the smallest deployable unit node of the database type. If the binary classification result of unit faults is negative, anomaly detection is performed on the application resource nodes related to the application group node to obtain a binary classification result of resource faults. If the binary classification result of resource faults is positive, causal detection is performed on the resource nodes to obtain a resource causal result. If the resource causal result representation passes, the root cause node is the application resource node.

[0065] As an example, the positive result is "yes", which is expressed as "YES" in English. Figure 4 In this context, a negative "Y" indicates "no," which is represented by "NO" in English. Figure 4 The “N” in Figure 4The phrase "traverse the pod nodes with calls_on" means traversing the smallest deployable unit node with a call relationship. A pod node is the smallest deployable unit node, and "pod resource exception" indicates an exception in the smallest deployable unit node. An application pod is the smallest deployable unit node for the application type, and a database pod is the smallest deployable unit node for the database type. Therefore, if a group-type node has an exception, the process iterates through the Set nodes contained in the group. The Set nodes are application group nodes. Then, the Pod nodes contained in the Set are traversed, and an exception is checked on each Pod node. If no exception is detected, the Pods with calls_on for that Pod node are traversed. If no anomalies are detected after traversing all nodes, a group-type node anomaly, i.e., an application anomaly, is output. If a Pod node anomaly is detected, causal detection is performed on the Pod. If a causal relationship exists, an application-type anomaly or a database-type anomaly is output, depending on whether the node type is an application Pod or a database Pod. If no causal relationship exists, anomaly detection and causal detection are performed on the resource-type nodes based on the runs_on relationship between the Pod and the resource-type nodes. If resource-layer nodes also have anomalies, the fault scope is resource-layer anomaly. At the same time, the type of resource anomaly is determined based on the fault detection results, including anomalies of CPU, disk, network, memory, etc.

[0066] This process can comprehensively cover the location status of root cause nodes, improving the scientific rationality and accuracy of root cause node location.

[0067] In this embodiment, if the root cause node is an application resource node and the fault behavior information indicates a resource behavior fault, the fault type is resource layer anomaly; if the root cause node is the smallest deployable unit node of the application type and the fault behavior information indicates an application behavior fault, the fault type is application anomaly; if the root cause node is the smallest deployable unit node of the database type and the fault behavior information indicates a database behavior fault, the fault type is database anomaly; if the root cause node is a middleware node and the fault behavior information indicates a middleware behavior fault, the fault type is middleware anomaly.

[0068] As an example, the fault behavior information may not necessarily represent the existence of one type of fault, but may represent the existence of multiple faults. However, as long as the fault information represents a fault that matches the root cause node and the root cause node type is determined, the fault type can be determined according to the determination method in this embodiment. Based on this, the fault type can be quickly mapped from the root cause node, which makes it easier for operation and maintenance personnel to quickly resolve distributed faults and improve the overall operation and maintenance efficiency of distributed applications.

[0069] Figure 5 This diagram illustrates the training of a graph attention network model in the distributed fault delimitation method of this application.

[0070] In this embodiment, the basic architecture of the graph construction model and the multi-task learning model is a graph attention network architecture based on an attention mechanism. The pre-training of the multi-task learning model includes: pre-training the graph attention network architecture using pre-acquired normal distributed data in a self-supervised manner, so that the graph attention network architecture learns the local information and neighborhood topology information of the normal distributed data to obtain the graph attention learning architecture; supervising the graph attention learning architecture using pre-acquired faulty distributed data to obtain a weighted loss function for the graph attention learning architecture; and using the optimal graph attention learning architecture as the multi-task learning model until the weighted loss function is lower than a preset weighting threshold, and using the graph encoder in the optimal graph attention learning architecture as a semi-supervised graph encoder.

[0071] As an example, both graph construction models and multi-task learning models are essentially graph attention network architectures. Graph Attention Networks (GATs) are a graph neural network architecture based on attention mechanisms. GATs aggregate neighbor node information in graph structure data by introducing attention mechanisms, thereby enabling more flexible and effective learning of the relationships between nodes.

[0072] In this example, the graph attention network architecture is trained using a combination of self-supervised and supervised methods. First, the graph attention network architecture is pre-trained using self-supervised methods on pre-acquired normal distributed data, allowing it to learn local and neighborhood topological information from the normal distributed data, resulting in a graph attention learning architecture. Then, the graph attention learning architecture is supervisedly trained using pre-acquired faulty distributed data, yielding a weighted loss function for the graph attention learning architecture. The multi-task learning model trained in this way, compared to existing graph convolutional networks, can improve the attention to node dependencies, thereby improving fault location accuracy. Furthermore, the pre-training process for the multi-task learning model is defined. This process is first based on self-supervised learning, then on semi-supervised learning (supervised plus self-supervised). The graph encoder in the optimal graph attention learning architecture is used as the semi-supervised graph encoder, and the graph decoder in the optimal graph attention learning architecture is used as the corresponding semi-supervised graph encoder. This semi-supervised training method can improve model accuracy while addressing the problems of limited training data and data imbalance, reducing the requirements for the training dataset and improving model accuracy and generalization ability under the constraint of a limited dataset.

[0073] As described above, the distributed fault delimitation method provided in this embodiment can solve the problem that existing technologies cannot flexibly and effectively capture data information in distributed applications. Based on the full-link topology map, it can acquire and display the status data of distributed applications in a timely and dynamic manner, improving the real-time monitoring and visualization of distributed applications. First, an anomaly topology map is obtained through preliminary detection based on anomaly detection indicators, and then fault delimitation is performed, thereby reducing the amount of input data for the subsequent graph attention network model. This allows the graph attention network model to quickly and accurately identify the fault location of the distributed application with minimal computational load, facilitating timely fault resolution and improving the fault resolution efficiency of distributed applications. Furthermore, it employs a method based on semi-supervised learning training. The graph attention network model obtains root cause nodes from the abnormal topology graph in the full-link topology graph generated based on the attention mechanism. Since it can take into account the weights related to each node in the abnormal topology graph, it can accurately obtain the degree of correlation between each node, thus facilitating the efficient and accurate identification of root cause nodes. Consequently, it can accurately identify the fault type caused by the root cause node, thereby improving the pertinence and effectiveness of fault resolution and increasing the efficiency and effectiveness of fault resolution. Furthermore, semi-supervised learning overcomes the problem of poor accuracy caused by insufficient training samples in supervised learning, ensuring that the generalization ability of the graph attention model can be improved with limited training samples, thereby improving the accuracy of root cause node acquisition and fault delimitation results in the application stage.

[0074] Based on the above-described distributed fault demarcation method, this application also provides a distributed fault demarcation device. The following will combine... Figure 6 The device is described in detail.

[0075] Figure 6 A schematic block diagram of a distributed fault delimitation device according to an embodiment of this application is shown.

[0076] like Figure 6 As shown, the distributed fault demarcation device 600 of this embodiment includes a link construction module 610, an anomaly initial identification module 620, an anomaly topology module 630, and a root cause detection module 640.

[0077] The link construction module 610 can perform operation S210, which constructs a dynamic full-link topology graph from the perspective of distributed applications based on the attention mechanism and the real-time acquired distributed application data. The full-link topology graph includes nodes that represent the attribute information of the operation and maintenance components in the distributed application and edges that represent the relationships between the operation and maintenance components. The operation and maintenance components include at least one of the following: application group, data model, minimum deployable unit, container, application middleware, and application resources.

[0078] The anomaly identification module 620 can perform operation S220, obtain reasonableness data of attribute information according to preset anomaly detection indicators, and in response to reasonableness data indicating that the node has an abnormal condition, the node is identified as an abnormal node.

[0079] The abnormal topology module 630 can perform operation S230 to obtain the preset order adjacent nodes of the abnormal nodes in the full link topology graph and obtain the abnormal topology graph.

[0080] The root cause detection module 640 can perform operation S240, which uses a graph attention network model trained based on semi-supervised learning to perform fault delimitation on the abnormal topology graph to obtain root cause nodes, and performs anomaly matching based on the root cause nodes to obtain fault delimitation results.

[0081] In this embodiment, the link construction module 610 constructs a dynamic full-link topology graph from the perspective of distributed applications based on the attention mechanism of the graph construction model in the graph attention network model and the distributed application data acquired in real time. This includes: acquiring attribute information and association information of distributed application data in real time; extracting features from the attribute information and association information to obtain attribute features and association features; constructing a graph based on the attribute features and association features to obtain a distributed operation and maintenance map; and dynamically assigning adjacency weights to nodes on the distributed operation and maintenance map based on the attribute features and association features using a multi-head attention mechanism to generate a dynamically changing full-link topology graph.

[0082] The anomaly detection module 620 includes: a sequence generation unit, used to extract features from the full-link topology graph to obtain a node time-series sequence with relevant attribute information; an indicator calculation unit, used to obtain the node type of the node in the full-link topology graph, call the anomaly detection indicator corresponding to the node type, and calculate the indicator difference based on the anomaly detection indicator and the node time-series sequence to obtain reasonableness data; wherein, the anomaly detection indicator includes at least one of the following: the golden indicator corresponding to the application group, the life and death indicator corresponding to the data model and the minimum deployable unit, the container indicator corresponding to the container, and the resource indicator corresponding to the application resource; and a loss calculation unit, used to determine that if the reasonableness data is greater than the node loss threshold corresponding to the node type, the reasonableness data indicates that the node is abnormal, and the node is identified as an abnormal node.

[0083] The root cause detection module 640 uses a multi-task learning model in the graph attention network model to perform fault delimitation on the abnormal topology graph to obtain root cause nodes, and performs anomaly matching based on the root cause nodes to obtain fault delimitation results. This includes: extracting features from the abnormal topology graph to obtain abnormal node features containing adjacency weights; aggregating the abnormal node features based on adjacency weights using a semi-supervised graph encoder generated by semi-supervised training in the multi-task learning model to obtain aggregated features, and performing graph encoding based on the aggregated features to generate a fault dependency graph; and extracting node dependency information from the fault dependency graph based on adjacency weights using a semi-supervised graph decoder corresponding to the semi-supervised graph encoder, and tracing the node origin based on the dependency information to obtain... Root cause nodes; based on root cause nodes and fault behavior information, categories are performed to obtain fault delimitation results including fault types and corresponding recovery methods; wherein, node tracing is performed based on dependency information to obtain root cause nodes, including: obtaining a multimodal representation of node dependency information based on a preset global attention mechanism; performing anomaly detection on fault nodes in the fault dependency graph sequentially based on upstream and downstream information in the multimodal representation to obtain a fault binary classification result; and performing causal detection based on the fault binary classification result to obtain root cause nodes; wherein, based on upstream and downstream information in the multimodal representation, performing anomaly detection on fault nodes in the fault dependency graph sequentially to obtain a fault binary classification result; and performing causal detection based on the fault binary classification result to obtain root cause nodes. The points include: performing anomaly detection on the access layer of application group nodes in the fault dependency graph to obtain access layer fault binary classification results; in response to a negative access layer fault binary classification result, performing anomaly detection on the component layer of application group nodes to obtain component layer fault binary classification results; in response to a negative component layer fault binary classification result, performing anomaly detection on the application middleware to obtain middleware fault binary classification results; wherein, when the middleware fault binary classification result is positive, the root cause node is the middleware node; when the middleware fault binary classification result is negative, the access layer fault binary classification result is positive, or the component layer fault binary classification result is positive, the root cause node is the middleware node. Anomaly detection is performed on the smallest deployable unit in the data model contained in the point to obtain a binary classification result of unit faults. If the binary classification result of the unit fault is positive, causal detection is performed on the smallest deployable unit node to obtain a unit causal result. In response to the successful characterization of the unit causal result, the unit node type of the smallest deployable unit is obtained. If the unit node type is an application type, the root cause node is the smallest deployable unit node of the application type; if the unit node type is a database type, the root cause node is the smallest deployable unit node of the database type. If the binary classification result of the unit fault is negative, anomaly detection is performed on the application resource nodes related to the application group node to obtain a binary classification result of resource faults.In this case, if the resource fault binary classification result is positive, causal detection is performed on the resource node to obtain the resource causal result. If the resource causal result representation passes, then the root cause node is the application resource node.

[0084] In one specific embodiment, if the root cause node is an application resource node and the fault behavior information indicates a resource behavior fault, then the fault type is resource layer anomaly; if the root cause node is the smallest deployable unit node of the application type and the fault behavior information indicates an application behavior fault, then the fault type is application anomaly; if the root cause node is the smallest deployable unit node of the database type and the fault behavior information indicates a database behavior fault, then the fault type is database anomaly; if the root cause node is a middleware node and the fault behavior information indicates a middleware behavior fault, then the fault type is middleware anomaly.

[0085] In this embodiment, the basic architecture of the graph construction model and the multi-task learning model is a graph attention network architecture based on an attention mechanism. The pre-training of the multi-task learning model includes: pre-training the graph attention network architecture using pre-acquired normal distributed data in a self-supervised manner, so that the graph attention network architecture learns the local information and neighborhood topology information of the normal distributed data to obtain a graph attention learning architecture; supervising the graph attention learning architecture using pre-acquired faulty distributed data to obtain a weighted loss function for the graph attention learning architecture; and using the optimal graph attention learning architecture as the multi-task learning model until the weighted loss function is lower than a preset weighting threshold, and using the graph encoder in the optimal graph attention learning architecture as a semi-supervised graph encoder.

[0086] Furthermore, according to embodiments of this application, any plurality of modules among the link construction module 610, the anomaly initialization module 620, the anomaly topology module 630, and the root cause detection module 640 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the link construction module 610, the anomaly initialization module 620, the anomaly topology module 630, and the root cause detection module 640 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in hardware or firmware, or in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the link construction module 610, the anomaly initial identification module 620, the anomaly topology module 630, and the root cause detection module 640 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0087] It should be noted that the implementation methods, technical problems solved, functions achieved, and technical effects of each module in the device embodiment are the same as or similar to the implementation methods, technical problems solved, functions achieved, and technical effects of each corresponding step in the method embodiment, and will not be repeated here.

[0088] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a distributed fault delimitation method according to an embodiment of this application.

[0089] like Figure 7 As shown, an electronic device 700 according to an embodiment of this application includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0090] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0091] According to embodiments of this application, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0092] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0093] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0094] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the distributed fault delimitation method provided in the embodiments of this application.

[0095] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0096] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0097] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0098] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0100] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A distributed fault localization method, characterized in that, The method comprises: Based on the attention mechanism, a dynamic full-link topology graph from the perspective of a distributed application is constructed according to real-time distributed application data, the full-link topology graph comprises nodes representing attribute information of operation and maintenance components in the distributed application and edges representing association relationships between the operation and maintenance components, the operation and maintenance components comprising at least one of an application group, a data model, a minimum deployable unit, a container, application middleware and application resources; According to a preset anomaly detection index, rationality data of the attribute information is obtained, and in response to the rationality data representing that the node has an abnormal condition, the node is taken as an abnormal node; In the full-link topology graph, preset order adjacent nodes of the abnormal node are obtained, and an abnormal topology graph is obtained; A graph attention network model generated based on semi-supervised learning training is used to perform fault localization on the abnormal topology graph to obtain a root cause node, and abnormal matching is performed based on the root cause node to obtain a fault localization result.

2. The distributed fault delimitation method according to claim 1, characterized in that, According to the graph atlas construction model in the graph attention network model, a dynamic full-link topology graph from the perspective of a distributed application is constructed based on the attention mechanism according to real-time distributed application data, which comprises: Real-time acquisition of attribute information and association information of distributed application data; Feature extraction is performed on the attribute information and the association information to obtain attribute features and association features; Based on the attribute features and the association features, a graph atlas is constructed to obtain a distributed operation and maintenance map; Based on a multi-head attention mechanism, adjacent weights are dynamically assigned to nodes on the distributed operation and maintenance map according to the attribute features and the association features to generate a dynamically changing full-link topology graph.

3. The distributed fault delimitation method according to claim 1, characterized in that, According to a preset anomaly detection index, rationality data of the attribute information is obtained, and in response to the rationality data representing that the node has an abnormal condition, the node is taken as an abnormal node, which comprises: Feature extraction is performed on the full-link topology graph to obtain a node time sequence related to the attribute information; The node type of the node in the full-link topology graph is obtained, an anomaly detection index corresponding to the node type is called, and index difference calculation is performed according to the anomaly detection index and the node time sequence to obtain rationality data; wherein the anomaly detection index comprises at least one of a golden index corresponding to the application group, a life and death index corresponding to the data model and the minimum deployable unit, a container index corresponding to the container, and a resource index corresponding to the application resources; In response to the rationality data being greater than a node loss threshold value corresponding to the node type, the rationality data represents that the node has an abnormality, and the node is taken as an abnormal node.

4. The distributed fault delimitation method according to claim 2, characterized in that, A multi-task learning model in the graph attention network model is used to perform fault localization on the abnormal topology graph to obtain a root cause node, and abnormal matching is performed based on the root cause node to obtain a fault localization result, which comprises: Feature extraction is performed on the abnormal topology graph to obtain abnormal node features comprising adjacent weights; The semi-supervised graph encoder generated based on semi-supervised training in the multi-task learning model performs feature aggregation on the abnormal node features based on the adjacency weights to obtain aggregated features, and performs graph encoding according to the aggregated features to generate a fault dependency graph; The semi-supervised graph decoder corresponding to the semi-supervised graph encoder extracts node dependency information in the fault dependency graph based on the adjacency weights, traces the root cause node according to the dependency information, and obtains the root cause node. According to the root cause node and the fault behavior information, a category is divided to obtain a fault bounding result containing a fault type and a recovery means corresponding to the fault type.

5. The distributed fault delimitation method according to claim 4, characterized in that, The root cause node is obtained by tracing the node according to the dependency information, and includes: Based on the preset global attention mechanism, a multi-modal representation of the node dependency information is obtained; Based on the upstream and downstream information in the multi-modal representation, abnormal detection is sequentially performed on the fault nodes in the fault dependency graph to obtain a fault binary classification result, and causal detection is performed based on the fault binary classification result to obtain the root cause node.

6. The distributed fault delimitation method according to claim 5, characterized in that, Based on the upstream and downstream information in the multi-modal representation, abnormal detection is sequentially performed on the fault nodes in the fault dependency graph to obtain a fault binary classification result, and causal detection is performed based on the fault binary classification result to obtain the root cause node, including: Abnormal detection is performed on the access layer of the application group node in the fault dependency graph to obtain an access layer fault binary classification result, in response to the access layer fault binary classification result being a negative result, abnormal detection is performed on the component layer of the application group node to obtain a component layer fault binary classification result, in response to the component layer fault binary classification result being a negative result, abnormal detection is performed on the application middleware to obtain a middleware fault binary classification result; wherein, In the case that the middleware fault binary classification result is a positive result, the root cause node is a middleware node; In the case that the middleware fault binary classification result is a negative result, the access layer fault binary classification result is a positive result, the access layer fault binary classification result is a positive result, or the component layer fault binary classification result is a positive result, abnormal detection is performed on the smallest deployable unit in the data model contained in the application group node to obtain a unit fault binary classification result; wherein, In the case that the unit fault binary classification result is a positive result, causal detection is performed on the smallest deployable unit node to obtain a unit causal result, and in response to the unit causal result being passed, the unit node type of the smallest deployable unit is obtained; wherein, if the unit node type is an application type, the root cause node is a smallest deployable unit node of the application type, and if the unit node type is a database type, the root cause node is a smallest deployable unit node of the database type; In a case where the unit fault binary classification result is a negative result, an application resource node related to the application group node is subjected to abnormality detection to obtain a resource fault binary classification result; in response to the resource fault binary classification result being a positive result, the resource node is subjected to causal detection to obtain a resource causal result, and in response to the resource causal result being pass, the root cause node is the application resource node.

7. The distributed fault localization method of claim 6, wherein, if the root cause node is an application resource node and the fault behavior information indicates that there is a resource behavior fault, the fault type is a resource layer anomaly; if the root cause node is an application type minimum deployable unit node and the fault behavior information indicates that there is an application behavior fault, the fault type is an application anomaly; if the root cause node is a database type minimum deployable unit node and the fault behavior information indicates that there is a database behavior fault, the fault type is a database anomaly; if the root cause node is a middleware node and the fault behavior information indicates that there is a middleware behavior fault, the fault type is a middleware anomaly.

8. The distributed fault delimitation method according to claim 4, characterized in that, The basic architecture of the graph construction model and the multi-task learning model is a graph attention network architecture based on an attention mechanism; wherein the multi-task learning model is pre-trained, including: pre-training the graph attention network architecture based on self-supervised manner using pre-acquired normal distributed data, so that the graph attention network architecture learns local information and neighborhood topology information of normal distributed data to obtain a graph attention learning architecture; supervised training the graph attention learning architecture using pre-acquired fault distributed data to obtain a weighted loss function for the graph attention learning architecture; until the weighted loss function is lower than a preset weighted threshold, the optimal graph attention learning architecture is taken as a multi-task learning model, and a graph encoder in the optimal graph attention learning architecture is taken as the semi-supervised graph encoder.

9. A distributed fault localization apparatus, characterized by, The apparatus comprises: a link construction module configured to construct a full-link topology graph from a distributed application perspective according to real-time acquired distributed application data, the full-link topology graph comprising nodes representing attribute information of operation and maintenance components in the distributed application and edges representing association relationships between the operation and maintenance components, the operation and maintenance components comprising at least one of an application group, a data model, a minimum deployable unit, a container, application middleware, and an application resource; an abnormality preliminary identification module configured to acquire rationality data of the attribute information according to a preset abnormality detection index, and in response to the rationality data indicating that the node has an abnormal condition, taking the node as an abnormal node; an abnormality topology module configured to acquire preset order adjacency nodes of the abnormal node in the full-link topology graph to obtain an abnormality topology graph; a root cause detection module configured to perform fault localization on the abnormality topology graph through a graph attention network model generated based on semi-supervised learning training to acquire a root cause node, and perform abnormality matching based on the root cause node to obtain a fault localization result.

10. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-8.

11. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instructions, when executed by a processor, implement the steps of the method according to any one of claims 1-8.

12. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions, when executed by a processor, implement the steps of the method according to any one of claims 1-8. The computer program or instructions, when executed by a processor, implement the steps of the method according to any one of claims 1-8.

Images