Fault positioning method, device, equipment, medium and program product

Abstract

The invention discloses a fault positioning method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining key index information of a node, a container group and a service which are involved when a target cluster breaks down; performing knowledge graph construction based on the nodes, the container groups and the key index information of the services to obtain a cluster topological graph; performing fault analysis and positioning on the cluster topological graph by adopting a target analysis algorithm to obtain one or more of a fault node, a fault container group and a fault service in the cluster topological graph and fault causes of one or more of the fault node, the fault container group and the fault service; wherein the target analysis algorithm is used for performing connectivity analysis or shortest path calculation or centrality analysis on the cluster topological graph.

Description

Technical Field

[0001] This application relates to the field of communication and network technology, and in particular to a fault location method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the widespread adoption of cloud-native technologies, Kubernetes (K8s) has become a core platform for managing large-scale containerized applications. In a K8s cluster environment, complex dependencies exist between nodes, container groups, services, and the network. When a system failure occurs, quickly locating the root cause becomes crucial for ensuring stable system operation. However, existing fault analysis methods lack effective automated analysis tools when faced with complex cluster topologies and dynamically changing system states, resulting in low efficiency and limited accuracy in fault location. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application provide a fault location method, apparatus, device, storage medium, and program product.

[0004] The fault location method provided in this application includes: Obtain key metrics information about the nodes, container groups, and services involved when the target cluster experiences a failure; A knowledge graph is constructed based on the key indicator information of the nodes, container groups, and services to obtain a cluster topology graph. The target analysis algorithm is used to perform fault analysis and location on the cluster topology graph to obtain one or more of the faulty nodes, faulty container groups, and faulty services in the cluster topology graph, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services; wherein, the target analysis algorithm is used to perform connectivity analysis, shortest path calculation, or centrality analysis on the cluster topology graph.

[0005] The fault location device provided in this application includes: The acquisition unit is used to acquire key indicator information of the nodes, container groups, and services involved when a target cluster fails. The construction unit is used to construct a knowledge graph based on the key indicator information of the nodes, container groups, and services to obtain a cluster topology graph. The analysis and localization unit is used to perform fault analysis and localization on the cluster topology graph using a target analysis algorithm, to obtain one or more of the faulty nodes, faulty container groups, and faulty services in the cluster topology graph, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services; wherein, the target analysis algorithm is used to perform connectivity analysis, shortest path calculation, or centrality analysis on the cluster topology graph.

[0006] The processing device provided in this application includes a processor and a memory. The memory is used to store computer programs, and the processor is used to call and run the computer programs stored in the memory to execute any of the above-described fault location methods.

[0007] The computer-readable storage medium provided in this application embodiment is used to store a computer program that causes a computer to execute any of the above-described fault location methods.

[0008] The computer program product provided in this application includes computer program instructions that cause a computer to execute any of the above-described fault location methods.

[0009] In the technical solution of this application embodiment, key indicator information of nodes, container groups, and services involved when a target cluster fails is obtained; a knowledge graph is constructed based on the key indicator information of nodes, container groups, and services to obtain a cluster topology graph; a target analysis algorithm is used to perform fault analysis and location on the cluster topology graph to obtain one or more of the faulty nodes, faulty container groups, and faulty services in the cluster topology graph, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services; wherein, the target analysis algorithm is used to perform connectivity analysis, shortest path calculation, or centrality analysis on the cluster topology graph. In this way, by collecting key indicator information from the target cluster, a cluster topology graph containing nodes, container groups, services, and their dependencies is constructed, enabling a structured representation of the complex node dependencies in the target cluster. Then, graph analysis algorithms such as connectivity analysis, shortest path calculation, and centrality analysis are used to analyze and process the graph structure in the cluster topology graph, which can quickly identify possible faulty nodes, faulty container groups, and faulty services, and further provide the corresponding causes of faulty nodes, faulty container groups, and faulty services. Thus, by combining dynamic graph modeling with graph algorithms, the dependencies between cluster resources can be displayed more intuitively, and efficient fault location can be achieved through automated analysis, while reducing the uncertainty brought about by subjective experience. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating a fault location method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a K8s cluster fault location method based on graph database and graph computing provided in an embodiment of this application. Figure 3 This is a schematic diagram of the cluster logical topology graph constructed using nodes, container groups, and services as vertices, their relationships as edges, and key indicator information of nodes, container groups, and services as attribute information, as provided in the embodiments of this application. Figure 4 This is a schematic diagram illustrating the principle of the shortest path calculation scenario provided in the embodiments of this application; Figure 5A This is a schematic diagram of the connectivity analysis scenario provided in the embodiments of this application. Figure 1 ; Figure 5B This is a schematic diagram of the connectivity analysis scenario provided in the embodiments of this application. Figure 2 ; Figure 6 This is a schematic diagram of the structure of a fault location device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the processing device provided in the embodiments of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0012] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and they all fall within the protection scope of the embodiments of this application.

[0013] With the rapid development of containerization technology, Kubernetes (K8s) has become the mainstream platform for managing and orchestrating containers. A K8s cluster contains numerous nodes, container groups, services, and network connections. As the cluster size and complexity increase, locating and troubleshooting becomes increasingly difficult. Traditional fault detection and root cause analysis typically rely on monitoring tools and log analysis techniques to manually locate and analyze problems, providing solutions based on subjective experience. This requires manual investigation of various K8s resource dependencies and health status, such as tracing the call chain between web services and the database, determining the scope of service chains potentially affected by a faulty node, locating critical cluster nodes, and assessing overload conditions.

[0014] In addition, related technologies propose a cloud-native fault analysis method based on knowledge graphs. This method locates cloud-native faults by utilizing the construction and analysis of knowledge graphs. It involves acquiring raw data, constructing knowledge graphs, and performing node detection on the constructed graph data through anomaly detection models. The similarity between abnormal nodes and their replica nodes is calculated, and the root cause of the fault is located based on the similarity. The source of the fault is determined by comparing the differences in their behavior or attributes.

[0015] However, the nodes, container groups, services, network, and storage components in a Kubernetes cluster have highly complex dependencies. In large-scale production environments, a single cluster may contain hundreds or thousands of nodes, tens of thousands, or even more container groups, and the system state can change at any time. Manually locating internal faults in a Kubernetes cluster is inefficient and highly complex. Subsequent fault location and resolution rely on subjective human experience, making it difficult to quickly pinpoint and resolve problems. In situations involving long dependency chains, frequent state changes, and insufficient monitoring data reliability, manual operations are time-consuming and lagging, making it difficult to analyze multi-layered dependency issues and hindering continuous learning and optimization of operational strategies.

[0016] The aforementioned methods for fault analysis of cloud-native systems based on knowledge graphs often rely on a large amount of prior knowledge or complex model training, which has high requirements for data quality and accuracy, making it difficult to complete multi-dimensional fault location tasks in real time and efficiently.

[0017] To address the aforementioned technical issues, this application proposes a fault location method. By acquiring key indicator information of nodes, container groups, and services involved when a target cluster fails, a cluster topology graph reflecting the cluster's resource dependencies and state characteristics is constructed. The method automatically employs target analysis algorithms such as connectivity analysis, shortest path calculation, or centrality analysis to intelligently analyze the cluster topology graph, thereby quickly locating potential faulty nodes, container groups, and services, and further determining the cause of the failure. This method can significantly improve the efficiency of fault diagnosis in large-scale Kubernetes clusters under complex dependency scenarios, reduce operational costs, and enhance system stability and reliability.

[0018] To facilitate understanding of the technical solutions of the embodiments of this application, the technical solutions of this application are described in detail below through specific embodiments. The above-mentioned related technologies are optional solutions and can be arbitrarily combined with the technical solutions of the embodiments of this application, all of which fall within the protection scope of the embodiments of this application. The embodiments of this application include at least some of the following contents.

[0019] This application proposes a fault location method, which can be executed by a fault location device. The fault location device can be an operation and maintenance system or a monitoring platform. Furthermore, the operation and maintenance system or monitoring platform can be a cloud-native monitoring tool, a K8s management platform, etc. Figure 1 This is a flowchart illustrating a fault location method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps: Step 101: Obtain key metrics information of the nodes, container groups, and services involved when the target cluster fails.

[0020] In this embodiment of the application, when the target cluster fails, the key indicator information of the nodes, container groups and services involved are all core resource types in the target cluster. Therefore, obtaining the key indicator information of the relevant nodes, container groups and services through the fault-triggered collection method can provide a reliable basis for subsequent fault location.

[0021] In this context, a node refers to a physical or virtual computing unit in the target cluster used to run containerized applications. Each node has attributes such as a unique identifier, resource usage status, and network status. A container group is the smallest deployable unit in the target cluster, typically containing one or more containers. These containers share storage and network namespaces, and each container group is run by its associated node and is bound to services through tags to form a dependency relationship. A service is an abstract interface in the target cluster used to define the access policy of a container group. It provides a stable access endpoint by associating with container groups, supports load balancing and traffic management, and there may be call chains between services, forming a call dependency relationship.

[0022] In some embodiments, key metrics information for nodes, container groups, and services includes one or more of the following: resource information of one or more nodes, resource information of one or more container groups, resource information of one or more services, dependencies between one or more nodes and one or more container groups, dependencies between one or more container groups and one or more services, connectivity between one or more nodes, and invocation relationships between one or more services.

[0023] Specifically, node resource information includes CPU utilization, memory utilization, storage utilization, network status, etc. Node resource information reflects the health status and load of the node. For example, if the CPU utilization of a node reaches 90%, this may indicate that the node is at risk of overload.

[0024] The resource information of a container group includes container group status, container status, CPU and memory usage, network request volume, bandwidth, latency, number of restarts, etc. The various indicators in the resource information of a container group can help determine whether the container group is running normally. For example, when a container group restarts frequently, it indicates that the container group has a stability problem.

[0025] The resource information of a service includes the service name, service identifier, list of associated container groups, health status, etc. The resource information of a service can be used to determine whether the service is available. For example, if a service is in an unhealthy state for a long time, it may be that the container group on which the service depends has failed.

[0026] The dependency relationship between nodes and container groups refers to the logical mapping relationship of which node a container group runs on. It is very important for understanding the distribution and scheduling strategy of container groups. For example, a container group can only run on a specific node. When that specific node fails, the affected container group can be quickly identified based on the dependency relationship between the node and the container group.

[0027] The dependency relationship between container groups and services refers to which container groups a service is bound to through a tag selector. This determines how the service routes traffic to the corresponding container group. For example, a front-end service may depend on multiple back-end service container groups. When one back-end service container group becomes unavailable, the front-end service can still continue to provide services through the remaining back-end service container groups.

[0028] The connectivity between nodes refers to whether nodes can communicate with each other. It is usually represented by a network topology diagram, which can be used to determine whether there are network partitions or broken links. For example, in a large-scale cluster, some nodes may be unable to communicate due to network failures, which may lead to service interruption.

[0029] The call relationship between services refers to the logical relationship between services through APIs or other means. The call relationship between services can form a complex call chain, which can be used to analyze the dependencies between services and the fault propagation path. For example, the front-end service calls the middleware service, and the middleware service calls the database service. If the database service fails, it will affect the entire call chain.

[0030] Step 102: Construct a knowledge graph based on key metrics information of nodes, container groups, and services to obtain a cluster topology graph.

[0031] In this embodiment, a knowledge graph is a form of knowledge organization represented by a graph structure, which consists of entities (vertices), relationships (edges), and attributes. By treating the nodes, container groups, and services involved when the target cluster fails as vertices in the graph, the dependencies between nodes, container groups, and services (such as subordination, invocation, and connectivity) as edges, and the relevant performance metrics of nodes, container groups, and services (such as network latency and error rate) as attribute values, a complete cluster topology graph can be constructed. For example, in a microservice architecture financial system, front-end services may depend on middleware services, and middleware services may depend on database services. Therefore, the dependencies between front-end services and middleware services, and between middleware services and database services, can be mapped to edges in the graph.

[0032] It's important to note that the cluster topology graph is a dynamically updated graph structure. As the cluster configuration changes (such as container group scheduling, node addition, service scaling, etc.), the vertices and edges in the cluster topology graph will also be updated accordingly. For example, when a new node joins the cluster and starts several container groups, the cluster topology graph will automatically add corresponding vertices and edges and update the attribute values ​​of these vertices and edges to reflect the latest cluster state.

[0033] In some embodiments, step 102 specifically includes steps 1021-1024: Step 1021: Designate one or more of the following as one or more vertices: one or more nodes, one or more container groups, or one or more services.

[0034] In this context, a vertex is the basic unit in the graph structure, representing resource entities in the target cluster, such as nodes, container groups, and services. Each vertex can represent a resource instance, and its attributes store detailed information about the resource instance, such as state, CPU utilization, and memory utilization. As the fundamental building blocks of a knowledge graph, vertices describe the mapping and dependency relationships between resources in the target cluster.

[0035] By treating one or more of the nodes, container groups, and services involved in a target cluster failure as one or more vertices, the distribution of resource entities can be intuitively represented, facilitating the subsequent construction of the cluster topology graph and graph computation analysis.

[0036] Step 1022: Treat one or more of the following as edges: the dependency relationship between one or more nodes and one or more container groups, the dependency relationship between one or more container groups and one or more services, the connectivity relationship between one or more nodes, and the call relationship between one or more services.

[0037] In this context, an edge refers to the logical relationship connecting two vertices in a graph structure. This logical relationship represents the dependency or communication relationships between resources in the target cluster. For example, a container group running on a node indicates a dependent edge between the container group and the node; a service calling another service indicates a calling edge between the two services; and network connectivity between nodes indicates the existence of connecting edges. Edge types can include dependent, calling, and connecting, and each edge can carry attribute information such as call frequency and response time.

[0038] By treating one or more of the following relationships as edges when a target cluster fails: the dependency relationships between one or more nodes and one or more container groups, the dependency relationships between one or more container groups and one or more services, the connectivity relationships between one or more nodes, and the calling relationships between one or more services, the dependency links between resources in the target cluster can be expressed intuitively and clearly, which helps to quickly identify the failure propagation path and critical nodes.

[0039] Furthermore, one or more edges can reflect not only static dependencies but also dynamic behaviors. For example, if service A frequently calls service B within a specific time period, such dynamic calls can be modeled and analyzed using the attributes of one or more edges.

[0040] Step 1023: Use one or more of the following as attribute information: resource information of one or more nodes, resource information of one or more container groups, or resource information of one or more services.

[0041] In this context, attribute information refers to metadata attached to one or more vertices or edges to describe the specific state or behavioral characteristics of a resource. For example, attribute values ​​such as CPU utilization of a node, number of restarts of a container group, and health status of a service can be added to one or more vertices or edges corresponding to the node, container group, or service.

[0042] By using one or more of the following as attribute information—resource information of one or more nodes, resource information of one or more container groups, or resource information of one or more services involved when a target cluster fails—the data dimensions of the graph model can be enriched. This can support subsequent graph algorithm analysis, such as centrality analysis and shortest path calculation, and enable graph algorithms to have stronger analytical capabilities, thereby improving the accuracy and intelligence level of fault diagnosis.

[0043] 1024: Construct a knowledge graph based on one or more vertices, one or more edges, and one or more attribute information to obtain a cluster topology graph.

[0044] Specifically, a structured graph database can be built using graph database tools based on one or more predefined vertices, edges, and their attribute information. This database displays the complex relationships between resources in the target cluster, and the data is updated in real time via an API interface, forming a dynamically changing cluster topology. Based on this cluster topology, fault propagation paths and critical nodes can be quickly identified, ultimately enabling efficient and automated fault analysis and localization.

[0045] In some embodiments, the type of one or more vertices includes one or more of node type, container group type, and service type; the type of one or more edges includes one or more of dependent type, connectivity type, and call type; and the type of one or more attribute information includes one or more of vertex type, edge type, and general type.

[0046] One or more vertices represent entity nodes in the cluster topology graph, used to represent resource objects in the target cluster, such as nodes, container groups, and services. These vertices can be further subdivided into different types, such as node types representing physical or virtual computing resources, container group types representing container instances running on one or more nodes, and service types representing externally exposed access points.

[0047] One or more edges represent relationships between entities in a cluster topology graph, describing the dependencies or connections between different resources. The types of edges include dependent (e.g., a container group belonging to a node), connected (e.g., network connections between nodes), and calling (e.g., communication calls between services). Edges not only represent connections but also carry specific semantic information, helping the system understand the interaction logic between resources.

[0048] One or more attribute information refers to metadata attached to one or more vertices or edges to describe the characteristics of entities or the nature of relationships. One or more attribute information can also be classified by type into vertex type (such as CPU utilization of container groups), edge type (such as response time of call relationships), and general type (such as health status, labels, etc.). Through these one or more attribute information, the data expression capability of the graph can be enriched, providing more comprehensive input for subsequent graph algorithm analysis.

[0049] Based on this, step 1024 specifically includes: Connections between vertices of type node are established based on edges of type connectivity; connections between vertices of type node and vertices of type container group, or between vertices of type container group and vertices of type service, are established based on edges of type dependency; and connections between vertices of type service are established based on edges of type invocation. By attaching vertex type attribute information to one or more of the vertices of node type, vertices of container group type, and vertices of service type, attaching edge type attribute information to one or more of the edges of dependent type, connected type, and calling type, and attaching general type attribute information to one or more of the edges of node type, vertices of container group type, vertices of service type, edges of dependent type, edges of connected type, and edges of calling type, a cluster topology graph is obtained.

[0050] Specifically, firstly, connectivity edges are used to establish connections between vertices of the node type, such as whether two vertices of the node type are network reachable. Secondly, dependency edges are used to establish dependencies between vertices of the node type and vertices of the container group type, or between vertices of the container group type and vertices of the service type, reflecting the hierarchical structure between vertices. Finally, call edges are used to establish call chains between vertices of the service type, demonstrating the functional interactions under the microservice architecture. Simultaneously, various attribute information is attached to the corresponding vertices or edges to enhance the information density and analytical capabilities of the graph. For example, vertex type attributes may include node CPU utilization, memory utilization, storage utilization, etc., edge type attributes may include call response time, error rate, bandwidth, etc., while general type attributes may include health status, labels, etc. Through this structured approach, the resulting cluster topology graph not only intuitively displays the dependencies between resources but also supports the efficient execution of various graph algorithms, thereby enabling rapid fault location and intelligent operation and maintenance decisions.

[0051] Step 103: Use the target analysis algorithm to perform fault analysis and location on the cluster topology graph, and obtain one or more of the faulty nodes, faulty container groups, and faulty services in the cluster topology graph, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services.

[0052] Among them, the target analysis algorithm is used to perform connectivity analysis, shortest path calculation, or centrality analysis on the cluster topology graph.

[0053] In the embodiments of this application, the target analysis algorithm includes, but is not limited to, graph algorithms such as connectivity analysis, shortest path calculation, and centrality analysis. By employing the target analysis algorithm to perform fault analysis and location on the cluster topology graph, abnormal connections, critical paths, or high-influence nodes in the cluster topology graph can be identified, thereby achieving rapid fault location and root cause determination, and obtaining one or more of the faulty nodes, faulty container groups, and faulty services, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services.

[0054] For example, in connectivity analysis scenarios, the system can detect whether there are broken call links in the cluster topology graph and quickly locate the specific fault point; in shortest path calculation scenarios, the system can find the shortest path from the faulty node to the affected node in the cluster topology graph so as to prioritize the repair of critical nodes; in centrality analysis scenarios, the system can identify nodes with high dependencies in the cluster topology graph so as to take risk control measures in advance.

[0055] In some embodiments, step 103 may include the following implementation method: Perform connectivity analysis between the starting and ending vertices in the cluster topology graph to identify one or more vertices with missing connections; Perform fault analysis and location on one or more vertices with lost connections to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the causes of the faults in one or more of these categories.

[0056] Specifically, connectivity analysis is a graph algorithm used to determine whether a path between two vertices is connected, i.e., whether there exists a sequence of edges that allows access from the starting vertex to the ending vertex. If a path is broken, it is considered that there is a disconnect between the two vertices. Connectivity analysis is particularly useful for identifying service unreachability issues caused by network outages, node failures, container group crashes, etc. The starting and ending vertices are two key nodes specified by the user or automatically selected by the system based on the fault scenario. In the target cluster, they typically represent the start and end points of the service call chain. A lost connection vertex refers to an intermediate node on the path between the starting and ending vertices where a valid connection cannot be established; these could be components at the container group, service, or node level. By performing connectivity analysis on one or more paths between the starting and ending vertices in the cluster topology graph, one or more lost connection vertices can be identified, thereby quickly locating the key fault point causing the link to be broken.

[0057] For example, in a microservice architecture, the front service (Front) attempts to access the database (DB) but finds the connection broken. By performing connectivity analysis on the path between Front and DB, it's possible to determine if an intermediate faulty service or container group is in an abnormal state. If the connection between the faulty service and the faulty container group is broken, it indicates that a faulty service or container group has failed, thus affecting the availability of the entire connection.

[0058] After identifying one or more vertices with lost connections, further fault analysis and localization can be performed on these vertices using attribute information from the cluster topology graph. This allows for the identification of one or more specific faulty nodes, container groups, or services, as well as the causes of these faults. Faulty nodes refer to nodes in the target cluster exhibiting abnormalities, such as high CPU utilization or insufficient storage space. Faulty container groups refer to container groups experiencing operational abnormalities, such as frequent restarts, resource usage exceeding limits, or abnormal status. Faulty services refer to service-level issues, such as failed health checks or closed ports. Causes of these faults may include network latency, decreased transmission rates, increased error rates, and resource bottlenecks.

[0059] For example, if a container group is marked as having lost connections in the graph, you can check the container group's CPU utilization, network latency, and other attributes to determine whether the slow response is due to insufficient resources, thus causing the connection loss problem.

[0060] Alternatively, in some embodiments, step 103 may include the following implementation method two: The shortest path is calculated between the starting and ending vertices in the cluster topology graph. Perform fault analysis and location on one or more vertices in the shortest path to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the causes of the faults in one or more of these categories.

[0061] Specifically, shortest path calculation is a graph algorithm used to find the path with the minimum weight between the starting and ending vertices. Weights are typically calculated based on edge metrics such as response time, network latency, and error rate. By performing shortest path calculation on one or more paths between the starting and ending vertices in the cluster topology graph, the shortest path between them can be obtained. Furthermore, by combining attribute information from the cluster topology graph, fault analysis and localization can be performed on one or more vertices in the shortest path to identify specific faulty nodes, faulty container groups, and faulty services, as well as the causes of these faults. In particular, shortest path calculation helps the system prioritize troubleshooting the most critical link, reducing troubleshooting time and improving repair efficiency.

[0062] For example, in a cloud-native application, if a response timeout occurs when the front-end service calls the intermediate service, the system can call the shortest path algorithm to find the shortest call path from Front to Service 1 to Service 2 to DB. It is found that the latency between Service 1 and Service 2 is significantly higher than the normal value, but the latency between Service 2 and DB is normal. Therefore, it can be determined that the source of the fault is Service 1 causing the latency. Then, the system can further determine whether it is caused by insufficient resources or code problems based on logs and network status.

[0063] Alternatively, in some embodiments, step 103 may include the following implementation method three: Perform centrality analysis on the cluster topology graph to obtain one or more vertices with the highest degree centrality; Perform fault analysis and location on one or more vertices with the highest degree centrality to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the causes of the faults in one or more of these categories.

[0064] Specifically, centrality analysis is a method for evaluating the importance of nodes in a graph. Degree centrality is one of the most common centrality metrics, used to measure the number of connections between a node and its neighbors. In a target cluster, nodes with high degree centrality are usually important components with complex dependencies and core functions. For example, a node may host container groups of multiple critical services and have high degree centrality. If this node fails, it will affect the normal operation of multiple downstream services. Therefore, by performing centrality analysis on each vertex in the cluster topology graph, calculating and sorting the degree centrality of each vertex, and selecting one or more vertices with the highest degree centrality, further analysis and location of the one or more vertices with the highest degree centrality can be performed by combining attribute information in the cluster topology graph. This can determine one or more of the specific faulty nodes, container groups, and services, as well as the causes of the failures. In addition, these critical nodes can be identified in advance, and preventive measures, such as increasing redundancy and optimizing resource allocation, can be taken to reduce the risk of failure.

[0065] In some embodiments, fault analysis and localization are performed on one or more vertices with lost connections to obtain one or more of the following: faulty node, faulty container group, and faulty service, as well as the cause of the fault in one or more of these components: Based on one or more of the attribute information of vertex type, edge type, and general type, the performance of one or more vertices with lost connections, and the path between one or more vertices with lost connections and other vertices in the cluster topology graph is performed to obtain the first detection result. Based on the first detection result, fault location is performed on one or more vertices with lost connections to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services.

[0066] Vertex type refers to the classification identifier used in the graph database to represent different resources (such as nodes, container groups, and services) in the target cluster. Each vertex type has different attribute information. For example, a node type vertex may contain attributes such as CPU utilization, memory utilization, and network status; a container group type vertex may contain attributes such as restart count, container status, and resource allocation; and a service type vertex may contain attributes such as health status and call relationships.

[0067] Edge type refers to the connection type in a graph database that represents the dependency or communication relationship between target resources. For example, dependent edges represent the runtime dependency between vertices of container group type and vertices of node type, connected edges represent the network connection between vertices of node type, and call edges represent the access link between vertices of service type. Edge types have their own attribute information, such as network latency, transmission rate, and error rate.

[0068] A generic type refers to common attributes that can be shared by all vertices and edges in a graph database, such as identifier, name, timestamp, and status. Attributes such as identifier, name, timestamp, and status in a generic type can serve as a unified data interface, supporting cross-type queries and analysis.

[0069] Specifically, attribute information of each lost vertex and its connected edges can be extracted based on one or more of the attribute information of vertex type, edge type, and general type. Then, according to a first preset indicator requirement, various indicator data related to the lost vertices and the connection paths (edges) between the lost vertices and other vertices in the cluster topology are quantitatively evaluated to determine whether there is abnormal behavior in these vertices and their connection paths. This yields a series of performance indicators and analytical conclusions (first detection results) after performance testing, which helps identify potential fault sources. For example, by analyzing the resource utilization, response time, and error rate of vertices, it can be determined whether a vertex is overloaded or in a failure state; by analyzing the network latency and bandwidth usage on the path, it can be determined whether there is a communication bottleneck. The first detection result may indicate that the network latency of a certain path is higher than a threshold, or that the CPU utilization of a vertex of a certain container group type is consistently too high. The first preset indicator requirement can be adjusted according to different fault scenarios and is not limited here.

[0070] Subsequently, the performance metrics and analytical conclusions regarding the lost connections and their connection paths in the first detection results can be analyzed. The vertex with the highest degree of anomaly among the lost connections can be identified as a potential fault point, and it can be determined whether a chain reaction of faults exists. Ultimately, one or more of the following can be identified: a faulty node, a faulty container group, or a faulty service. For example, if the CPU utilization of a container group-type vertex among one or more lost connections reaches over 90% and frequently restarts, then that container group-type vertex may be the source of the fault. Or, if a node-type vertex associated with a container group-type vertex among one or more lost connections has high network latency, then the problem with that node-type vertex may simultaneously affect the communication between container group-type vertices and service-type vertices.

[0071] Finally, by combining the attribute information of potential failure points (such as network status and memory usage) and the attribute information of the edges connected to potential failure points (such as the error rate of the call chain), the root cause of the failure can be comprehensively determined, ultimately revealing one or more causes of the failure among the faulty node, the faulty container group, and the faulty service. For example, a failure in a container group type vertex may be caused by a variety of factors, such as insufficient vertex resources for the node type, defects in the code of the container group type vertex itself, or an interruption in the call chain between service type vertices.

[0072] In some embodiments, fault analysis and localization are performed on one or more vertices involved in the shortest path to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the causes of the faults in one or more of the faulty nodes, faulty container groups, and faulty services, including: Perform fault analysis on one or more vertices and one or more edges in the shortest path to obtain abnormal paths; Based on one or more of the attribute information of vertex type, edge type, and general type, performance testing is performed on one or more vertices in the abnormal path, as well as the path between one or more vertices in the abnormal path and other vertices in the shortest path, to obtain a second detection result. Based on the second detection result, fault location is performed on one or more vertices in the abnormal path to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services.

[0073] Specifically, by performing fault analysis on one or more vertices and edges involved in the shortest path, when a sub-path in the shortest path is found to have problems such as excessive latency, increased error rate, or network interruption, the sub-path is marked as an abnormal path. For the identified abnormal paths, the attribute information of each vertex and the edges connected to each vertex in the abnormal path (including attribute information of vertex type, edge type, and general type) is further extracted, such as CPU utilization, memory usage, network latency, error rate, bandwidth, response time, etc., and the various index data of each vertex and its connected edges in the abnormal path are quantitatively evaluated according to the second preset index requirements, resulting in a series of performance indicators and analysis conclusions after performance testing (second detection results). The second preset index requirements can be adjusted according to different fault scenarios and are not limited here.

[0074] Next, the performance indicators and analysis conclusions of each vertex and its connecting edges in the abnormal path can be analyzed in the second detection result. The vertex with the highest degree of abnormality in the abnormal path can be identified as the potential fault point, and it can be determined whether there is a chain of faults. Finally, one or more of the fault nodes, fault container groups, and fault services can be obtained. For example, if the CPU utilization of service 1 is consistently high and the downstream calls of service 1 frequently time out, then service 1 is likely to be a faulty service. Once the potential fault node is identified, the attribute information of the potential fault point (such as network status and memory utilization) and the attribute information of the edges connected to the potential fault point (such as the error rate of the call chain) can be combined to comprehensively determine the root cause of the fault, and finally, one or more of the fault node, fault container group, and fault service can be obtained.

[0075] In some embodiments, fault analysis and localization are performed on one or more vertices with the highest degree centrality to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the causes of the faults in one or more of these categories, including: Based on one or more of the attribute information of vertex type, edge type, and general type, the performance of one or more vertices with the highest degree centrality and the paths between one or more vertices with the highest degree centrality and other vertices in the cluster topology graph is tested to obtain the third test result. Based on the third detection result, fault location is performed on one or more vertices with the highest degree centrality to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services.

[0076] Specifically, after identifying one or more vertices with the highest degree centrality in the cluster topology graph, attribute information of each vertex with the highest degree centrality and the edges connected to each vertex can be extracted based on one or more of the attribute information of vertex type, edge type, and general type. Then, according to the third preset index requirements, the various index data involved in the connection paths (edges) between each vertex with the highest degree centrality and other vertices in the cluster topology graph are quantitatively evaluated, resulting in a series of performance indicators and analytical conclusions (third detection results) after performance testing. The third preset index requirements can be adjusted according to different fault scenarios and are not limited here.

[0077] Next, the performance indicators and analysis conclusions of the vertices with the highest degree centrality and their connected edges in the third detection results can be analyzed. The vertex with the highest degree centrality is identified as the potential fault point, and it is determined whether there is a chain of faults. Finally, one or more of the faulty nodes, faulty container groups, and faulty services can be obtained. For example, if the number of restarts of a container group increases frequently and the response time of the path where the container group is located also increases significantly, it can be inferred that the container group may have a stability problem. Furthermore, the attribute information of the potential fault point (such as network status and memory usage) and the attribute information of the edges connected to the potential fault point (such as the error rate of the call chain) can be combined to comprehensively determine the root cause of the fault, and finally obtain one or more of the faulty nodes, faulty container groups, and faulty services.

[0078] It should be noted that after obtaining one or more of the faulty nodes, faulty container groups, and faulty services, as well as the causes of one or more of the faulty nodes, faulty container groups, and faulty services using the above methods, a large model approach can be used to intelligently generate corresponding solutions based on one or more of the faulty nodes, faulty container groups, and faulty services. Alternatively, a detailed fault report can be generated based on the above information. This report lists the specific faulty objects (such as faulty nodes, faulty container groups, and faulty services) and the causes of each faulty object. This fault report is then fed back to the operations and maintenance personnel so that they can take targeted remedial measures based on the information in the fault report, such as expanding the resources of the nodes, restarting the container groups, and optimizing the code logic of the services.

[0079] In the technical solution of this application embodiment, key indicator information of nodes, container groups, and services involved when a target cluster fails is obtained; a knowledge graph is constructed based on the key indicator information of nodes, container groups, and services to obtain a cluster topology graph; a target analysis algorithm is used to perform fault analysis and location on the cluster topology graph to obtain one or more of the faulty nodes, faulty container groups, and faulty services in the cluster topology graph, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services; wherein, the target analysis algorithm is used to perform connectivity analysis, shortest path calculation, or centrality analysis on the cluster topology graph. In this way, by collecting key indicator information from the target cluster, a cluster topology graph containing nodes, container groups, services, and their dependencies is constructed, enabling a structured representation of the complex node dependencies in the target cluster. Then, graph analysis algorithms such as connectivity analysis, shortest path calculation, and centrality analysis are used to analyze and process the graph structure in the cluster topology graph, which can quickly identify possible faulty nodes, faulty container groups, and faulty services, and further provide the corresponding causes of faulty nodes, faulty container groups, and faulty services. Thus, by combining dynamic graph modeling with graph algorithms, the dependencies between cluster resources can be displayed more intuitively, and efficient fault location can be achieved through automated analysis, while reducing the uncertainty brought about by subjective experience.

[0080] This application also proposes a method for fault location in a Kubernetes cluster based on graph databases and graph computing. Figure 2 This is a flowchart illustrating a Kubernetes cluster fault location method based on graph databases and graph computing, as provided in an embodiment of this application. Figure 2 As shown, the method includes the following steps: Step 201: Collect key metrics information of nodes, container groups, and services in the Kubernetes cluster using the Prometheus tool and custom component modules.

[0081] Specifically, the APIServer, the control plane component of the Kubernetes cluster, is responsible for providing cluster resource information and querying dependencies between them through API interfaces. Cluster health, node resource usage, and logs are collected by the monitoring module (Prometheus and other custom modules). The key metrics for nodes, container groups, and services that need to be collected are shown in Table 1 below.

[0082] Table 1

[0083] Data collection can be performed periodically or triggered by fault alarms. Typically, data collection is initiated when the cluster starts up or undergoes large-scale changes (such as deploying new services or scaling). Utilizing the interfaces provided by the Kubernetes API Server, automated scripts or existing monitoring tools are written to periodically request resource status information from the API Server. The request results can be returned in JSON format, which is then processed to map the dependencies between resources to nodes, edges, and attribute information in a graph database.

[0084] Step 202: Construct a cluster topology graph using nodes, container groups, and services as vertices, their relationships as edges, and key metrics of nodes, container groups, and services as attribute information.

[0085] An integrated graph database (using Neo4j or JanusGraph) is responsible for storing and managing the data. Nodes, container groups, and services in Kubernetes are used as vertices of the graph, and dependencies are used as edges, such as the dependency relationship between container groups and nodes, the dependency relationship between services and container groups, and the connectivity relationships between nodes. Descriptive information such as status, name, resources used, and weight can be used as attributes of the vertices. Any changes to the cluster, such as container group scheduling, adding nodes, or service expansion, are promptly updated in the graph database. The graph database API is used to complete data import and updates, forming a dynamic cluster topology graph. The key schema storage information for constructing the graph is shown in Tables 2-4 below. Table 2 contains information related to the graph vertices, Table 3 contains information related to the graph edges, and Table 4 contains information related to the graph property fields. The final logical topology graph of the cluster, constructed with nodes, container groups, and services as vertices, their interrelationships as edges, and key metrics of nodes, container groups, and services as attributes, is shown below. Figure 3 As shown.

[0086] Table 2

[0087] Table 3

[0088] Table 4

[0089] Step 203: Import the collected data into the graph in real time using the graph database API tool, and use the Loader import tool to import the data into the graph on a scheduled basis.

[0090] Step 204: When a fault occurs, select the corresponding graph algorithm based on the fault scenario.

[0091] Specifically, the graph computing engine, its built-in graph database computing module, or an external graph algorithm framework is responsible for executing graph algorithms to analyze the root causes of failures. When a cluster failure occurs, the system triggers the graph computing engine to perform fault analysis using graph algorithms, mainly involving the following graph algorithms: shortest path algorithm, connectivity analysis algorithm, and degree centrality analysis algorithm.

[0092] Step 205: Determine if the shortest link is needed. If yes, proceed to step 206; otherwise, proceed to step 207.

[0093] Step 206: Select the shortest path algorithm to calculate the shortest link in the cluster topology graph, and proceed to step 211.

[0094] Specifically, the shortest path algorithm is used to find the shortest path from the faulty node to the affected nodes, helping to locate the propagation path of the fault, so as to quickly locate the problem, prioritize the repair of critical nodes and links, reduce the weak links in the system, and improve reliability. For example, Figure 4 The diagram shown is a schematic representation of the shortest path calculation scenario provided in this application embodiment.

[0095] In a microservices scenario, the dependencies between services are extremely complex, and troubleshooting them one by one would be very time-consuming. For example, the front-end component Front experienced significant latency in its call chain to the database. The actual solution was as follows: (1) Collect data and construct a service dependency graph based on the collected data. The call path between services represents the edge, and the response time, network latency, error rate and other indicators are the call edge attributes.

[0096] (2) Call the shortest path algorithm to perform shortest path analysis on the service dependency graph to find the minimum weight path from Front to DB. For example, after calculation, Front->Service1->Service2->DB has the minimum weight: 90+500+100=690 (ms). Therefore, the path corresponding to Front->Service1->Service2->DB is the shortest path.

[0097] (3) At this time, the node links of the shortest path are analyzed and it is found that there is an unexpected delay between service 1 and service 2, but service 2 to DB is normal. Therefore, it is determined that the fault is caused by the delay of service 1.

[0098] (4) By analyzing container group logs, network status, etc., it was found that the cause of the failure was insufficient resources. At this time, the performance problem can be solved by expanding capacity, optimizing code, or adjusting resource allocation.

[0099] Step 207: Determine whether connectivity analysis is needed. If yes, proceed to step 208; otherwise, proceed to step 209.

[0100] Step 208: Select a connectivity analysis algorithm to perform connectivity analysis on the cluster topology graph, and proceed to step 211.

[0101] Specifically, connectivity analysis algorithms are used to determine which nodes or container groups or services have lost connectivity, identifying the potential scope of the failure and helping to troubleshoot and identify service unavailability caused by node or container group failures. For example, Figure 5A The diagram shown illustrates the principle of the connectivity analysis scenario provided in this application embodiment. Figure 1 ,like Figure 5B The diagram shown illustrates the principle of the connectivity analysis scenario provided in this application embodiment. Figure 2 The troubleshooting process for connectivity analysis scenarios is as follows: (1) such as Figure 5B As shown, Service 1 initiates a call to Service 3 via Service 2, at which point the link is broken.

[0102] (2) Collect data and input it into the graph, and construct a service dependency graph based on the collected data.

[0103] (3) Call the connectivity analysis algorithm to perform connectivity analysis on the service dependency graph, and calculate the edges of the call type to quickly locate the specific point of disconnection as container group 4.

[0104] (4) Calculating edges of connected type and edges of dependent type can eliminate the problem of node vertices.

[0105] (5) By calling the attribute information (network latency, transmission speed, error rate) on the edge of the type, the possible causes of disconnection can be further analyzed.

[0106] (5) Design a repair plan based on the cause of the fault and repair the fault.

[0107] Step 209: Determine whether degree centrality analysis is needed. If yes, proceed to step 210; otherwise, return to step 204.

[0108] Step 210: Select a degree centrality analysis algorithm to perform degree centrality analysis on the cluster topology graph, and proceed to step 211.

[0109] Specifically, degree centrality analysis algorithms are used to identify vertices in a graph that are highly dependent or important, and the extent to which these vertices might be affected if they fail, thus enabling proactive risk control. In a Kubernetes cluster scenario, degree centrality can help identify critical vertices (such as important services or computing resources) and provide a basis for fault localization and solution generation. The degree centrality analysis process is as follows: (1) Vertex degree: The degree of a vertex is the number of edges connected to it. For an undirected graph, the degree of a vertex is equal to the number of its directly connected vertices; for a directed graph, it is divided into in-degree (the number of edges pointing to the vertex) and out-degree (the number of edges pointing from the vertex to other vertices). (2) Traverse the nodes, container groups, etc. in the K8s cluster and map them to the vertices in the graph, count the degree of each vertex and generate a degree list.

[0110] (3) Calculate the degree centrality of each vertex based on the degree of each vertex. The degree can be used as the centrality value. Then sort the degree centrality of each vertex and find the vertex with the highest degree centrality from the sorted vertices.

[0111] Furthermore, in a Kubernetes cluster, degree centrality helps administrators identify the most critical vertices, especially in a microservice architecture where some vertices may depend on many others. Degree centrality analysis allows for the rapid identification of services or vertices crucial to cluster stability, facilitating proactive protection and appropriate resource allocation.

[0112] Step 211: Based on the calculated shortest link, connectivity analysis results, or degree centrality analysis results, locate the source of the fault and the cause of the fault.

[0113] Step 212: Take corresponding solutions to resolve the fault based on the source and cause of the fault.

[0114] In the technical solution provided in this application embodiment, K8s cluster indicator data is collected and stored in a graph database to construct a K8s cluster analysis graph. Nodes, container groups, and services are used as graph vertices; connectivity between nodes, dependencies between nodes and container groups, dependencies between container groups and services, and access relationships between services are used as graph edges; and network latency, health status, resource allocation, and other moiré data are used as attribute information for graph edges and vertices. The data in the graph is updated in real-time or periodically to construct a dynamic K8s cluster topology graph. This allows for the processing of complex topologies based on the graph database, especially when the cluster is large. Efficient relational queries can quickly locate relevant nodes, facilitating visual analysis and relational queries. This enables analysis in cases of node or service disconnection. In this scenario, connectivity analysis algorithms are used to analyze the cluster topology graph to identify out-of-circuit fault nodes. In scenarios where fault nodes affect linked networks, shortest path algorithms are used to find the shortest affected link in the cluster topology graph and resolve the fault as quickly as possible. Furthermore, in risk control scenarios, degree centrality algorithms are used to calculate critical nodes in the cluster topology graph to proactively protect nodes and allocate resources, achieving risk warnings and reasonable control over the probability and impact of faults. This allows for efficient handling of complex dependencies between nodes, container groups, and services based on different graph algorithms. By using shortest path algorithms and connectivity analysis, potential fault points and their propagation paths can be quickly identified, reducing troubleshooting time and significantly improving the intelligence of automated fault analysis.

[0115] This application also proposes a fault location device. Figure 6 This is a schematic diagram of the structure of a fault location device provided in an embodiment of this application, as shown below. Figure 6 As shown, the device includes: The acquisition unit 601 is used to acquire key indicator information of the nodes, container groups and services involved when the target cluster fails.

[0116] Building unit 602 is used to construct a knowledge graph based on key indicator information of nodes, container groups and services to obtain a cluster topology graph.

[0117] The analysis and localization unit 603 is used to perform fault analysis and localization on the cluster topology graph using a target analysis algorithm, and to obtain one or more of the faulty nodes, faulty container groups, and faulty services in the cluster topology graph, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services; wherein, the target analysis algorithm is used to perform connectivity analysis, shortest path calculation, or centrality analysis on the cluster topology graph.

[0118] In some embodiments, key metrics information for nodes, container groups, and services includes one or more of the following: resource information of one or more nodes, resource information of one or more container groups, resource information of one or more services, dependencies between one or more nodes and one or more container groups, dependencies between one or more container groups and one or more services, connectivity between one or more nodes, and invocation relationships between one or more services.

[0119] In some embodiments, the construction unit 602 is specifically used for: One or more of the following can be used as one or more vertices: one or more nodes, one or more container groups, or one or more services. One or more of the following can be used as one or more edges: the dependency relationship between one or more nodes and one or more container groups, the dependency relationship between one or more container groups and one or more services, the connectivity relationship between one or more nodes, and the invocation relationship between one or more services. One or more of the resource information of one or more nodes, the resource information of one or more container groups, and the resource information of one or more services are used as one or more attribute information. A knowledge graph is constructed based on one or more vertices, one or more edges, and one or more attribute information to obtain a cluster topology graph.

[0120] In some embodiments, the type of one or more vertices includes one or more of node type, container group type, and service type; the type of one or more edges includes one or more of dependent type, connectivity type, and call type; and the type of one or more attribute information includes one or more of vertex type, edge type, and general type.

[0121] In some embodiments, the construction unit 602 is further specifically used for: Connections between vertices of type node are established based on edges of type connectivity; connections between vertices of type node and vertices of type container group, or between vertices of type container group and vertices of type service, are established based on edges of type dependency; and connections between vertices of type service are established based on edges of type invocation. By attaching vertex type attribute information to one or more of the vertices of node type, vertices of container group type, and vertices of service type, attaching edge type attribute information to one or more of the edges of dependent type, connected type, and calling type, and attaching general type attribute information to one or more of the edges of node type, vertices of container group type, vertices of service type, edges of dependent type, edges of connected type, and edges of calling type, a cluster topology graph is obtained.

[0122] In some embodiments, the analysis and positioning unit 603 is specifically used for: Perform connectivity analysis between the starting and ending vertices in the cluster topology graph to identify one or more vertices with missing connections; Perform fault analysis and localization on one or more vertices with lost connections to identify one or more of the following: faulty node, faulty container group, and faulty service; or, The shortest path is calculated between the starting and ending vertices in the cluster topology graph. Perform fault analysis and location on one or more vertices in the shortest path to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the causes of the faults in one or more of these categories; or, Perform centrality analysis on the cluster topology graph to obtain one or more vertices with the highest degree centrality; Perform fault analysis and location on one or more vertices with the highest degree centrality to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the causes of the faults in one or more of these categories.

[0123] In some embodiments, the analysis and positioning unit 603 is further specifically used for: Based on one or more of the attribute information of vertex type, edge type, and general type, the performance of one or more vertices with lost connections, and the path between one or more vertices with lost connections and other vertices in the cluster topology graph is performed to obtain the first detection result. Based on the first detection result, fault location is performed on one or more vertices with lost connections to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services.

[0124] In some embodiments, the analysis and positioning unit 603 is further specifically used for: Perform fault analysis on one or more vertices and one or more edges in the shortest path to obtain abnormal paths; Based on one or more of the attribute information of vertex type, edge type, and general type, performance testing is performed on one or more vertices in the abnormal path, as well as the path between one or more vertices in the abnormal path and other vertices in the shortest path, to obtain a second detection result. Based on the second detection result, fault location is performed on one or more vertices in the abnormal path to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services.

[0125] In some embodiments, the analysis and positioning unit 603 is further specifically used for: Based on one or more of the attribute information of vertex type, edge type, and general type, the performance of one or more vertices with the highest degree centrality and the paths between one or more vertices with the highest degree centrality and other vertices in the cluster topology graph is tested to obtain the third test result. Based on the third detection result, fault location is performed on one or more vertices with the highest degree centrality to obtain one or more of the faulty nodes, faulty container groups, and faulty services, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services.

[0126] Those skilled in the art should understand that Figure 6 The functions of each unit in the communication device shown can be understood by referring to the relevant description of the aforementioned method. Figure 6 The functions of each unit in the communication device shown can be implemented by a program running on a processor or by specific logic circuits.

[0127] Figure 7 This is a schematic diagram of the processing device provided in an embodiment of this application. The processing device may be a terminal device or a network device. Figure 7 The processing device shown includes a processor 701, which can call and run computer programs from memory to implement the methods in the embodiments of this application.

[0128] Optionally, such as Figure 7 As shown, the processing device may further include a memory 702. The processor 701 can retrieve and run computer programs from the memory 702 to implement the methods described in this embodiment.

[0129] The memory 702 can be a separate device independent of the processor 701, or it can be integrated into the processor 701.

[0130] Optionally, such as Figure 7 As shown, the processing device may also include a transceiver 703, which the processor 701 can control to communicate with other devices. Specifically, it can send information or data to other devices or receive information or data sent by other devices.

[0131] The transceiver 703 may include a transmitter and a receiver. The transceiver 703 may further include an antenna, which may be one or more.

[0132] The processing device may specifically be the fault location device in the embodiments of this application, and the processing device can implement the corresponding processes of the various methods in the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0133] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0134] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0135] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to the processing device in this application embodiment, and the computer program causes the computer to execute the corresponding processes implemented by the various methods in this application embodiment; for brevity, further details are omitted here.

[0136] This application also provides a computer program product, including computer program instructions. This computer program product can be applied to the processing device in this application embodiment, and the computer program instructions cause the computer to execute the corresponding processes implemented by the various methods in this application embodiment; for brevity, further details are omitted here.

[0137] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0140] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0142] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A fault location method, characterized in that, The method includes: Obtain key metrics information about the nodes, container groups, and services involved when the target cluster experiences a failure; A knowledge graph is constructed based on the key indicator information of the nodes, container groups, and services to obtain a cluster topology graph. The target analysis algorithm is used to perform fault analysis and location on the cluster topology graph to obtain one or more of the faulty nodes, faulty container groups, and faulty services in the cluster topology graph, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services; wherein, the target analysis algorithm is used to perform connectivity analysis, shortest path calculation, or centrality analysis on the cluster topology graph.

2. The method according to claim 1, characterized in that, The key metrics information of the nodes, container groups, and services include one or more of the following: resource information of one or more nodes, resource information of one or more container groups, resource information of one or more services, dependencies between one or more nodes and one or more container groups, dependencies between one or more container groups and one or more services, connectivity between one or more nodes, and call relationships between one or more services.

3. The method according to claim 2, characterized in that, The knowledge graph is constructed based on the key indicator information of the nodes, container groups, and services to obtain the cluster topology graph, including: One or more of the one or more nodes, the one or more container groups, and the one or more services are used as one or more vertices; One or more of the following can be used as one or more edges: the dependency relationship between the one or more nodes and the one or more container groups, the dependency relationship between the one or more container groups and the one or more services, the connectivity relationship between the one or more nodes, and the invocation relationship between the one or more services. One or more of the resource information of the one or more nodes, the resource information of the one or more container groups, and the resource information of the one or more services are used as one or more attribute information. A knowledge graph is constructed based on the one or more vertices, the one or more edges, and the one or more attribute information to obtain the cluster topology graph.

4. The method according to claim 3, characterized in that, The type of the one or more vertices includes one or more of node type, container group type, and service type; the type of the one or more edges includes one or more of dependent type, connected type, and invoke type; the type of the one or more attribute information includes one or more of vertex type, edge type, and general type. The process of constructing a knowledge graph based on the one or more vertices, the one or more edges, and the one or more attribute information to obtain the cluster topology graph includes: Based on the edges of the connectivity type, establish connection relationships between vertices of the node type; based on the edges of the dependency type, establish connection relationships between vertices of the node type and vertices of the container group type, or between vertices of the container group type and vertices of the service type; and based on the edges of the call type, establish connection relationships between vertices of the service type. The attribute information of the vertex type is attached to one or more of the vertices of the node type, the vertices of the container group type, and the vertices of the service type. The attribute information of the edge type is attached to one or more of the edges of the dependent type, the edges of the connected type, and the edges of the calling type. The attribute information of the general type is attached to one or more of the vertices of the node type, the vertices of the container group type, the vertices of the service type, the edges of the dependent type, the edges of the connected type, and the edges of the calling type to obtain the cluster topology graph.

5. The method according to claim 4, characterized in that, The step of using a target analysis algorithm to perform fault analysis and location on the cluster topology graph, obtaining one or more of the faulty nodes, faulty container groups, and faulty services in the cluster topology graph, as well as the causes of the faults of one or more of the faulty nodes, faulty container groups, and faulty services, includes: A connectivity analysis is performed between the starting and ending vertices in the cluster topology graph to identify one or more vertices with lost connections. Perform fault analysis and location on one or more vertices of the lost connection to obtain one or more of the faulty node, the faulty container group, and the faulty service, as well as the cause of the fault in one or more of the faulty node, the faulty container group, and the faulty service; or, The shortest path is calculated between the starting vertex and the ending vertex in the cluster topology graph to obtain the shortest path; Perform fault analysis and location on one or more vertices in the shortest path to obtain one or more of the faulty node, the faulty container group, and the faulty service, as well as the cause of the fault in one or more of the faulty node, the faulty container group, and the faulty service; or, Perform a centrality analysis on the cluster topology graph to obtain one or more vertices with the highest degree centrality; Fault analysis and localization are performed on one or more vertices with the highest degree centrality to obtain one or more of the fault node, the fault container group, and the fault service, as well as the fault cause of one or more of the fault node, the fault container group, and the fault service.

6. The method according to claim 5, characterized in that, The step of performing fault analysis and localization on one or more vertices of the lost connection to obtain one or more of the faulty node, the faulty container group, and the faulty service, as well as the cause of the fault in one or more of the faulty node, the faulty container group, and the faulty service, includes: Based on one or more of the attribute information of the vertex type, the attribute information of the edge type, and the attribute information of the general type, performance detection is performed on one or more vertices with lost connections, and the paths between one or more vertices with lost connections and other vertices in the cluster topology graph, to obtain a first detection result. Based on the first detection result, fault location is performed on one or more vertices of the lost connection to obtain one or more of the faulty node, the faulty container group, and the faulty service, as well as the fault cause of one or more of the faulty node, the faulty container group, and the faulty service.

7. The method according to claim 5, characterized in that, The step of performing fault analysis and location on one or more vertices involved in the shortest path to obtain one or more of the faulty nodes, the faulty container groups, and the faulty services, as well as the causes of the faults in the faulty nodes, the faulty container groups, and the faulty services, includes: Fault analysis is performed on one or more vertices and one or more edges involved in the shortest path to obtain abnormal paths; Based on one or more of the attribute information of the vertex type, the attribute information of the edge type, and the attribute information of the general type, performance testing is performed on one or more vertices in the abnormal path and the paths between one or more vertices in the abnormal path and other vertices in the shortest path to obtain a second detection result; Based on the second detection result, fault location is performed on one or more vertices in the abnormal path to obtain one or more of the fault node, the fault container group, and the fault service, as well as the fault cause of one or more of the fault node, the fault container group, and the fault service.

8. The method according to claim 5, characterized in that, The step of performing fault analysis and localization on one or more vertices with the highest degree centrality to obtain one or more of the faulty node, the faulty container group, and the faulty service, as well as the fault cause of one or more of the faulty node, the faulty container group, and the faulty service, includes: Based on one or more of the attribute information of the vertex type, the attribute information of the edge type, and the attribute information of the general type, performance testing is performed on the one or more vertices with the highest degree centrality, and the paths between the one or more vertices with the highest degree centrality and other vertices in the cluster topology graph, to obtain a third detection result; Based on the third detection result, fault location is performed on one or more vertices with the highest degree centrality to obtain one or more of the fault node, the fault container group, and the fault service, as well as the fault cause of one or more of the fault node, the fault container group, and the fault service.

9. A fault location device, characterized in that, The device includes: The acquisition unit is used to acquire key indicator information of the nodes, container groups, and services involved when a target cluster fails. The construction unit is used to construct a knowledge graph based on the key indicator information of the nodes, container groups, and services to obtain a cluster topology graph. The analysis and localization unit is used to perform fault analysis and localization on the cluster topology graph using a target analysis algorithm, to obtain one or more of the faulty nodes, faulty container groups, and faulty services in the cluster topology graph, as well as the fault causes of one or more of the faulty nodes, faulty container groups, and faulty services; wherein, the target analysis algorithm is used to perform connectivity analysis, shortest path calculation, or centrality analysis on the cluster topology graph.

10. A processing apparatus, characterized in that, include: A processor and a memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes computer program instructions that cause a computer to perform the method as described in any one of claims 1 to 8.

Images