IT asset fault propagation prediction method and system based on dynamic evolution of knowledge graph

Abstract

The invention discloses an IT asset fault propagation prediction method and system based on dynamic evolution of a knowledge graph, and relates to the technical field of cloud computing and large-scale IT operation and maintenance management. Through an asynchronous message bus and a logic clock, the knowledge graph is updated immediately when resources are abnormal and a scheduling event occurs; the knowledge graph uniformly integrates physical connection, logic dependence and multi-copy redundancy, so that the cross-machine-room asset relationship is clear at a glance. And then, based on a weighted logistic regression model, node features and relation weights in the knowledge graph are fused, the node fault probability is accurately calculated, the limitation of traditional single-dimensional analysis is solved, self-healing operation is supported, end-to-end intelligent operation and maintenance from fault detection to prediction and early warning to closed-loop self-healing are realized, and the fault detection efficiency is improved. The problems that in a cross-machine-room and multi-live-site environment, resource topology is split, real-time state and alarm information cannot be fused with an asset dependence model, and large-scale real-time deployment of a traditional single-dimensional fault analysis and high-complexity prediction algorithm is difficult are effectively solved.

Description

Technical Field

[0001] The present application relates to the technical field of cloud computing and large-scale IT operation and maintenance management, and in particular to a method and system for predicting IT asset fault propagation based on the dynamic evolution of knowledge graphs. Background Art

[0002] With the popularization of cloud computing and distributed systems, modern IT infrastructure has shown characteristics such as multi-computer room multi-active deployment and containerized dynamic scheduling, which has complicated asset dependencies (such as cross-domain network links, microservice call chains, and database multi-copy synchronization) and enhanced the concealment of fault propagation paths.

[0003] Related technologies employ static topology modeling tools for topology construction and single-dimensional monitoring solutions for monitoring. These solutions rely on traditional fault prediction models combined with independently determined self-healing actions to address IT asset fault propagation. However, the applicants recognized that static topology modeling tools are unable to capture dynamic scheduling events in real time, resulting in a significant deviation between the graph and the actual environment. Single-dimensional monitoring solutions, lacking a topology-dependent perspective, can mistakenly report storage timeouts caused by packet loss across data center networks as disk failures and ignore slow propagation paths. Furthermore, traditional fault prediction models fail to incorporate dynamic edge weights, and self-healing actions rely on independent decision-making, resulting in an excessively high error rate. Summary of the Invention

[0004] In view of this, the present application provides an IT asset fault propagation prediction method and system based on the dynamic evolution of knowledge graphs. The main purpose is to solve the problem that static topology modeling tools cannot capture dynamic scheduling events in real time, resulting in a large deviation between the graph and the actual environment. The single-dimensional monitoring solution lacks a topology-dependent perspective and will mistakenly report storage timeouts caused by packet loss across computer room networks as disk failures. It will also ignore slow propagation paths. Moreover, traditional fault prediction models do not integrate dynamic edge weights, and self-healing actions make independent decisions, resulting in an excessively high error operation rate.

[0005] According to the first aspect of the present application, a method for predicting IT asset fault propagation based on the dynamic evolution of a knowledge graph is provided, the method comprising:

[0006] Obtaining physical connections and logical dependencies of IT assets, and constructing an initial topology using the physical connections and logical dependencies of IT assets;

[0007] Collecting monitoring indicators of each topological node in the initial topology and injecting them into the initial topology to generate an initial knowledge graph, wherein the monitoring indicators include resource utilization, health status score and historical fault data;

[0008] In response to resource anomalies or load scheduling events, the initial knowledge graph is updated through an asynchronous message bus and a logical clock mechanism to generate a dynamic knowledge graph;

[0009] Taking the fault node in the dynamic knowledge graph as the starting point, a fault propagation subgraph is constructed based on the connectivity relationship of the computer room network links, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph;

[0010] Calculating the failure probability of each node in the fault propagation subgraph based on a weighted logistic regression model;

[0011] When the failure probability of a node in the fault propagation subgraph is greater than or equal to the failure threshold, a fault isolation operation is triggered, and the fault isolation operation is at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment.

[0012] According to the second aspect of the present application, there is provided an IT asset fault propagation prediction system based on the dynamic evolution of a knowledge graph, the system comprising a topology construction module, a state acquisition module, an event evolution module, a fault propagation construction module, a fault prediction module, and a self-healing and early warning module;

[0013] The topology building module is used to obtain physical connections and logical dependencies of IT assets and use the physical connections and logical dependencies of IT assets to build an initial topology;

[0014] The state collection module is used to collect monitoring indicators of each topological node in the initial topology and inject them into the initial topology to generate an initial knowledge graph, wherein the monitoring indicators include resource utilization, health status score and historical fault data;

[0015] The event evolution module is used to respond to resource anomalies or load scheduling events, update the initial knowledge graph through an asynchronous message bus and a logical clock mechanism, and generate a dynamic knowledge graph;

[0016] The fault propagation construction module is used to construct a fault propagation subgraph based on the fault node in the dynamic knowledge graph, the connectivity relationship of the computer room network links, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph;

[0017] The fault prediction module is used to calculate the fault probability of each node in the fault propagation subgraph based on a weighted logistic regression model;

[0018] The self-healing and early warning module is used to trigger a fault isolation operation when the failure probability of a node in the fault propagation subgraph is greater than or equal to a fault threshold. The fault isolation operation is at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment.

[0019] By means of the above technical solution, the technical solution provided by the embodiment of the present invention has at least the following advantages:

[0020] The present application provides a method and system for predicting IT asset fault propagation based on the dynamic evolution of a knowledge graph. The present application obtains the physical connections and logical dependencies of IT assets, constructs an initial topology using the physical connections and logical dependencies of IT assets, collects monitoring indicators of each topological node in the initial topology and injects them into the initial topology to generate an initial knowledge graph. In response to resource anomalies or load scheduling events, the initial knowledge graph is updated through an asynchronous message bus and a logical clock mechanism to generate a dynamic knowledge graph. Starting from the fault node in the dynamic knowledge graph, a fault propagation subgraph is constructed based on the connectivity relationship of the computer room network links, the dependencies between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph. Based on a weighted logistic regression model, the failure probability of each node in the fault propagation subgraph is calculated. When the failure probability of a node in the fault propagation subgraph is greater than or equal to the failure threshold, a fault isolation operation is triggered. The monitoring indicators include resource utilization, health status score, and historical fault data. The fault isolation operation is at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment. Since static graphs cannot reflect the dynamic changes of IT assets, this application uses asynchronous message buses and logical clocks to immediately update the knowledge graph when resource anomalies and scheduling events occur. The knowledge graph uniformly integrates physical connections, logical dependencies, and multi-copy redundancy, making the relationship between assets across computer rooms clear at a glance. Then, based on the weighted logistic regression model, the node features and relationship weights in the knowledge graph are integrated to accurately calculate the probability of node failure in the fault propagation subgraph, solving the limitations of traditional single-dimensional analysis, and supporting self-healing operations such as cross-computer room traffic redirection, data copy reconstruction, and container scheduling adjustment, achieving end-to-end intelligent operation and maintenance from fault detection to prediction and warning to closed-loop self-healing, effectively solving the problems of resource topology fragmentation in cross-computer room and multi-active site environments, the inability to integrate real-time status and alarm information with asset dependency models, and the difficulty of large-scale real-time deployment of traditional single-dimensional fault analysis and high-complexity prediction algorithms.

[0021] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0023] Figure 1A schematic diagram of a method flow for predicting IT asset fault propagation based on the dynamic evolution of a knowledge graph provided in an embodiment of the present application is shown;

[0024] Figure 2A A schematic diagram of another method flow for predicting IT asset fault propagation based on the dynamic evolution of a knowledge graph provided by an embodiment of the present application is shown;

[0025] Figure 2B A schematic diagram of a fault propagation prediction process of a knowledge graph dynamic evolution system for IT assets provided by an embodiment of the present application is shown;

[0026] Figure 3 A schematic diagram of the structure of an IT asset fault propagation prediction system based on the dynamic evolution of a knowledge graph provided by an embodiment of the present application is shown;

[0027] Figure 4 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0029] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0030] In this application, unless otherwise specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they can refer to fixed connection, detachable connection, or integral connection; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on specific circumstances.

[0031] The following describes exemplary embodiments of the present application in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0032] The present application embodiment provides an IT asset fault propagation prediction method based on the dynamic evolution of knowledge graphs, such as Figure 1 As shown, the method includes:

[0033] 101. Obtain the physical connections and logical dependencies of IT assets, and use the physical connections and logical dependencies of IT assets to build the initial topology.

[0034] In the embodiments of the present application, the actual physical connections between IT assets such as virtual machines, storage, network equipment, and business services, as well as the dependencies of IT assets at the logical level, are comprehensively sorted out. These physical connections and logical dependencies are presented in the form of a topology diagram to form an initial topology, which lays the foundation for subsequent operations such as building a knowledge graph and conducting fault analysis. In traditional IT operations and maintenance, physical networks, servers, business systems, storage, etc. are often managed in a decentralized manner, and the relationships between them are vague. By constructing an initial topology, the physical connections and logical dependencies are clearly sorted out and presented, solving the pain points of scattered asset relationships, difficult cross-scenario analysis, and slow root cause location in traditional operations and maintenance.

[0035] 102. Collect monitoring indicators of each topological node in the initial topology and inject them into the initial topology to generate an initial knowledge graph.

[0036] In an embodiment of the present application, for each node in the initial topology, such as the topological nodes corresponding to IT assets such as virtual machines, storage clusters, and network devices, monitoring indicators such as resource utilization, health status scores, and historical fault data are collected. Then, the collected indicator data is filled into the nodes corresponding to the initial topology, so that the topology that originally only presented the asset connection relationship is upgraded to an initial knowledge graph that contains information such as asset operation status and historical performance. The initial topology only statically presents the physical connection and logical dependency of IT assets, while the initial knowledge graph generated after injecting monitoring indicators can reflect the operation status of each node in real time. For example, through resource utilization, you can know whether the current CPU of the server is almost fully loaded; with the help of health status scores, you can intuitively judge whether the operation of the storage device is stable; combined with historical fault data, you can understand the past health status of the node. In this way, IT assets are no longer a simple relationship diagram, but have become a dynamic information library containing the actual operation status of assets. Operation and maintenance personnel can more intuitively and comprehensively grasp the real-time status of IT assets.

[0037] 103. In response to resource anomalies or load scheduling events, the initial knowledge graph is updated through the asynchronous message bus and logical clock mechanism to generate a dynamic knowledge graph.

[0038] In an embodiment of the present application, when an IT asset has a resource anomaly event or a load scheduling event, the event information is quickly and asynchronously transmitted with the aid of an asynchronous message bus. Resource anomalies include a sudden increase in server CPU usage and an excessive packet loss rate in a network link, and load scheduling events include container migration and business traffic redistribution. At the same time, the node attributes, relationship weights, topology, etc. in the initial knowledge graph are updated in combination with the logical clock mechanism, so that the knowledge graph can reflect the latest status of IT assets in real time, and finally generate a dynamic knowledge graph. Among them, the asynchronous message bus can be simply understood as a channel that can efficiently transmit event information, such as Kafka, and the logical clock mechanism is a technology used to process the order of events and solve timing problems in a distributed environment. The operating status of IT assets changes dynamically. Resource anomalies and load scheduling are common change scenarios. Traditional static knowledge graphs cannot reflect these changes in a timely manner. However, through the asynchronous message bus, events can be quickly captured and update instructions can be transmitted. Combined with the logical clock mechanism, the timing of events can be handled well, allowing the knowledge graph to keep up with the status changes of IT assets in real time. For example, after container migration, the dynamic knowledge graph can immediately update the relationship between the container and the host machine, and the resource allocation status, ensuring that the graph is always a real-time mirror of IT assets, providing an accurate basis for subsequent fault analysis and prediction.

[0039] 104. Taking the fault node in the dynamic knowledge graph as the starting point, a fault propagation subgraph is constructed based on the connectivity relationship of the computer room network links, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph.

[0040] In an embodiment of the present application, after the dynamic knowledge graph marks the faulty node, the node is used as the starting point for analysis, and three types of key relationships are extracted from the dynamic knowledge graph: one is the connectivity relationship of the network links in the computer room, such as the network connection between the server where the faulty node is located and the switch, and between switches in different computer rooms; the second is the dependency relationship between business components, such as the business service corresponding to the faulty node, and the call and data interaction dependency with upstream and downstream business components; the third is the multi-copy redundant replication relationship, such as the synchronization relationship between the master-slave replicas of the database and the storage redundant replicas involved in the faulty node. Based on these three types of relationships, the paths and related nodes that the fault may spread and affect starting from the faulty node are sorted out, and a subgraph specifically used for analyzing fault propagation is constructed, that is, the fault propagation subgraph, which can avoid the redundant operation of performing a full analysis of the entire knowledge graph, while ensuring the timeliness of the fault propagation analysis.

[0041] 105. Based on the weighted logistic regression model, the failure probability of each node in the fault propagation subgraph is calculated.

[0042] In an embodiment of the present application, a weighted logistic regression model is used to take the relevant features of each node in the fault propagation subgraph as the model input, and the model operation outputs the probability value of each node failing in the current fault propagation scenario, thereby quantitatively assessing the risk of the fault spreading to the node, thereby achieving a quantitative analysis of the fault risk. Moreover, the fault propagation subgraph has screened out the nodes and relationships related to fault propagation. When calculating the model, it is only necessary to focus on these key ranges, without performing a full calculation on the entire IT asset knowledge graph. On the one hand, it reduces the scale of calculation, improves the efficiency of calculation, and can obtain the fault probability results faster, meeting the timeliness requirements of operation and maintenance. On the other hand, it can avoid irrelevant nodes and relationships from interfering with the calculation, making the probability prediction more focused and accurate, so that operation and maintenance personnel can clearly distinguish the fault risk levels of different nodes based on these probability values, such as giving priority to nodes with high failure probability, and improving the targeted response to faults.

[0043] 106. When the failure probability of a node in the fault propagation subgraph is greater than or equal to the failure threshold, a fault isolation operation is triggered.

[0044] In an embodiment of the present application, after completing the construction of the fault propagation subgraph and the calculation of the fault probability, a fault threshold is set, and the fault probability of each node in the fault propagation subgraph is continuously monitored. Once the failure probability of a node is found to reach or exceed the threshold, a pre-defined fault isolation operation is automatically triggered. These operations include at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment, so as to prevent the fault from spreading further and reduce the impact on the business. Among them, the cross-computer room traffic redirection operation is suitable for scenarios where network link failures and cross-computer room businesses are affected. It quickly adjusts the traffic direction to avoid the fault-affected area and ensures business continuity; the data copy reconstruction operation is for storage failures and data redundancy is threatened. It rebuilds a healthy copy to ensure that data is not lost, the business can be read and written normally, and data availability is guaranteed; the container scheduling strategy adjustment operation can adjust the scheduling rules in a containerized deployment environment when the risk of container node failure is high, avoid the faulty node, migrate the container to a healthy node, and maintain business operation. Traditional O&M often waits until a fault actually occurs and service disruptions become apparent before handling it, effectively incurring losses. However, by monitoring the probability of a fault in real time and triggering isolation actions once a threshold is reached, we can proactively cut off the fault's propagation path before or immediately after a fault actually occurs. For example, if a node's failure probability reaches a threshold, cross-data center traffic redirection can be used to divert service traffic away from the faulty link or node, preventing the fault from spreading and causing widespread service disruption, effectively reducing the scope of the fault and the resulting losses.

[0045] An embodiment of the present application provides an IT asset fault propagation prediction method based on the dynamic evolution of a knowledge graph. Compared with the prior art, the embodiment of the present application obtains the physical connections and logical dependencies of IT assets, constructs an initial topology using the physical connections and logical dependencies of IT assets, collects monitoring indicators of each topological node in the initial topology and injects them into the initial topology to generate an initial knowledge graph. In response to resource anomalies or load scheduling events, the initial knowledge graph is updated through an asynchronous message bus and a logical clock mechanism to generate a dynamic knowledge graph. Starting from the fault node in the dynamic knowledge graph, a fault propagation subgraph is constructed based on the connectivity relationship of the computer room network links, the dependencies between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph. Based on a weighted logistic regression model, the failure probability of each node in the fault propagation subgraph is calculated. When the failure probability of a node in the fault propagation subgraph is greater than or equal to the failure threshold, a fault isolation operation is triggered. The monitoring indicators include resource utilization, health status score, and historical fault data. The fault isolation operation is at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment. Since static graphs cannot reflect the dynamic changes of IT assets, this application uses asynchronous message buses and logical clocks to immediately update the knowledge graph when resource anomalies and scheduling events occur. The knowledge graph uniformly integrates physical connections, logical dependencies, and multi-copy redundancy, making the relationship between assets across computer rooms clear at a glance. Then, based on the weighted logistic regression model, the node features and relationship weights in the knowledge graph are integrated to accurately calculate the probability of node failure in the fault propagation subgraph, solving the limitations of traditional single-dimensional analysis, and supporting self-healing operations such as cross-computer room traffic redirection, data copy reconstruction, and container scheduling adjustment, achieving end-to-end intelligent operation and maintenance from fault detection to prediction and warning to closed-loop self-healing, effectively solving the problems of resource topology fragmentation in cross-computer room and multi-active site environments, the inability to integrate real-time status and alarm information with asset dependency models, and the difficulty of large-scale real-time deployment of traditional single-dimensional fault analysis and high-complexity prediction algorithms.

[0046] Furthermore, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, the embodiment of this application provides another IT asset fault propagation prediction method based on the dynamic evolution of the knowledge graph, such as Figure 2A As shown, the method includes:

[0047] 201. Obtain the physical connections and logical dependencies of IT assets, and use the physical connections and logical dependencies of IT assets to build an initial topology.

[0048] In an embodiment of the present application, a multi-active site topology acquisition component is used to scan IT assets in multiple computer rooms to obtain performance configuration data, physical link connection relationships, dependencies between business components, multi-copy redundant replication relationships, and container / virtual machine affinity scheduling relationships, where IT assets include virtual machines, storage, storage clusters, network equipment, and business services. Performance configuration data includes the number of CPU cores, memory size, network bandwidth configuration, and response timeout of business services; physical link connection relationships include the physical wiring between servers and switches, and fiber optic connections between switches in different computer rooms; dependencies between business components include the dependency logic of order services calling payment services and inventory services in e-commerce systems; multi-copy redundant replication relationships include the synchronous replication relationships between database master nodes and slave nodes, and multiple copies of storage clusters; and container / virtual machine affinity scheduling relationships include rules for prioritizing certain containers to be scheduled to specific physical machines or virtual machines. Next, performance configuration data, physical link connection relationships, dependencies between business components, multi-copy redundant replication relationships, and container / virtual machine affinity scheduling relationships are used as the physical connections and logical dependencies of IT assets. Real-time performance indicators (such as current CPU usage and memory occupancy) and static configuration attributes (such as the server's factory CPU model and total storage capacity) are extracted from the performance configuration data for each knowledge graph node. Virtual machines, containers, storage clusters, network devices, and business services are then mapped into multiple knowledge graph nodes. Physical link connection relationships, dependencies between business components, multi-copy redundant replication relationships, and container / virtual machine affinity scheduling relationships are mapped into edges between multiple knowledge graph nodes. The initial topology is then derived by combining the real-time performance indicators and static configuration attributes of each knowledge graph node.

[0049] In an IT environment with multiple active sites (distributed deployment in multiple computer rooms), IT assets are widely distributed and have complex relationships. Through the multi-active site topology collection component, various IT assets can be comprehensively scanned across computer rooms, covering multi-dimensional relationships such as physical connections, business dependencies, and redundant replication. This can solve the problem that traditional topology construction tools are difficult to adapt to complex multi-active scenarios, resulting in incomplete sorting of asset relationships. For example, cross-computer room business call dependencies and multi-copy relationships of off-site storage can be accurately collected and presented, providing a complete topological foundation for subsequent fault analysis. Moreover, not only hard connections such as physical links are collected, but also soft logic such as business component dependencies, multi-copy redundancy, and affinity scheduling are covered to construct a three-dimensional IT asset topology relationship. The fusion of multi-dimensional relationships makes the initial topology no longer a simple device connection diagram, but a relationship network that can reflect the actual operating logic of IT assets. It can solve the problem of difficult, incomplete, and inaccurate sorting of IT asset topology in a complex multi-active environment.

[0050] 202. Based on Prometheus instances deployed in multiple computer rooms, the resource usage of each topology node is collected and stored through the Thanos / Cortex federated architecture.

[0051] In an embodiment of the present application, Prometheus instances are deployed in different computer rooms. Prometheus is an open source system monitoring and alarm toolkit that is commonly used to collect monitoring indicators. By deploying Prometheus instances in each computer room, resource utilization can be collected close to local IT assets, avoiding incomplete or inaccurate data collection due to problems such as cross-computer room network delays and permission restrictions. With the help of these instances, the resource utilization of each node in the constructed initial topology can be collected. After the data is collected, the Thanos / Cortex federated architecture is used to centrally store and manage the topological node resource utilization data collected by the Prometheus instances in each computer room, so that the CPU, memory and other monitoring indicators of all computer rooms can see the latest and unified data in the center or at any query point.

[0052] Resource utilization includes CPU utilization, memory utilization, disk I / O utilization, network bandwidth utilization, etc. For the subsequent knowledge graph to be dynamically updated, resource utilization is one of the important monitoring indicators. The collected and stored resource utilization data can be provided to the knowledge graph engine in real time, allowing the knowledge graph to be updated and analyzed based on the latest resource status. For example, a sudden increase in the CPU utilization of a node triggers a resource anomaly event. The knowledge graph can quickly obtain this data for topology updates and fault propagation prediction, ensuring the continuity and timeliness of intelligent operation and maintenance analysis. The Thanos / Cortex federated architecture is a technical solution that can achieve data aggregation, long-term storage, and unified querying of multiple Prometheus instances. On the one hand, it can solve the problem of decentralized storage and difficulty in global management of resource utilization data in multi-computer room scenarios. On the other hand, it can break through the limitations of single-machine storage capacity and query capabilities. Even long-term historical resource data can be properly stored and quickly queried, facilitating subsequent trend analysis and fault tracing.

[0053] 203. Collect the health status score of each topology node based on the Consul+Gossip protocol.

[0054] In an embodiment of the present application, Consul (an open source service discovery and configuration management tool) is combined with the Gossip protocol (a protocol for information dissemination between nodes in a distributed system, which has the characteristics of decentralization and good fault tolerance) to allow each node to exchange and disseminate health status related information, such as the node's service operation status, error logs, heartbeat signals, etc. Based on the received information, Consul evaluates each topological node through a built-in health check mechanism, such as detecting whether the service is available and whether the response has timed out, and calculates and outputs a health status score including a simple heartbeat, port connectivity test, service response time, etc., to reflect the health of the node.

[0055] In a multi-datacenter, large-scale distributed IT environment, traditional centralized health status collection methods are prone to incomplete or ineffective collection due to single points of failure and network latency. The Consul+Gossip protocol utilizes a distributed architecture, allowing every node to participate in the dissemination and exchange of health status information, eliminating the need to rely on a single centralized collection point. For example, in a distributed system spanning multiple data centers, even if some network links are temporarily interrupted, the Gossip protocol can still relay information through other nodes, ensuring comprehensive and continuous health status collection for topological nodes.

[0056] The Gossip protocol's information dissemination allows changes in node health status to be rapidly propagated across the network. When a node's health status deviates, relevant information is rapidly transmitted to Consul and other nodes via Gossip, enabling real-time updates of the health score. Furthermore, due to its distributed nature, node failures or network fluctuations do not cause the entire collection system to fail, providing excellent fault tolerance. Consul itself provides a rich set of health check mechanisms. Beyond basic service liveness checks, it can also integrate health assessments with business logic, such as verifying that a specific service interface is returning data properly. By collecting multi-dimensional node status information via the Gossip protocol, such as heartbeats, error codes, and resource usage associations, Consul integrates this information to calculate a more comprehensive and accurate health score for each topology node. Consul is commonly used in operational maintenance scenarios such as service discovery and configuration management, and offers strong compatibility and interoperability within the IT operations ecosystem. Health scores collected using the Consul + Gossip protocol can be easily integrated with other operations tools, such as Prometheus monitoring and knowledge graph engines, ensuring smooth collaboration across the entire intelligent operations process.

[0057] 204. Collect the fault timestamp and fault type of each topological node from the centralized log system, write the fault timestamp and fault type of each topological node into the centralized log / event library, obtain the topological node fault frequency based on Alertmanage continuous statistics, and generate historical fault data for each topological node.

[0058] In an embodiment of the present application, a centralized log system is relied upon to filter out fault logs related to each topological node and extract key information: first, the fault timestamp at the time when the fault occurred is accurately recorded; second, the fault types such as CPU overload fault, network connection timeout fault, and service crash fault; then, these extracted fault timestamps and fault type information are organized and written into a centralized log / event library for persistent storage; at the same time, Alertmanager is used to continuously count the frequency of faults of each topological node, that is, the number of times the fault occurs within a certain period of time, and then the fault timestamp, fault type, fault frequency and other information are integrated to form historical fault data of each topological node for subsequent analysis.

[0059] Alertmanager is a component used for alert management in the Prometheus ecosystem. Centralized logging systems, such as the ELK stack and Graylog, are used to centrally collect, store, and manage logs generated by IT assets. In a complex environment with multiple data centers and diverse IT asset types, decentralized storage of fault logs from each node can be prone to loss and difficult to manage. A centralized logging system collects fault information, aggregates data such as fault timestamps and fault types from all topological nodes, and stores them uniformly in a log / event repository, preventing the loss of fault information caused by decentralized storage.

[0060] By collecting fault timestamps and fault types, and combining them with the fault frequency statistics from Alertmanager, we construct historical fault data for topological nodes from three dimensions: time, type, and frequency. The timestamp can accurately locate the moment a fault occurs, facilitating analysis of whether the fault is periodic; the fault type can distinguish the nature of the fault; and the fault frequency can reflect the health and stability of the node. Through multi-dimensional data integration, we can more comprehensively restore the historical picture of node failures and provide rich feature dimensions for subsequent fault prediction models. Centralized logging systems and Alertmanager are both mature O&M monitoring ecosystem components, widely used in actual O&M scenarios. Collecting and integrating data based on these existing components can be well integrated into the existing O&M system without the need to build additional complex data collection links.

[0061] 205. The resource usage rate, health status score and historical fault data of each topological node are used as monitoring indicators of each topological node. The monitoring indicators of each topological node are injected into the initial topology to generate an initial knowledge graph.

[0062] In an embodiment of the present application, three types of key data of each topological node: resource utilization, health status score, and historical fault data are defined as monitoring indicators of the node. These monitoring indicators are then filled into the initial topology, so that the topology that originally only reflects the connection relationship of IT assets is upgraded to an initial knowledge graph containing information such as the real-time status of assets and historical performance. At the same time, when the monitoring indicators of a topological node reach the preset event trigger conditions, such as changes in CPU usage, node DOWN / UP, and increased alarm counts, the monitoring indicators of multiple topological nodes are sent to the knowledge graph engine through Kafka (a high-throughput message queue for asynchronous data transmission). The engine updates the attributes of the corresponding nodes (i.e., CPU, memory, health status, number of failures) and dependency weights based on these indicators to ensure that the graph always reflects the most realistic operating status, and finally generates the initial knowledge graph.

[0063] The initial topology was originally just a skeleton of IT asset relationships. By incorporating monitoring metrics such as resource utilization, health status, and historical faults, the knowledge graph evolved to reflect resource distribution, operational status, and dependencies across active-active environments across data centers, providing a rich basis for subsequent fault analysis and prediction. Furthermore, Kafka was introduced to implement an event-driven graph update mechanism: when monitoring metrics trigger conditions, data is immediately asynchronously pushed to update the graph. Compared to traditional scheduled full updates, this approach is more efficient and real-time. For example, if a node experiences a sudden CPU overload, Kafka can transmit the event in seconds, and the knowledge graph engine quickly updates the node's status, ensuring that the graph always reflects the latest state of IT assets, making subsequent fault propagation analysis and prediction more accurate.

[0064] 206. In response to resource anomalies or load scheduling events, the initial knowledge graph is updated through the asynchronous message bus and logical clock mechanism to generate a dynamic knowledge graph.

[0065] In an embodiment of the present application, in response to resource anomalies or load scheduling events, the resource anomalies or load scheduling events are standardized to obtain standardized events containing information such as event type, affected nodes, and occurrence time. Among them, resource anomaly events include CPU overload and memory leak, and load scheduling events include container migration and traffic redistribution.

[0066] Static knowledge graphs cannot reflect the dynamic changes of IT assets, and fault analysis is easily based on outdated data, leading to wrong decisions. Therefore, the embodiment of the present application uses an asynchronous message bus to capture resource anomalies and load scheduling events in real time, triggering dynamic updates of the knowledge graph to ensure the accuracy of fault analysis and prediction.

[0067] Standardized events are then published to an asynchronous message bus, which decouples event production and consumption. Based on this asynchronous message bus, vector clocks are configured for standardized events. These events are then sent to topics corresponding to their event types, such as resource overload events and container migration events. Vector clocks are used to mark event timing. Using an asynchronous message bus, such as Kafka, to deliver events prevents system crashes caused by concurrent events and ensures the stability of IT operations systems.

[0068] Then, based on the knowledge graph engine, event information is obtained from the asynchronous message bus according to the pre-subscribed topics. Based on the knowledge graph engine, the vector clock of the standardized event is compared with the local clock of the corresponding node in the initial knowledge graph.

[0069] If there is a timing conflict between the vector clock of the standardized event and the local clock of the corresponding node in the initial knowledge graph, for example, the event timestamp is inconsistent with the timestamp of the node state in the graph, the initial knowledge graph is corrected by the logical timestamp sorting algorithm to obtain a corrected knowledge graph. In a multi-computer room and multi-node environment, the timing of events is easily confused, and directly updating the graph will lead to state contradictions. Therefore, the embodiment of the present application adds a time vector to each event, and the knowledge graph engine compares the event vector clock with the clock maintained locally by the node in the graph, and uses the timestamp sorting algorithm to re-sort the order of events to ensure that the knowledge graph is updated in the order in which the events actually occur, avoiding topological relationship confusion caused by timing errors, thereby resolving timing conflicts.

[0070] Finally, based on the knowledge graph engine, event information is used to update dynamic attributes, adjust relationship weights, and modify the topology of corresponding nodes in the modified knowledge graph, generating a dynamic knowledge graph. This includes updating node dynamic attributes, such as updating a node's CPU usage; adjusting relationship weights, such as reducing the weight of a failed node's relationship with its original dependent nodes due to container migration; and modifying the topology, such as adding a new container to a new host.

[0071] 207. Taking the fault node in the dynamic knowledge graph as the starting point, a fault propagation subgraph is constructed based on the connectivity relationship of the computer room network links, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph.

[0072] In an embodiment of the present application, after the dynamic knowledge graph is updated in real time and identifies fault nodes such as servers that trigger alarms due to resource overload and components that crash business services, the fault node is used as the starting point for analysis, and three types of core relationships are extracted from the dynamic knowledge graph: connectivity relationships of computer room network links, such as switches and routers connected to the server where the fault node is located, and connectivity of cross-computer room network links; dependencies between business components, such as upstream and downstream services called by the faulty business service, dependent database components, etc.; multi-copy redundant replication relationships, such as database master-slave replica synchronization links involved in the faulty node, and replication relationships of multiple copies of the storage cluster, etc. Based on these three types of relationships, the paths along which the fault may propagate and spread from the faulty node, as well as other nodes that may be affected, are sorted out, and these paths and nodes are constructed into a subgraph specifically for analyzing fault propagation, namely, a fault propagation subgraph, which allows operation and maintenance personnel to instantly lock the key paths of fault propagation.

[0073] 208. Based on the weighted logistic regression model, the failure probability of each node in the fault propagation subgraph is calculated.

[0074] In an embodiment of the present application, the resource utilization rate, health status score and historical fault data of each node in the fault propagation subgraph are obtained, and a weighted summation calculation is performed on the resource utilization rate, health status score and historical fault data of each node to obtain a comprehensive characteristic value of each node, and the multi-dimensional state is condensed into a single value to facilitate model calculation.

[0075] Then, based on the weighted logistic regression model, the comprehensive characteristic value of each node is used to calculate the failure probability of each node. The calculation formula is as follows:

[0076] Formula 1: P i =σ(αx i +β∑ j∈N(i) w ij y j +b),

[0077] Among them, P i represents the failure probability of node i in the next T hours, σ(·) represents the sigmoid function, and x i represents the comprehensive eigenvalue of node i, N(i) represents the set of neighboring nodes of node i, y j Indicates the current fault status of neighbor node j, fault is 1, normal is 0, w ijrepresents the association weight between node i and its neighbor node j, α represents the first model parameter, β represents the second model parameter, and b represents the third model parameter. T hours is obtained through historical data analysis and business collaboration analysis. T is configured or dynamically adjusted to meet the requirements of both early warning and sufficient self-healing response time. The association weight is derived by normalizing the link bandwidth, latency, packet loss rate, or replication delay. The first, second, and third model parameters are obtained through maximum likelihood estimation or gradient descent optimization training of historical fault propagation data. The model input integrates multi-dimensional features such as resource pressure, health, historical reliability, and neighbor influence. In actual fault propagation, node failures are not only determined by the node itself but also by dependencies. Accurately characterizing the correlation of fault propagation makes predictions more relevant to real IT environments. This can solve the problems of qualitative ambiguity, neglect of correlation, and inability to adapt to the environment in traditional fault prediction, and achieve accurate quantification and dynamic prediction of fault risks.

[0078] 209. When the failure probability of a node in the fault propagation subgraph is greater than or equal to the failure threshold, a fault isolation operation is triggered.

[0079] In an embodiment of the present application, when the failure probability of a node in the fault propagation subgraph is greater than or equal to a fault threshold, a fault isolation operation is triggered, wherein the fault threshold can be set in combination with the reliability requirements of the business.

[0080] By adjusting the traffic weight of the computer room through the global load balancing mechanism or remapping the transmission path through the SDN controller, cross-computer room traffic redirection operations can be achieved. This can cut off the traffic input of the faulty node, prevent the impact of the fault from being amplified by the traffic impact, and ensure that business requests flow to healthy nodes, such as when facing network layer failures such as computer room link interruption.

[0081] In the dynamic knowledge graph, healthy copies that have a multi-copy redundant replication relationship with the faulty node are identified, and the target data pulled from the healthy copies are deployed to the redundant nodes in the dynamic knowledge graph to implement data copy reconstruction operations. This can restore data redundancy protection and avoid data loss or unavailability caused by failures, such as data layer failures such as storage node failures.

[0082] In response to container node failures, the Kubernetes scheduler dynamically changes the affinity / anti-affinity rules based on the affinity / anti-affinity rules between containers and nodes in the dynamic knowledge graph. The Kubernetes scheduler reschedules the container based on the changed affinity / anti-affinity rules to implement container scheduling policy adjustment operations, which can avoid faulty nodes and ensure the continuous operation of containerized services, such as containerized environment failures such as node resource exhaustion. Traditional self-healing operations rely on static configurations and are prone to failure due to asset changes. The dynamic knowledge graph of this application ensures real-time and accurate operations, improves the success rate of self-healing, and can solve the problems of slow response, poor scenario adaptation, and low operation accuracy of traditional fault self-healing.

[0083] In an alternative implementation, JMeter (a stress testing tool) or Locust (an open-source load testing framework) is used to inject failure scenarios, including IT asset resource overload, IT asset network failure, and IT asset storage failure. Performance data is then collected, along with the failure probability of each node in the fault propagation subgraph. Performance data includes response time, throughput, and error rate. This performance data is then compared with the failure probability of each node in a multi-dimensional analysis to obtain comparison results. Based on these comparison results, a gradient descent algorithm is used to optimize the model parameters of the weighted logistic regression model, making the failure probability prediction more consistent with actual performance degradation. Adjustments are also made to the computer room network link connectivity relationships in the dynamic knowledge graph (for example, link weights are modified to reflect the impact of actual network latency on failure propagation), dependencies between business components (for example, dependency strength is adjusted to match the performance correlation of actual business calls), and multi-copy redundant replication relationships (for example, replica synchronization strategies are optimized to reduce the risk of data loss during storage failures). By injecting failure scenarios through JMeter / Locust, realistic threats such as resource overload, network failure, and storage failure are simulated in advance, verifying the effectiveness of the fault prediction model and self-healing operations.

[0084] In summary, if Figure 2BAs shown, the topology modeling module builds the initial topology, the status collection module uses Prometheus to accurately collect resource utilization, combines Consul+Gossip protocol to obtain health status scores, collects historical fault data from the centralized log system and Alertmanager, and injects these multi-dimensional data into the initial topology to generate an initial knowledge graph that can reflect the real-time status and historical information of assets; when IT assets have resource anomalies or dynamic changes such as load scheduling, the event evolution module starts, relying on the Kafka asynchronous message bus to achieve efficient event transmission, and uses the logical clock mechanism to handle timing problems in a distributed environment, dynamically updates the initial knowledge graph, generates a dynamic knowledge graph, ensures that the knowledge graph is always synchronized with the actual status of IT assets, and provides real-time and accurate topological relationships for fault analysis; the performance evaluation module uses JMeter or Locust tools to simulate real fault scenarios such as IT asset resource overload, network failure, storage failure, etc., and collects response time, throughput, and error The system uses performance data such as error rate and other performance data to conduct a multi-dimensional comparative analysis in combination with the node failure probability in the fault propagation subgraph, and then optimizes the weighted logistic regression model parameters through the gradient descent algorithm. At the same time, the asset relationship in the dynamic knowledge graph is adjusted to enable the system to have self-evolution capabilities and continuously improve the accuracy of fault prediction and self-healing. The fault propagation modeling module takes the fault node in the dynamic knowledge graph as the starting point, and constructs the fault propagation subgraph based on the relationships such as the network link of the computer room, business component dependency, and multi-copy redundant replication, clearly presenting the possible spread path and impact range of the fault. The prediction calculation module is based on the weighted logistic regression model, combined with node resource utilization, health status, historical faults and other characteristics, to calculate the failure probability of each node in the subgraph and quantify the failure risk. If the probability exceeds the threshold, the self-healing and early warning module will perform automatic replica reconstruction, cross-computer room traffic switching or operation and maintenance alarm, realizing an intelligent operation and maintenance closed loop from fault discovery, prediction to automatic intervention, ensuring the stable operation of IT assets, reducing the risk of business interruption, and helping enterprises build an efficient and intelligent IT operation and maintenance system.

[0085] An embodiment of the present application provides an IT asset fault propagation prediction method based on the dynamic evolution of a knowledge graph. Compared with the prior art, the embodiment of the present application obtains the physical connections and logical dependencies of IT assets, constructs an initial topology using the physical connections and logical dependencies of IT assets, collects monitoring indicators of each topological node in the initial topology and injects them into the initial topology to generate an initial knowledge graph. In response to resource anomalies or load scheduling events, the initial knowledge graph is updated through an asynchronous message bus and a logical clock mechanism to generate a dynamic knowledge graph. Starting from the fault node in the dynamic knowledge graph, a fault propagation subgraph is constructed based on the connectivity relationship of the computer room network links, the dependencies between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph. Based on a weighted logistic regression model, the failure probability of each node in the fault propagation subgraph is calculated. When the failure probability of a node in the fault propagation subgraph is greater than or equal to the failure threshold, a fault isolation operation is triggered. The monitoring indicators include resource utilization, health status score, and historical fault data. The fault isolation operation is at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment. Since static graphs cannot reflect the dynamic changes of IT assets, this application uses asynchronous message buses and logical clocks to immediately update the knowledge graph when resource anomalies and scheduling events occur. The knowledge graph uniformly integrates physical connections, logical dependencies, and multi-copy redundancy, making the relationship between assets across computer rooms clear at a glance. Then, based on the weighted logistic regression model, the node features and relationship weights in the knowledge graph are integrated to accurately calculate the probability of node failure in the fault propagation subgraph, solving the limitations of traditional single-dimensional analysis, and supporting self-healing operations such as cross-computer room traffic redirection, data copy reconstruction, and container scheduling adjustment, achieving end-to-end intelligent operation and maintenance from fault detection to prediction and warning to closed-loop self-healing, effectively solving the problems of resource topology fragmentation in cross-computer room and multi-active site environments, the inability to integrate real-time status and alarm information with asset dependency models, and the difficulty of large-scale real-time deployment of traditional single-dimensional fault analysis and high-complexity prediction algorithms.

[0086] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides an IT asset fault propagation prediction system based on the dynamic evolution of the knowledge graph, which is used to implement the IT asset fault propagation prediction method based on the dynamic evolution of the knowledge graph, such as Figure 3 As shown, the system includes a topology construction module 301, a state acquisition module 302, an event evolution module 303, a fault propagation construction module 304, a fault prediction module 305, and a self-healing and early warning module 306;

[0087] A topology construction module 301 is used to obtain physical connections and logical dependencies of IT assets and construct an initial topology using the physical connections and logical dependencies of IT assets;

[0088] The status collection module 302 is used to collect monitoring indicators of each topological node in the initial topology and inject them into the initial topology to generate an initial knowledge graph. The monitoring indicators include resource utilization, health status score and historical fault data.

[0089] The event evolution module 303 is used to respond to resource anomalies or load scheduling events, update the initial knowledge graph through the asynchronous message bus and logical clock mechanism, and generate a dynamic knowledge graph;

[0090] The fault propagation construction module 304 is used to construct a fault propagation subgraph based on the fault node in the dynamic knowledge graph and the connectivity relationship of the computer room network links, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph;

[0091] A fault prediction module 305 is used to calculate the fault probability of each node in the fault propagation subgraph based on a weighted logistic regression model;

[0092] The self-healing and early warning module 306 is used to trigger a fault isolation operation when the failure probability of a node in the fault propagation subgraph is greater than or equal to the fault threshold. The fault isolation operation is at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment.

[0093] In a specific application scenario, the topology construction module 301 is used to use a multi-active site topology collection component to scan IT assets in multiple computer rooms to obtain performance configuration data, physical link connection relationships, dependencies between business components, multi-copy redundant replication relationships, and container / virtual machine affinity scheduling relationships. IT assets include virtual machines, storage, storage clusters, network equipment, and business services; the performance configuration data, physical link connection relationships, dependencies between business components, multi-copy redundant replication relationships, and container / virtual machine affinity scheduling relationships are used as physical connections and logical dependencies of IT assets; the real-time performance indicators and static configuration attributes of each knowledge graph node are extracted from the performance configuration data; virtual machines, containers, storage clusters, network equipment, and business services are mapped to multiple knowledge graph nodes, and the physical link connection relationships, dependencies between business components, multi-copy redundant replication relationships, and container / virtual machine affinity scheduling relationships are mapped to edges between multiple knowledge graph nodes, and the real-time performance indicators and static configuration attributes of each knowledge graph node are combined to obtain the initial topology.

[0094] In a specific application scenario, the status collection module 302 is used to collect the resource utilization rate of each topological node based on the Prometheus instances deployed in multiple computer rooms, and store the resource utilization rate of each topological node through the Thanos / Cortex federal architecture; collect the health status score of each topological node based on the Consul+Gossip protocol; collect the fault timestamp and fault type of each topological node from the centralized log system, write the fault timestamp and fault type of each topological node into the centralized log / event library, and obtain the topological node failure frequency based on Alertmanage continuous statistics to generate historical fault data for each topological node; use the resource utilization rate, health status score and historical fault data of each topological node as monitoring indicators of each topological node; when the monitoring indicators of a topological node meet the event triggering conditions, send the monitoring indicators of multiple topological nodes to the knowledge graph engine through Kafka, so that the knowledge graph engine uses the monitoring indicators of multiple topological nodes to update the initial topology and generate an initial knowledge graph.

[0095] In a specific application scenario, the event evolution module 303 is used to respond to resource anomalies or load scheduling events, perform standardized processing on resource anomalies or load scheduling events, and obtain standardized events; publish standardized events to the asynchronous message bus, configure a vector clock for the standardized events based on the asynchronous message bus, and send the standardized events to the corresponding topics according to the event type of the standardized events; based on the knowledge graph engine, obtain event information from the asynchronous message bus according to the pre-subscribed topics; based on the knowledge graph engine, compare the vector clock of the standardized event with the local clock of the corresponding node in the initial knowledge graph; if there is a timing conflict between the vector clock of the standardized event and the local clock of the corresponding node in the initial knowledge graph, the initial knowledge graph is corrected by the logical timestamp sorting algorithm to obtain a corrected knowledge graph; based on the knowledge graph engine, use event information to perform updates on dynamic attributes, adjust relationship weights, and change topology operations on the corresponding nodes in the corrected knowledge graph to generate a dynamic knowledge graph.

[0096] In a specific application scenario, the fault prediction module 305 is used to obtain the resource utilization rate, health status score, and historical fault data of each node in the fault propagation subgraph, perform weighted sum calculation on the resource utilization rate, health status score, and historical fault data of each node to obtain the comprehensive characteristic value of each node; and calculate the failure probability of each node using the comprehensive characteristic value of each node based on the weighted logistic regression model.

[0097]

[0098] Among them, P irepresents the failure probability of node i, σ(·) represents the sigmoid function, x i represents the comprehensive eigenvalue of node i, N(i) represents the set of neighboring nodes of node i, y j represents the current fault status of neighbor node j, w ij represents the association weight between node i and its neighbor node j, α represents the first model parameter, β represents the second model parameter, and b represents the third model parameter.

[0099] In a specific application scenario, the fault prediction module 305 is used to obtain historical fault propagation data, perform maximum likelihood estimation on the historical fault propagation data, and obtain first model parameters, second model parameters, and third model parameters; or, obtain historical fault propagation data, perform gradient descent optimization training on the historical fault propagation data to obtain first model parameters, second model parameters, and third model parameters.

[0100] In a specific application scenario, the self-healing and early warning module 306 is used to adjust the traffic weight of the computer room through a global load balancing mechanism or remap the transmission path through an SDN controller to implement cross-computer room traffic redirection operations; identify healthy copies that have multiple redundant copies of the faulty node in the dynamic knowledge graph, and deploy the target data pulled from the healthy copies to the redundant nodes in the dynamic knowledge graph to implement data copy reconstruction operations; for container node failures, the Kubernetes scheduler dynamically changes the affinity / anti-affinity rules based on the affinity / anti-affinity rule markings of the container and the node in the dynamic knowledge graph, and reschedules the container based on the changed affinity / anti-affinity rules through the Kubernetes scheduler to implement container scheduling policy adjustment operations.

[0101] In a specific application scenario, the device also includes a performance evaluation module 307, which is used to use JMeter or Locust tools to inject fault scenarios, including IT asset resource overload, IT asset network failure, and IT asset storage failure; collect performance data and the failure probability of each node in the fault propagation subgraph, and the performance data includes response time, throughput, and error rate; perform a multi-dimensional comparative analysis of the performance data and the failure probability of each node to obtain a comparison result; based on the comparison result, use the gradient descent algorithm to optimize the model parameters of the weighted logistic regression model, and adjust the network link connectivity relationship of the computer room, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph.

[0102] The IT asset fault propagation prediction system based on the dynamic evolution of the knowledge graph of this application relies on the computing power of the server to provide services to users. The server can be an independent server or a server that provides basic cloud computing such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0103] An embodiment of the present application provides an IT asset fault propagation prediction system based on the dynamic evolution of a knowledge graph. Compared with the prior art, the embodiment of the present application obtains the physical connections and logical dependencies of IT assets, constructs an initial topology using the physical connections and logical dependencies of IT assets, collects monitoring indicators of each topological node in the initial topology and injects them into the initial topology to generate an initial knowledge graph. In response to resource anomalies or load scheduling events, the initial knowledge graph is updated through an asynchronous message bus and a logical clock mechanism to generate a dynamic knowledge graph. Starting from the fault node in the dynamic knowledge graph, a fault propagation subgraph is constructed based on the connectivity relationship of the computer room network links, the dependencies between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph. Based on a weighted logistic regression model, the failure probability of each node in the fault propagation subgraph is calculated. When the failure probability of a node in the fault propagation subgraph is greater than or equal to the fault threshold, a fault isolation operation is triggered. The monitoring indicators include resource utilization, health status score, and historical fault data. The fault isolation operation is at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment. Since static graphs cannot reflect the dynamic changes of IT assets, this application uses asynchronous message buses and logical clocks to immediately update the knowledge graph when resource anomalies and scheduling events occur. The knowledge graph uniformly integrates physical connections, logical dependencies, and multi-copy redundancy, making the relationship between assets across computer rooms clear at a glance. Then, based on the weighted logistic regression model, the node features and relationship weights in the knowledge graph are integrated to accurately calculate the probability of node failure in the fault propagation subgraph, solving the limitations of traditional single-dimensional analysis, and supporting self-healing operations such as cross-computer room traffic redirection, data copy reconstruction, and container scheduling adjustment, achieving end-to-end intelligent operation and maintenance from fault detection to prediction and warning to closed-loop self-healing, effectively solving the problems of resource topology fragmentation in cross-computer room and multi-active site environments, the inability to integrate real-time status and alarm information with asset dependency models, and the difficulty of large-scale real-time deployment of traditional single-dimensional fault analysis and high-complexity prediction algorithms.

[0104] It should be noted that for other corresponding descriptions of the functional units involved in the IT asset fault propagation prediction system based on the dynamic evolution of the knowledge graph provided in the embodiment of the present application, please refer to Figure 1 and Figures 2A to 2B The corresponding description in will not be repeated here.

[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0106] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

[0108] In an exemplary embodiment, see Figure 4 A computer device is also provided, comprising a bus, a processor, a memory, and a communication interface. The device may also include an input / output interface and a display device, wherein the various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor is configured to execute the program stored in the memory and implement the IT asset fault propagation prediction method based on the dynamic evolution of a knowledge graph in the above-described embodiment.

[0109] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the IT asset fault propagation prediction method based on the dynamic evolution of the knowledge graph.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented through hardware or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0111] Those skilled in the art will understand that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application.

[0112] Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the implementation scenario description, or can be modified accordingly and located in one or more devices different from the implementation scenario. The modules in the above implementation scenario can be combined into one module or further split into multiple submodules.

[0113] The above application serial numbers are for description only and do not represent the advantages or disadvantages of the implementation scenarios.

[0114] The above disclosure only describes several specific implementation scenarios of the present application. However, the present application is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present application.

Claims

1. A method for predicting IT asset fault propagation based on the dynamic evolution of knowledge graph, characterized by: include: Obtaining physical connections and logical dependencies of IT assets, and constructing an initial topology using the physical connections and logical dependencies of IT assets; Collecting monitoring indicators of each topological node in the initial topology and injecting them into the initial topology to generate an initial knowledge graph, wherein the monitoring indicators include resource utilization, health status score and historical fault data; In response to resource anomalies or load scheduling events, the initial knowledge graph is updated through an asynchronous message bus and a logical clock mechanism to generate a dynamic knowledge graph; Taking the fault node in the dynamic knowledge graph as the starting point, a fault propagation subgraph is constructed based on the connectivity relationship of the computer room network links, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph; Calculating the failure probability of each node in the fault propagation subgraph based on a weighted logistic regression model; When the failure probability of a node in the fault propagation subgraph is greater than or equal to the failure threshold, a fault isolation operation is triggered, and the fault isolation operation is at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment.

2. The method according to claim 1, characterized in that The obtaining of physical connections and logical dependencies of IT assets and constructing an initial topology using the physical connections and logical dependencies of IT assets includes: Use the multi-active site topology collection component to scan IT assets in multiple computer rooms, obtaining performance configuration data, physical link connection relationships, dependencies between business components, multi-copy redundant replication relationships, and container / virtual machine affinity scheduling relationships. The IT assets include virtual machines, storage, storage clusters, network equipment, and business services. The performance configuration data, the physical link connection relationship, the dependency relationship between the business components, the multi-copy redundant replication relationship, and the container / virtual machine affinity scheduling relationship are used as the physical connection and logical dependency relationship of the IT assets; Extracting the real-time performance indicators and static configuration attributes of each knowledge graph node from the performance configuration data; The virtual machine, the container, the storage cluster, the network device, and the business service are mapped into multiple knowledge graph nodes, the physical link connection relationship, the dependency relationship between the business components, the multi-copy redundant replication relationship, and the container / virtual machine affinity scheduling relationship are mapped into edges between multiple knowledge graph nodes, and the real-time performance indicators and static configuration properties of each knowledge graph node are combined to obtain the initial topology.

3. The method according to claim 1, characterized in that The collecting monitoring indicators of each topological node in the initial topology and injecting them into the initial topology to generate an initial knowledge graph includes: Based on Prometheus instances deployed in multiple computer rooms, the resource usage of each topology node is collected and stored through the Thanos / Cortex federated architecture. Collect the health status score of each topology node based on the Consul+Gossip protocol; Collect the fault timestamp and fault type of each topological node from the centralized log system, write the fault timestamp and fault type of each topological node into the centralized log / event library, obtain the topological node fault frequency based on Alertmanage continuous statistics, and generate historical fault data for each topological node; Using the resource usage rate, health status score and historical fault data of each topological node as monitoring indicators of each topological node; When the monitoring indicators of a topological node meet the event triggering conditions, the monitoring indicators of multiple topological nodes are sent to the knowledge graph engine through Kafka, so that the knowledge graph engine uses the monitoring indicators of the multiple topological nodes to update the initial topology and generate an initial knowledge graph.

4. The method according to claim 1, wherein The response to resource anomalies or load scheduling events is to update the initial knowledge graph through an asynchronous message bus and a logical clock mechanism to generate a dynamic knowledge graph, including: In response to a resource anomaly or a load scheduling event, performing standardized processing on the resource anomaly or the load scheduling event to obtain a standardized event; Publishing the standardized event to the asynchronous message bus, configuring a vector clock for the standardized event based on the asynchronous message bus, and sending the standardized event to a corresponding topic according to the event type of the standardized event; Based on the knowledge graph engine, event information is obtained from the asynchronous message bus according to the pre-subscribed topics; Based on the knowledge graph engine, comparing the vector clock of the standardized event with the local clock of the corresponding node in the initial knowledge graph; If there is a timing conflict between the vector clock of the standardized event and the local clock of the corresponding node in the initial knowledge graph, the initial knowledge graph is corrected by a logical timestamp sorting algorithm to obtain a corrected knowledge graph; Based on the knowledge graph engine, the event information is used to perform operations of updating dynamic attributes, adjusting relationship weights, and changing topological structures on corresponding nodes in the revised knowledge graph to generate the dynamic knowledge graph.

5. The method according to claim 1, wherein The calculating the failure probability of each node in the fault propagation subgraph based on the weighted logistic regression model includes: Obtaining resource usage, health status score, and historical fault data of each node in the fault propagation subgraph, performing weighted sum calculation on the resource usage, health status score, and historical fault data of each node to obtain a comprehensive characteristic value of each node; Based on the weighted logistic regression model, the failure probability of each node is calculated using the comprehensive characteristic value of each node. P i =σ(αx i +β∑ j∈N(i) In ij and j +b), Among them, P i represents the failure probability of node i, σ(·) represents the sigmoid function, x i represents the comprehensive eigenvalue of node i, N(i) represents the set of neighboring nodes of node i, y j represents the current fault status of neighbor node j, w ij represents the association weight between node i and its neighbor node j, α represents the first model parameter, β represents the second model parameter, and b represents the third model parameter.

6. The method according to claim 5, characterized in that The method further comprises: Acquire historical fault propagation data, perform maximum likelihood estimation on the historical fault propagation data, and obtain the first model parameter, the second model parameter, and the third model parameter; or, Historical fault propagation data is obtained, and gradient descent optimization training is performed on the historical fault propagation data to obtain the first model parameter, the second model parameter, and the third model parameter.

7. The method according to claim 1, characterized in that The triggering of the fault isolation operation includes: The cross-computer room traffic redirection operation is achieved by adjusting the computer room traffic weight through a global load balancing mechanism or remapping the transmission path through an SDN controller; Identify a healthy copy in the dynamic knowledge graph that has a multi-copy redundant replication relationship with the faulty node, and deploy the target data pulled from the healthy copy to the redundant node in the dynamic knowledge graph to implement the data copy reconstruction operation; In response to container node failures, the Kubernetes scheduler dynamically changes the affinity / anti-affinity rules based on the affinity / anti-affinity rule markings of the container and the node in the dynamic knowledge graph. The Kubernetes scheduler reschedules the container based on the changed affinity / anti-affinity rules to implement the container scheduling policy adjustment operation.

8. The method according to claim 1, characterized in that The method further comprises: Use JMeter or Locust to inject failure scenarios, including IT asset resource overload, IT asset network failure, and IT asset storage failure. Collecting performance data, including response time, throughput, and error rate, and the failure probability of each node in the fault propagation subgraph; Performing a multi-dimensional comparative analysis on the performance data and the failure probability of each node to obtain a comparison result; Based on the comparison results, the gradient descent algorithm is used to optimize the model parameters of the weighted logistic regression model, and the connectivity relationship of the computer room network links, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph are adjusted.

9. An IT asset fault propagation prediction system based on the dynamic evolution of knowledge graph, characterized by: It includes topology construction module, state collection module, event evolution module, fault propagation construction module, fault prediction module, self-healing and early warning module; The topology building module is used to obtain physical connections and logical dependencies of IT assets and use the physical connections and logical dependencies of IT assets to build an initial topology; The state collection module is used to collect monitoring indicators of each topological node in the initial topology and inject them into the initial topology to generate an initial knowledge graph, wherein the monitoring indicators include resource utilization, health status score and historical fault data; The event evolution module is used to respond to resource anomalies or load scheduling events, update the initial knowledge graph through an asynchronous message bus and a logical clock mechanism, and generate a dynamic knowledge graph; The fault propagation construction module is used to construct a fault propagation subgraph based on the fault node in the dynamic knowledge graph, the connectivity relationship of the computer room network links, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph; The fault prediction module is used to calculate the fault probability of each node in the fault propagation subgraph based on a weighted logistic regression model; The self-healing and early warning module is used to trigger a fault isolation operation when the failure probability of a node in the fault propagation subgraph is greater than or equal to a fault threshold. The fault isolation operation is at least one of cross-computer room traffic redirection, data copy reconstruction, and container scheduling strategy adjustment.

10. The system according to claim 9, characterized in that It also includes a performance evaluation module for injecting failure scenarios using JMeter or Locust tools, including IT asset resource overload, IT asset network failure, and IT asset storage failure. Collecting performance data, including response time, throughput, and error rate, and the failure probability of each node in the fault propagation subgraph; Performing a multi-dimensional comparative analysis on the performance data and the failure probability of each node to obtain a comparison result; Based on the comparison results, the gradient descent algorithm is used to optimize the model parameters of the weighted logistic regression model, and the connectivity relationship of the computer room network links, the dependency relationship between business components, and the multi-copy redundant replication relationship in the dynamic knowledge graph are adjusted.

Images