Method for preventing and controlling information system operation risk of multi-dimensional solid topology and fault propagation tree

Abstract

The application provides a multi-dimensional three-dimensional topology and fault propagation tree information system operation risk prevention and control method, relates to the technical field of information system operation maintenance and security assurance, and constructs a multi-dimensional three-dimensional topology of the information system according to multi-source heterogeneous data; according to operation indexes in the multi-source heterogeneous data, health scores of each topology node in the multi-dimensional three-dimensional topology are evaluated, and risk nodes are identified according to all the health scores; a fault propagation tree is generated according to a root cause node and the multi-dimensional three-dimensional topology, nodes in the fault propagation tree are fault nodes, abnormal affected nodes are obtained according to a propagation path of the fault nodes, risk disposal operations are performed on the affected nodes, and risk disposal results are obtained; and operation rule parameters of the information system are reset through the risk disposal results. The application can perform cross-level quantitative analysis on the global operation state of the information system, so that the fault root cause positioning efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of information system operation, maintenance and security technology. More specifically, this application relates to a method for preventing and controlling information system operation risks using a multi-dimensional topology and fault propagation tree. Background Technology

[0002] With the widespread deployment of hybrid information system architectures and the rapid development of intelligent operation and maintenance technologies, the application value of information system operation, maintenance, and security technologies in the management of complex IT systems is becoming increasingly prominent. This technology enables continuous monitoring, correlation analysis, and unified management of the status, dependencies, and security events of various components, providing technical support for fault diagnosis, risk assessment, and proactive protection. This technology needs to consider cross-level dependency mapping, multi-source heterogeneous data fusion, and time-series state evolution analysis, proactively adapting to the operational needs of deeply coupled business and infrastructure scenarios under hybrid architectures.

[0003] However, existing methods for preventing and controlling operational risks in information systems largely rely on monitoring data from a single operations and maintenance platform and fixed threshold alarm mechanisms. These methods fail to fully depict the dynamic changes in cross-layer dependencies within a hybrid architecture, resulting in anomaly detection that can only match single-layer call views or static configuration ledgers. This leads to a lack of tracing the scope of cross-layer fault propagation paths and narrow root cause localization results, making it difficult to support refined fault handling and proactive risk prevention in complex heterogeneous systems. Therefore, how to conduct cross-layer quantitative analysis of the overall operational status of information systems to improve the efficiency of fault root cause localization is a key challenge facing the industry. Summary of the Invention

[0004] This application provides a method for preventing and controlling operational risks in information systems based on multidimensional topology and fault propagation tree, which can perform cross-level quantitative analysis of the overall operational status of information systems to improve the efficiency of fault root cause localization.

[0005] Firstly, this application provides a method for preventing and controlling operational risks in information systems using multi-dimensional topology and fault propagation trees, comprising the following steps: Acquire multi-source heterogeneous data generated during the operation of the information system, and construct a multi-dimensional topology of the information system based on the multi-source heterogeneous data; Based on the operational metrics in the multi-source heterogeneous data, the health score of each topological node in the multi-dimensional topology is evaluated, and risk nodes are identified based on all health scores. Starting from the risk node, a cross-layer backtracking search is performed along the inter-layer dependencies of the multi-dimensional topology until the root cause node causing the current anomaly is located. A fault propagation tree is generated based on the root cause node and the multidimensional topology. The nodes in the fault propagation tree are fault nodes. Based on the propagation path of the fault nodes, the abnormal affected nodes are obtained. Risk handling operations are performed on the affected nodes to obtain the risk handling results. The operating rule parameters of the information system are reset based on the risk handling results.

[0006] In this embodiment, constructing the multidimensional topology of the information system based on the multi-source heterogeneous data specifically includes: The call intensity between each service node is determined based on the service call chain data in the multi-source heterogeneous data. Obtain the resource deployment relationships, host ownership relationships, network association relationships, and data lineage relationships in the information system; By performing topological fusion of all call intensities with the resource deployment relationship, the host affiliation relationship, the network association relationship, and the data lineage relationship, a multi-dimensional three-dimensional topology of the information system is obtained.

[0007] In this embodiment, the topological fusion of the call intensity with the resource deployment relationship, the host affiliation relationship, the network association relationship, and the data lineage relationship to obtain the multidimensional three-dimensional topology of the information system specifically includes: Determine the dependencies between each topology node based on the call intensity and the data lineage; The mapping relationship between each topology node is determined based on the resource deployment relationship and the host affiliation relationship; The network connection relationships between each topology node are determined based on the network association relationships. The dependencies, mappings, and network connections are associated to generate a multi-dimensional topology of the information system. The dependencies and mappings form inter-layer dependencies, and the dependencies, mappings, and network connections form topological dependencies.

[0008] In this embodiment, associating the dependencies, mappings, and network connections to generate the multidimensional topology of the information system specifically includes: The operational indicators in the multi-source heterogeneous data are normalized to obtain standard operational indicators; Determine the fusion weight of each evaluation indicator in the standard operation indicators, with each evaluation indicator corresponding to a fusion weight; The health score of the topological nodes in the multidimensional topology is determined based on the standard operating indicators and the fusion weights. The operational risk of all topology nodes is classified using the health score to obtain the operational risk level; Risk nodes are selected based on the stated operational risk level.

[0009] In this embodiment, determining the fusion weight of each evaluation indicator in the standard operating indicators specifically includes: Determine the subjective and objective weights of each evaluation indicator in the standard operating indicators; The subjective weights and objective weights of each evaluation indicator are combined to obtain the combined weights of each evaluation indicator.

[0010] In this embodiment, starting from the risk node, a cross-layer backtracking search is performed along the inter-layer dependencies of the multi-dimensional topology until the root cause node leading to the current anomaly is located. Specifically, this includes: Perform dual-mode anomaly detection on the risk nodes to obtain the anomaly category of the risk nodes; Based on the anomaly category, starting from the risk node, the process traces downwards layer by layer along the inter-layer dependencies of the multi-dimensional topology, checking the operational status of each topology node at each level for anomalies, until the root cause node causing the current anomaly is located. The inter-layer dependencies include the dependency relationship and the mapping relationship.

[0011] In this embodiment, generating a fault propagation tree based on the root cause node and the multidimensional topology specifically includes: Starting from the root cause node, the topological dependencies of the multidimensional topology are expanded outward layer by layer to determine the upstream and downstream relationships and propagation paths between each topological node, forming a set of nodes and edges of the fault propagation tree. The topological dependencies include the dependency relationship, the mapping relationship, and the network connection relationship. The failure propagation probability and service impact range of each topology node are determined step by step. The failure propagation probability is used as the propagation probability label of each edge in the edge set, and the service impact range is used as the state label of each node in the node set. A fault propagation tree is generated based on the set of nodes, the set of edges, the propagation probability label, and the state label.

[0012] In this embodiment, based on the propagation path of the faulty node, the abnormal affected nodes are identified, and risk mitigation operations are performed on the affected nodes to obtain the risk mitigation results, specifically including: Multiple affected nodes are identified based on the propagation path of the faulty node in the fault propagation tree; Based on the depth and scope of each affected node in the propagation path, determine the risk level and handling priority of each affected node; Based on the risk level and handling priority of each affected node, a corresponding preset handling strategy is matched. The preset handling strategy includes at least one of fault isolation, traffic switching, service degradation and node restart. The risk management operation is performed on each affected node using the preset management strategy to obtain the risk management result.

[0013] In this embodiment, determining the risk level and handling priority of each affected node based on its hierarchical depth and impact range within the propagation path specifically includes: Obtain the propagation intensity of each edge along the propagation path from the root cause node to the affected node in the fault propagation tree. Based on the level depth of each affected node and the propagation intensity of each edge along the propagation path, obtain the fault propagation probability of each affected node. Obtain the current operating data, normal baseline value, and the proportion of service request volume carried by each affected node; assess the extent of service performance degradation based on the difference between the current operating data and the normal baseline value; and assess the range of affected traffic based on the proportion of service request volume. Based on the failure propagation probability, the extent of service performance degradation, and the range of affected traffic for each affected node, the risk level and handling priority of each affected node are determined.

[0014] In this embodiment, resetting the operating rule parameters of the information system based on the risk handling result specifically includes: The risk handling results are obtained, including the matching results of actual abnormal events and topology node alarms, the validity verification results of each health score indicator, and the comparison results of root cause location results and actual root cause information. Based on the matching results, the updated value of the anomaly detection threshold of the information system is determined; based on the validity verification results, the updated value of the health score weight of the information system is determined; and based on the comparison results, the updated value of the root cause localization strategy of the information system is determined. The operating rule parameters of the information system are reset based on the updated values ​​of the anomaly detection threshold, the health score weight, and the root cause localization strategy.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The system acquires multi-source heterogeneous data generated during the operation of the information system, and constructs a multi-dimensional topology of the information system based on the multi-source heterogeneous data. Based on the operational indicators in the multi-source heterogeneous data, it evaluates the health score of each topology node in the multi-dimensional topology and identifies risk nodes based on all health scores. Starting from the risk node, it performs a cross-layer backtracking search along the inter-layer dependencies of the multi-dimensional topology until the root cause node causing the current anomaly is located. A fault propagation tree is generated based on the root cause node and the multi-dimensional topology, where the nodes in the fault propagation tree are fault nodes. Based on the propagation path of the fault nodes, the affected nodes are identified, and risk mitigation operations are performed on the affected nodes to obtain the risk mitigation result. The system resets the operating rule parameters of the information system based on the risk mitigation result.

[0016] Therefore, this application, firstly, by acquiring multi-source heterogeneous data generated during the operation of the information system, integrates various relationships such as service call chains, resource deployment, host ownership, network associations, and data lineage to construct a multi-dimensional topology covering the front-end service layer, business service layer, cloud resource layer, and physical resource layer. This enables the heterogeneous data scattered across various operation and maintenance platforms under a hybrid architecture to be connected into a unified full-stack dependency graph, solving the problems of cross-layer dependencies not being intuitively displayed and fault propagation paths being unclear in existing technologies. It achieves a visual association between business applications and underlying infrastructure at the topology level. Secondly, by evaluating the health score of each topology node based on the operational indicators in the multi-source heterogeneous data and identifying risk nodes in an abnormal state, it can combine expert experience with real-time operation. By combining data, the system avoids generating a large number of false alarms during peak business periods using fixed threshold alarms, thus improving the accuracy of anomaly identification. Furthermore, by using risk nodes as the starting point and tracing back along the inter-layer dependencies of the multi-dimensional topology to locate the root cause node, the system can transform the root cause location of faults from relying on manual experience into an automated and quantifiable search process, significantly shortening the fault location time. Finally, by generating a fault propagation tree based on the root cause node, identifying the affected nodes and performing risk mitigation operations, and then resetting the operating rule parameters based on the mitigation results, the experience gained from each risk mitigation can be accumulated into continuous optimization of the system configuration, forming a complete risk prevention and control system. This enables efficient location and proactive prevention of cross-layer faults during the operation of hybrid architecture information systems.

[0017] In summary, the technical solution adopted in this application can perform cross-level quantitative analysis of the overall operational status of information systems, thereby improving the efficiency of fault root cause location. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is the main program flowchart of an information system operation risk prevention and control method based on a multi-dimensional three-dimensional topology and fault propagation tree provided in this application; Figure 2 It is a multi-dimensional stereoscopic topology diagram provided in this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application provides a method for preventing and controlling operational risks in information systems based on multi-dimensional topology and fault propagation trees. The core of this method involves acquiring multi-source heterogeneous data generated during the operation of the information system; constructing a multi-dimensional topology of the information system based on the multi-source heterogeneous data; evaluating the health scores of each topology node in the multi-dimensional topology based on operational indicators in the multi-source heterogeneous data; identifying risk nodes based on all health scores; performing a cross-layer backtracking search along the inter-layer dependencies of the multi-dimensional topology, starting from the risk node, until the root cause node causing the current anomaly is located; generating a fault propagation tree based on the root cause node and the multi-dimensional topology, where nodes in the fault propagation tree are fault nodes; identifying affected nodes based on the propagation path of the fault nodes; performing risk mitigation operations on the affected nodes to obtain risk mitigation results; and resetting the operational rule parameters of the information system based on the risk mitigation results.

[0022] The technical solutions provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings. Figure 1 This document presents an overall flowchart of an information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree, as provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps: In step S1, multi-source heterogeneous data generated during the operation of the information system is acquired, and a multi-dimensional topology of the information system is constructed based on the multi-source heterogeneous data.

[0023] In practice, the acquisition of multi-source heterogeneous data generated during the operation of the information system can be achieved through the following steps: First, deploy non-intrusive agent probes on each business server of the information system. Capture the request and response data between services through bytecode enhancement technology. Extract the call relationship and call count information from the request and response data between services, and use the call frequency between two service nodes per unit time as the call intensity. Then, perform deduplication, missing value imputation, outlier filtering, and protocol unification processing on the collected request and response data between services. The missing value imputation adopts the linear interpolation method, and the outlier filtering adopts the 3σ principle to remove data points that exceed the mean ± 3 times the standard deviation. The processed data is used as multi-source heterogeneous data.

[0024] In this embodiment, constructing the multidimensional topology of the information system based on the multi-source heterogeneous data can be achieved through the following steps: The call intensity between each service node is determined based on the service call chain data in the multi-source heterogeneous data. Obtain the resource deployment relationships, host ownership relationships, network association relationships, and data lineage relationships in the information system; By performing topological fusion of all call intensities with the resource deployment relationship, the host affiliation relationship, the network association relationship, and the data lineage relationship, a multi-dimensional three-dimensional topology of the information system is obtained.

[0025] It should be noted that multi-source heterogeneous data refers to breaking down data silos across various operation and maintenance platforms, providing a unified data foundation for topology construction, health assessment, and root cause analysis; service call chain data represents the call relationships and frequencies between service nodes; call intensity quantifies the tightness of dependencies between services, with higher values ​​indicating easier fault propagation along the chain; resource deployment relationships establish a vertical mapping between the business service layer and the cloud resource layer; host affiliation relationships establish a vertical mapping between the cloud resource layer and the physical resource layer; network association relationships supplement the network communication attributes of each topology node, assisting in the location and analysis of network-level faults; data lineage relationships complete implicit dependencies not driven by service calls, preventing faults caused by abnormal data flow from being missed; topology fusion integrates multiple heterogeneous relationships into a unified graph structure, transforming previously scattered dependencies into a computable whole; and multi-dimensional topology serves as the spatial foundation for full-stack dependencies.

[0026] In specific implementation, firstly, the call frequency between two service nodes per unit time in the service call chain data of multi-source heterogeneous data is used as the call intensity between the corresponding service nodes. Secondly, resource deployment relationships are synchronously obtained through the CMDB (Configuration Management Database) interface. The CMDB stores the configuration information of IT infrastructure and the relationships between configuration items. By calling the CMDB's REST API (Representational State Transmission Application Programming Interface), the deployment mapping relationship between business service layer nodes and cloud resource layer nodes is queried, and the returned results include fields such as service name, container / virtual machine identifier, and deployment time. Host ownership relationships are also synchronously obtained through the CMDB interface. The ownership mapping relationship between cloud resource layer nodes and physical resource layer nodes is queried through the CMDB, and the returned results include fields such as container / virtual machine identifier, physical machine identifier, data center, and rack. Network relationships are obtained through port scanning using Nmap (a network mapping tool), combined with network topology discovery methods such as SNMP (Simple Network Management Protocol) query and LLDP (Link Layer Discovery Protocol) parsing. Specifically, Nmap is used to perform TCP mapping on the target network segment. SYN (Transmission Control Protocol Synchronization) scanning and UDP (User Datagram Protocol) probing are used to identify active hosts and their open ports and services; SNMP (Simplified Chinese SNMP) queries are used to query the ARP (Address Resolution Protocol) and MAC (Media Access Control) address tables of network devices to obtain Layer 2 connection relationships between devices; LLDP (Limited Link Protocol) is used to parse information about adjacent devices to obtain direct link connections between devices; network connection relationships are established between each topology node, including IP addresses, ports, communication protocols, and network latency information; data lineage is obtained by parsing data flow logs and data access records, collecting data processing task logs, API (Application Programming Interface) call records, and database audit logs from various business services, and extracting the data source, flow, processing process, and final purpose from the logs. Information such as data flow paths between different systems and services is analyzed to obtain the upstream and downstream dependencies between topology nodes during data flow, which are then used as data lineage relationships. Finally, the call frequency between two service nodes per unit time in the service call chain data is used as the call intensity. This is combined with resource deployment relationships and host ownership relationships synchronized from the configuration management database, network association relationships obtained through network scanning, and data lineage relationships obtained by parsing data flow logs. The unique identifier of each topology node is used as the key value for association alignment and deduplication merging. Finally, the nodes are classified and stored according to four layers: front-end service layer, business service layer, cloud resource layer, and physical resource layer. The edges are classified and stored according to three types: dependency edges, mapping edges, and network connection edges, generating a multi-dimensional three-dimensional topology covering the four-layer architecture.

[0027] In this embodiment, the multi-dimensional topology of the information system is obtained by topologically fusing all call intensities with the resource deployment relationships, host affiliation relationships, network association relationships, and data lineage relationships. This can be achieved through the following steps: Determine the dependencies between each topology node based on the call intensity and the data lineage; The mapping relationship between each topology node is determined based on the resource deployment relationship and the host affiliation relationship; The network connection relationships between each topology node are determined based on the network association relationships. The dependencies, mappings, and network connections are associated to generate a multi-dimensional topology of the information system. The dependencies and mappings form inter-layer dependencies, and the dependencies, mappings, and network connections form topological dependencies.

[0028] It should be noted that dependency relationships are the path basis for locating the root cause from risk nodes level by level in cross-layer backtracking search; mapping relationships are the vertical correspondence between the business service layer and the cloud resource layer, and between the cloud resource layer and the physical resource layer; network connection relationships refer to the network connectivity and communication link relationships between each topology node.

[0029] In specific implementation, firstly, a first mapping relationship is determined between business service layer nodes and cloud resource layer nodes based on resource deployment relationships. This first mapping relationship records the deployment correspondence between each service node in the business service layer and the container or virtual machine node in the cloud resource layer. Secondly, a second mapping relationship is determined between cloud resource layer nodes and physical resource layer nodes based on host affiliation relationships. This second mapping relationship records the affiliation correspondence between each container or virtual machine node in the cloud resource layer and the physical machine node in the physical resource layer. When concatenating the first and second mapping relationships, the business service layer node identifier and the corresponding cloud resource layer node identifier of each record in the first mapping relationship are extracted, and the cloud resource layer node identifier and the corresponding physical resource layer node identifier of each record in the second mapping relationship are extracted. Using the cloud resource layer node identifier as the association field, records with the same cloud resource layer node identifier in the first and second mapping relationships are matched. The business service layer node identifier, cloud resource layer node identifier, and physical resource layer node identifier in the successfully matched records are combined into a three-segment mapping record. After traversing all records and completing the matching, a complete mapping path is formed, starting from the business service layer node, passing through the cloud resource layer node, and reaching the physical resource layer node. After cascading, the mapping relationship between each topology node is obtained. Then, the IP address and open port list of each topology node are extracted from the Nmap port scan results, the communication protocol type of each topology node is extracted from the SNMP protocol query results, and the link delay information between adjacent topology nodes is extracted from the LLDP protocol parsing results. After summarizing the above information, the network connection relationship between each topology node is established using the source node identifier and the target node identifier as keys. Each record contains the source node IP, target node IP, communication protocol type, port number, link delay, and connection status. Finally, the dependencies, mappings, and network connections are aggregated and merged using the unique identifier of each topology node as the key. Duplicate node identifiers are deduplicated, and for cases where multiple relationships exist between the same node pair, all relationship types are retained and labeled separately, forming a node set and an edge set. Nodes are classified according to four layers: front-end service layer, business service layer, cloud resource layer, and physical resource layer. Edges are classified according to three types: dependency edges, belonging edges, and network connection edges. Nodes and edges are created in the database, and a layer label is set for each node. A relationship type label and weight attribute are set for each edge, generating a multi-dimensional topology covering the front-end service layer, business service layer, cloud resource layer, and physical resource layer.

[0030] In step S2, the health score of each topological node in the multidimensional topology is evaluated based on the operating indicators in the multi-source heterogeneous data, and risk nodes in abnormal states are identified based on all the health scores.

[0031] In this embodiment, the health score of each topological node in the multidimensional topology is evaluated based on the operational indicators in the multi-source heterogeneous data, and risk nodes in an abnormal state are identified based on all health scores. This can be achieved through the following steps: The operational indicators in the multi-source heterogeneous data are normalized to obtain standard operational indicators; Determine the fusion weight of each evaluation indicator in the standard operation indicators, with each evaluation indicator corresponding to a fusion weight; The health score of the topological nodes in the multidimensional topology is determined based on the standard operating indicators and the fusion weights. The operational risk of all topology nodes is classified using the health score to obtain the operational risk level; Risk nodes are selected based on the stated operational risk level.

[0032] It should be noted that: operational metrics refer to the raw measurement data reflecting the real-time operational status of each topology node; normalization processing means mapping the raw data of each operational metric to the [0,1] interval using the Min-Max normalization method to eliminate the influence of different dimensions between metrics; standard operational metrics refer to the dimensionless values ​​corresponding to each operational metric after normalization processing; fusion weight refers to the comprehensive weight used for the objectivity of data in health score calculation; health score is a comprehensive quantitative score that measures the operational health of each topology node; operational risk level represents the level of operational risk of topology nodes based on health score, including four levels: low, medium, high, and severe; risk node refers to a topology node with an operational risk level of high or severe.

[0033] In practice, the process begins with deploying non-intrusive agent probes on each business server of the information system. These probes periodically collect operational metrics data from the server's operating system and applications. For server-level metrics, the probes obtain CPU and memory utilization by reading the operating system's / proc (process file system), disk I / O wait times by reading disk I / O statistics, and network packet loss rate by reading network interface statistics. For application-level metrics, the probes intercept application request entry and exit points, counting the total number of requests, erroneous requests, and slow requests per unit time. They then calculate the error rate and slow request ratio, where the error rate is calculated as the number of erroneous requests divided by the total number of requests, and the slow request ratio is calculated as the number of requests with response times exceeding a threshold divided by the total number of requests. The response time for each request is recorded to calculate the average response time and throughput fluctuations. The agent probes collect the above eight indicator data at uniform time intervals. In this embodiment, the interval can be set to 1 minute, and the data is reported to the data aggregation terminal of the application performance monitoring platform via HTTP (Hypertext Transfer Protocol) or gRPC (Google Remote Procedure Call). The data aggregation terminal merges the data reported by each agent according to node identifiers to form the raw operating indicator dataset for each topology node within a unified time window. Then, the raw operating indicator data of each topology node is subjected to Min-Max normalization. For each indicator, the minimum value of the raw operating indicator within the observation period is taken as the lower limit and the maximum value as the upper limit. The normalized indicator value is calculated according to the following formula: in The current value, This represents the minimum value of the indicator within the observation period. This represents the maximum value of the indicator within the observation period. These are the normalized indicator values. After normalization, all indicator values ​​fall within the [0,1] interval, eliminating the influence of different dimensions between indicators. The normalized indicator values ​​are used as the standard operating indicators. Then, for the eight indicators—response time, error rate, throughput fluctuation, slow requests, CPU utilization, memory utilization, disk I / O wait, and network packet loss rate—historical operating data for each indicator within the observation period is obtained. The initial weights of each indicator are determined based on the degree of fluctuation and the amount of information in the historical data. Indicators with greater fluctuation and higher information content are assigned higher weights. The initial weights are then adjusted by combining the historical fault records of the information system operation. The weights of indicators with high correlation to the system health status are appropriately increased, and the weights of indicators with low correlation are appropriately decreased. The adjusted indicator weights are then normalized so that the sum of all weights equals 1, resulting in the fused weights of each evaluation indicator. Secondly, the eight standard operational indicators of each topology node are weighted and summed with their corresponding weights to obtain a comprehensive anomaly index. This comprehensive anomaly index is then mapped to the [0,1] interval to obtain a health score H. A value closer to 1 indicates a healthier node operation, while a value closer to 0 indicates a worse node operation. Finally, the operational risk of all topology nodes is classified using the health score to obtain an operational risk level. A risk level of H ≥ 0.8 is low, 0.6 ≤ H < 0.8 is medium, 0.4 ≤ H < 0.6 is high, and H < 0.4 is severe. Topology nodes with high or severe risk levels are identified as risk nodes and used as the starting point for subsequent cross-layer backtracking searches.

[0034] In this embodiment, the fusion weight of each evaluation indicator in the standard operating indicators can be determined by the following steps: Determine the subjective and objective weights of each evaluation indicator in the standard operating indicators; The subjective weights and objective weights of each evaluation indicator are combined to obtain the combined weights of each evaluation indicator.

[0035] It should be noted that subjective weights reflect the prior judgment of the importance of the indicator by domain knowledge; objective weights reflect the degree of difference and information content of the data itself, and the greater the difference, the higher the weight of the indicator.

[0036] In practical implementation, the first step is to determine the subjective weights of each evaluation metric. Using the Analytic Hierarchy Process (AHP), the importance of eight metrics—response time, error rate, throughput fluctuation, slow requests, CPU utilization, memory utilization, disk I / O wait, and network packet loss rate—is compared pairwise to construct a judgment matrix and calculate the subjective weight vector for each metric. Then, the objective weights of each evaluation indicator are determined. The Entropy Weight Method (EW) is used to calculate the information entropy of each indicator based on its historical performance data during the observation period. The objective weight vector of each indicator is then determined using this information entropy. The lower the information entropy, the greater the difference in the indicator data and the greater the amount of information contained, and the higher the corresponding weight. Finally, subjective weights and objective weights are merged. The merging method is to combine the subjective weights... With objective weight The weighted sums are calculated using a weighted average of 0.5:0.5 to obtain the combined weight of each evaluation indicator. The formula is .

[0037] In step S3, starting from the risk node, a cross-layer backtracking search is performed along the inter-layer dependencies of the multi-dimensional topology until the root cause node causing the current anomaly is located. In this embodiment, starting from the risk node, a cross-layer backtracking search is performed along the inter-layer dependencies of the multi-dimensional topology until the root cause node leading to the current anomaly is located. This can be achieved through the following steps: Perform dual-mode anomaly detection on the risk nodes to obtain the anomaly category of the risk nodes; Based on the anomaly category, starting from the risk node, the process traces downwards layer by layer along the inter-layer dependencies of the multi-dimensional topology, checking the operational status of each topology node at each level for anomalies, until the root cause node causing the current anomaly is located. The inter-layer dependencies include the dependency relationship and the mapping relationship.

[0038] It should be noted that dual-mode anomaly detection refers to the comprehensive anomaly judgment of risk nodes by two mechanisms; the anomaly category is output by dual-mode anomaly detection, including error rate surge, success rate decline, response time exceedance, and latency jitter; the inter-layer dependency relationship is a vertical dependency chain used to support the tracing of anomalies from the level where the risk node is located downwards; the root cause node represents the lowest-level fault source node that caused the current anomaly.

[0039] In practical implementation, the dual-mode anomaly detection first includes two detection mechanisms: a dynamic threshold-based detection mode and a statistical model-based detection mode. The dynamic threshold-based detection mode is used to identify anomalies such as sudden increases in error rate and decreases in success rate. This mode employs a sliding window dynamic threshold, with the window size configured by operations personnel through the system's basic settings page. The window size ranges from 5 to 30 minutes, using a smaller value during peak business hours to improve sensitivity, a middle value during normal periods, and a larger value during off-peak periods to reduce false alarms. Within the sliding window, the error rate and success rate of each service node are statistically analyzed, where error rate = number of erroneous requests / total number of requests, and success rate = number of successful requests / total number of requests. The statistical values ​​of the current time window are compared with historical baseline values. When the error rate exceeds the sum of the historical baseline value and the dynamic threshold, a sudden increase in error rate is detected; when the success rate is lower than the difference between the historical baseline value and the dynamic threshold, a decrease in success rate is detected. The statistical model-based detection mode is used to identify anomalies such as response time exceeding limits and latency jitter. The statistical model uses an nσ model for detection, where the σ value is calculated based on historical data from the same time period over the past 7 days. For the response time data of each service node, the mean and standard deviation σ of the response time for the same time period over the past 7 days are calculated. When the current response time exceeds the sum of the mean and three times σ, a response time exceeding limit anomaly judgment is triggered; when the standard deviation or fluctuation of the current response time exceeds twice the standard deviation of the historical time period, a latency jitter anomaly judgment is triggered. When either detection mode triggers an anomaly judgment, the risk node is marked as an anomalous node, and the corresponding anomaly category is output according to the triggered detection mode. The anomaly category triggered by the dynamic threshold detection mode is either a sudden increase in error rate or a decrease in success rate; the anomaly category triggered by the statistical model detection mode is either a response time exceeding limit or latency jitter, thus obtaining the anomaly category of the risk node. Next, the search strategy for cross-layer backtracking is determined based on the anomaly category. If the anomaly category is a sudden increase in error rate or a decrease in success rate, the investigation should first trace down from the layer where the risk node is located to the cloud resource layer, checking the CPU utilization, memory utilization, and error logs of containers or virtual machines in the cloud resource layer to confirm whether there is resource contention or container anomaly. If the anomaly category is a response time exceeding limit or latency jitter, the investigation should first trace down from the layer where the risk node is located to the physical resource layer, checking the CPU utilization, memory utilization, disk I / O wait time, and network packet loss rate of physical machines in the physical resource layer to confirm whether there is a physical resource bottleneck or network failure. Starting from the risk node, the investigation should proceed layer by layer down along the inter-layer dependencies of the multi-dimensional topology.If the risk node is located in the business service layer, the first step is to check whether the operating status of the cloud resource layer nodes deployed by the business service is abnormal. This check includes the CPU utilization, memory utilization, disk I / O wait time, and network packet loss rate of the cloud resource layer nodes. If the operating status of the cloud resource layer nodes is abnormal, the process is further traced down to the physical resource layer to check whether the operating status of the physical machines to which the cloud resource layer nodes belong is abnormal. This check includes the CPU utilization, memory utilization, disk I / O wait time, network packet loss rate, response time, and error rate of the physical machines. During this layer-by-layer tracing process, the operating status of each layer's topology nodes is checked sequentially. When tracing along the inter-layer dependencies to a node at a certain level, if the operating status of all lower-level nodes of that node is normal, but the node itself is abnormal, then that node is identified as the root cause node. If tracing back to the physical resource layer still reveals abnormal operating status of physical resource layer nodes, then the abnormal physical machine node in the physical resource layer is identified as the root cause node. This identifies the root cause node leading to the current anomaly.

[0040] In step S4, a fault propagation tree is generated based on the root cause node and the multidimensional topology. The nodes in the fault propagation tree are fault nodes. Based on the propagation path of the fault nodes, the abnormal affected nodes are obtained. Risk handling operations are performed on the affected nodes to obtain the risk handling results.

[0041] In this embodiment, generating a fault propagation tree based on the root cause node and the multidimensional topology can be achieved through the following steps: Starting from the root cause node, the topological dependencies of the multidimensional topology are expanded outward layer by layer to determine the upstream and downstream relationships and propagation paths between each topological node, forming a set of nodes and edges of the fault propagation tree. The topological dependencies include the dependency relationship, the mapping relationship, and the network connection relationship. The failure propagation probability and service impact range of each topology node are determined step by step. The failure propagation probability is used as the propagation probability label of each edge in the edge set, and the service impact range is used as the state label of each topology node in the node set. A fault propagation tree is generated based on the set of nodes, the set of edges, the propagation probability label, and the state label.

[0042] It should be noted that topological dependency is the set of dependencies that serve as the basis for expanding the search path outward from the root cause node; fault propagation probability is a measure of the likelihood of a fault propagation path being strong; business impact range refers to the degree of impact on business operations after the fault propagates to each topological node; status label is the current anomaly type of the topological node and the business scope affected by its fault; and fault propagation tree is a tree structure used to visualize the fault impact range and propagation path.

[0043] In practical implementation, firstly, the multidimensional topology is stored in a graph database, containing a set of nodes and a set of edges. Nodes are organized into four layers: front-end service layer, business service layer, cloud resource layer, and physical resource layer. Each edge is labeled with a type label (dependency edge, belonging edge, or network connection edge) and a corresponding propagation strength weight. The propagation strength weight is calculated based on the call intensity between two nodes, resource mapping relationship, and network communication quality. Starting from the identified root cause node, all associated edges of that node in the multidimensional topology are queried from the graph database. For each edge found, its type label and propagation strength weight are obtained. The other end node of this edge is designated as the downstream node of the current node, and the upstream-downstream relationship between the current node and this downstream node is recorded as a propagation path. Then, using this downstream node as the new current node, the query continues to retrieve its associated edges in the multidimensional topology, but with the following constraint: the direction of the edge radiates outward from the root cause node, without backtracking to already visited nodes, to avoid loops in the propagation path. When a node is reached, if all associated edges of that node have been visited, the expansion outward from that node stops. Repeat the above query and expansion process until all reachable nodes have been visited. Record all nodes traversed along the propagation path as the node set of the fault propagation tree, and record all edges traversed along the propagation path as the edge set of the fault propagation tree. Specifically, starting from the root cause node, first visit all topological nodes directly adjacent to the root cause node, and treat these directly adjacent topological nodes as the first-level topological nodes. Then, starting from the first-level topological nodes, continue to visit topological nodes adjacent to the first-level topological nodes that have not yet been visited, and treat these topological nodes as the second-level topological nodes. Expand outward layer by layer in the above manner until all reachable topological nodes have been traversed. During the layer-by-layer outward expansion process, calculate the fault propagation probability of each topological node at each level. For the first-level topological nodes, the formula for calculating the fault propagation probability is: ,in This represents the conditional probability of the current node failing given that the root cause node has failed. This conditional probability is determined based on the fault statistics accumulated from the chaotic fault injection exercise. The edge weight is determined based on the ratio of call frequency between components; the higher the call frequency, the greater the edge weight. For topology nodes at the second level and higher, the fault propagation probability is the product of the propagation probabilities of each edge along the propagation path. Based on the fault propagation probability, the business impact range of each topology node is evaluated. The business impact range includes two dimensions: the magnitude of business performance degradation ΔR and the range of affected traffic ΔQ. The magnitude of business performance degradation is determined based on the increase in response time and the increase in error rate for each topology node; the greater the increase in response time and the higher the increase in error rate, the greater the magnitude of business performance degradation. The range of affected traffic is determined based on the proportion of business requests carried by each topology node; the higher the fault propagation probability and the greater the proportion of business requests carried by the node, the wider the range of affected traffic. Finally, the fault propagation probability and business impact range corresponding to each topology node are obtained. Then, with the root cause node as the root node and all topology nodes traversed during the layer-by-layer expansion process as child nodes, a node set for the fault propagation tree is constructed. For each child node in the node set, the edge between it and its parent node is determined based on the path accessed by that node during the layer-by-layer expansion process, forming the edge set of the fault propagation tree. For each edge in the edge set, the probability of fault propagation is... The propagation probability label is used as the edge's propagation probability label. For each node in the node set, the risk status of the node is determined based on the fault propagation probability and the scope of business impact. Nodes with a fault propagation probability higher than a preset threshold and a business performance degradation exceeding a preset threshold are marked as high-risk, while nodes with a fault propagation probability lower than a preset threshold or a business performance degradation not exceeding a preset threshold are marked as low-risk. The node set, edge set, edge propagation probability labels, and node risk status are summarized to generate a fault propagation tree with the root cause node as the root node. Next, multiple affected nodes are determined based on the propagation path of the fault node in the fault propagation tree. All nodes in the fault propagation tree are collectively referred to as fault nodes, including the root cause node and all its child nodes. Starting from the root cause node, the propagation path of the fault propagation tree is traversed layer by layer downwards, and each fault node encountered during the traversal is filtered. For each traversed fault node, the fault propagation probability and the business performance degradation of the fault node are obtained. The fault propagation probability of the fault node is compared with a preset probability threshold, and the business performance degradation of the fault node is compared with a preset degradation threshold. When the failure propagation probability of a faulty node exceeds a preset probability threshold and the service performance degradation exceeds a preset magnitude threshold, the faulty node is identified as an affected node. If the failure propagation probability of a faulty node does not exceed the preset probability threshold or the service performance degradation does not exceed the preset magnitude threshold, the faulty node is not considered an affected node. After traversing all faulty nodes in the above manner, multiple affected nodes are obtained. Next, the failure propagation probability of each affected node is determined based on its depth in the failure propagation tree. The root cause node has a depth of 0, its direct downstream nodes have a depth of 1, and so on, increasing layer by layer. The failure propagation probability of any node is the product of the probabilities along each edge of the propagation path; a smaller depth indicates a closer proximity to the fault source and a higher propagation probability. Based on the propagation probability, the service performance degradation and affected traffic range of each node are assessed. The former is determined by the increase in response time and the increase in error rate, while the latter is determined by the proportion of service requests carried by that node. The risk level is determined by comprehensively comparing the magnitude of the performance degradation and the affected traffic range with preset thresholds. The risk level is categorized into four levels: low, medium, high, or severe. Higher risk levels correspond to higher handling priorities. This process iterates through all affected nodes to obtain the risk level and handling priority for each node. Simultaneously, preset handling strategies are pre-stored in a strategy database. Each strategy record in the database includes an applicable risk level field, an applicable handling priority field, and a handling action field. When matching a preset handling strategy is required, the risk level and handling priority of the affected node are used as query conditions. The database is then searched for strategy records with the same applicable risk level and handling priority as the current node. The handling action field of that strategy record is then used as the preset handling strategy for that affected node.For the affected nodes with the highest risk level, the pre-defined handling strategy is a combination of fault isolation and traffic switching. Fault isolation is achieved by triggering emergency plan-related operations, specifically by sending the identifier and abnormal status information of the affected node to the emergency plan management system, which then returns the corresponding emergency plan document and a list of contact persons. Simultaneously, a traffic switching operation is triggered, switching the business traffic of the affected node to a backup node or backup service instance. For affected nodes with a high risk level, the pre-defined handling strategy is service degradation, specifically achieved by triggering automatic scaling or rate limiting operations. This involves sending a scaling request to the container orchestration platform to increase the number of replicas of the service instance where the affected node resides, or setting traffic limiting rules at the service entry point of the affected node to limit the number of requests exceeding a preset threshold, thereby freeing up system resources to ensure the availability of core business operations. For affected nodes with a medium or low risk level, the pre-defined handling strategy is node restart. This is achieved by triggering an alarm push or generating an operation and maintenance work order. Specifically, the node identifier, risk level, anomaly category, and fault propagation path of the affected node are assembled into an alarm message, which is sent to the receiving end of the operation and maintenance personnel via a message push channel. Simultaneously, an operation and maintenance work order is created by calling the work order system interface. The work order content includes the node identifier, risk level, matching pre-defined handling strategy, and recommended handling measures of the affected node. The operation and maintenance personnel then perform a restart operation on the faulty node based on the alarm or work order information. After traversing all affected nodes and matching the corresponding pre-defined handling strategy for each affected node, the pre-defined handling strategy for each affected node is obtained. After the risk handling operation is completed, the handling information for each affected node is recorded, including the handling operation type, handling object, handling time, and recovery status. The handling operation type record specifies which handling operation was performed, including one or more of the following: emergency plan association, traffic switching, automatic capacity expansion, rate limiting, alarm push, or maintenance work order generation; the handling object record identifies the affected node from which the handling operation was performed; the handling time record records the length of time from anomaly discovery to handling completion; and the recovery status record records whether the node has returned to normal operation after handling. The risk handling result is obtained by summarizing the handling information for each affected node.

[0044] In this embodiment, based on the propagation path of the faulty node, the abnormal affected nodes are obtained, and risk mitigation operations are performed on the affected nodes to obtain the risk mitigation results. This can be achieved through the following steps: Multiple affected nodes are identified based on the propagation path of the faulty node in the fault propagation tree; Based on the depth and scope of each affected node in the propagation path, determine the risk level and handling priority of each affected node; Based on the risk level and handling priority of each affected node, a corresponding preset handling strategy is matched. The preset handling strategy includes at least one of fault isolation, traffic switching, service degradation and node restart. The risk management operation is performed on each affected node using the preset management strategy to obtain the risk management result.

[0045] It should be noted that fault isolation refers to cutting off the communication connection between the faulty node and other parts of the system to prevent the fault from spreading to upstream and downstream nodes along the topology dependency; traffic switching refers to scheduling the business request traffic of the faulty node to other healthy nodes that are operating normally, so that the healthy nodes can take over the business load of the faulty node; service degradation refers to shutting down the non-core functions of the faulty node or the nodes affected by the fault, releasing system resources to ensure the availability and response speed of core businesses; node restart refers to performing a restart operation on the faulty node to restore the faulty node to its initial normal operating state.

[0046] In specific implementation, firstly, the node set and edge set of the fault propagation tree are obtained. Nodes without upstream parent nodes are selected from the node set as root nodes. Starting from the root node, the path is traversed downwards along the edge set to identify all propagation paths from the root node to each leaf node. For each propagation path, all nodes on the path except the root node are extracted as affected nodes. After summarizing the affected nodes extracted from each propagation path, the nodes are deduplicated and merged using the unique node identifier as the key. When the same node appears repeatedly in multiple propagation paths, only one node instance is retained. Multiple affected nodes and their associated information are output. The associated information includes the node's level, level depth, incoming edge propagation strength, and state label. Then, the depth of each affected node in the fault propagation tree is obtained. The depth of the depth is the number of edges in the propagation path from the root cause node to that node. A larger depth indicates that the node is farther from the fault source, and the greater the latency and uncertainty affected by the fault propagation. The status label of each affected node is then obtained. The status label includes the proportion of the service request volume carried by that node to the total service request volume of the system, which serves as the scope of service impact. A larger proportion of service request volume indicates a higher degree of impact of the node's failure on the overall system service. The risk level is then determined using the following method: using the depth of ... The scope serves as a quantitative indicator of the severity of the fault. Both are input into a preset risk assessment matrix. This risk assessment matrix is ​​divided into multiple propagation probability levels according to hierarchical depth from shallow to deep, and multiple severity levels according to the scope of business impact from large to small. Based on the combination of different propagation probability levels and different severity levels, a corresponding risk level is mapped. Then, the handling priority is obtained based on the risk level mapping. The higher the risk level, the higher the handling priority. The handling priority is used to determine the handling order of each affected node when multiple nodes are waiting to be handled at the same time. The node with the highest risk level is handled first. Finally, the risk level and handling priority of each affected node are output.Secondly, the system has a pre-defined mapping relationship between risk levels and handling strategies. For nodes with the highest risk level, fault isolation is applied, cutting off the communication connection between the node and other parts of the system to prevent the fault from spreading upstream and downstream along the topology dependency. For nodes with a medium risk level, traffic switching is applied, redirecting the business request traffic of the faulty node to other normally operating healthy nodes, which then take over the business load of the faulty node. For nodes with a low risk level and that are in an overloaded state, service degradation is applied, shutting down the non-core functions of the node to release system resources and ensure the availability of core services. For nodes with the lowest risk level and that are in a suspended or deadlocked state, node restart is applied, performing a restart operation to restore the node to its initial normal operating state. Among these, fault isolation is used to cut off the communication connection between the faulty node and other parts of the system to prevent the fault from spreading upstream and downstream along the topology dependency; traffic switching is used to redirect the business request traffic of the faulty node to other normally operating healthy nodes to take over the business load of the faulty node; service degradation is used to shut down the non-core functions of the faulty node or the nodes affected by the fault to release system resources and ensure the availability and response speed of core services; and node restart is used to perform a restart operation on the faulty node to restore it to its initial normal operating state.Finally, risk mitigation operations are executed sequentially according to the priority of each affected node, from highest to lowest. For multiple affected nodes with the same priority, operations are executed sequentially according to their risk level, from highest to lowest. If the preset mitigation strategy is fault isolation, access control list rules or firewall rules are issued to the network device corresponding to the node to cut off network communication between the node and other parts of the system, and the node's operating status is continuously monitored within the preset isolation observation time window. If the preset mitigation strategy is traffic switching, traffic scheduling instructions are issued to the load balancer or service mesh corresponding to the node to switch the node's business request traffic to other normally operating healthy nodes, so that the healthy nodes can take over the node's business load. If the preset mitigation strategy is service degradation, degradation instructions are issued to the application service corresponding to the node to shut down the node's non-core functional modules, leaving only the core functional modules running. If the preset mitigation strategy is node restart, restart instructions are issued to the cloud management platform or operating system corresponding to the node. The node performs a restart operation and verifies whether it has returned to normal operation after restarting. Before each risk mitigation operation, the current operating status data of the affected node is recorded as a pre-management status snapshot. After the risk mitigation operation, the node's operating status continues to be monitored, and the post-management operating status data is recorded as a post-management status snapshot. The pre-management status snapshot and the post-management status snapshot are compared to generate a single-node management record for the node. The single-node management record includes the management strategy type executed by the node, the management start time, the management end time, a comparison of key operating indicators before and after management, and the judgment result of whether the management was successful. After all affected nodes to be managed have completed the risk mitigation operation, the single-node management records of each node are summarized to generate the risk mitigation result. The risk mitigation result includes the management record of each affected node, the overall management success rate, a comprehensive judgment information on whether the system operating status has returned to normal level after management, and a list of affected nodes that were not successfully managed.

[0047] In this embodiment, determining the risk level and handling priority of each affected node based on its hierarchical depth and influence range in the propagation path can be achieved through the following steps: Obtain the propagation intensity of each edge along the propagation path from the root cause node to the affected node in the fault propagation tree. Based on the level depth of each affected node and the propagation intensity of each edge along the propagation path, obtain the fault propagation probability of each affected node. Obtain the current operating data, normal baseline value, and the proportion of service request volume carried by each affected node; assess the extent of service performance degradation based on the difference between the current operating data and the normal baseline value; and assess the range of affected traffic based on the proportion of service request volume. Based on the failure propagation probability, the extent of service performance degradation, and the range of affected traffic for each affected node, the risk level and handling priority of each affected node are determined.

[0048] It should be noted that the performance degradation rate indicates the degree of performance degradation of a node due to fault propagation; the affected traffic range indicates the proportion of business requests carried by the node that are affected by the fault.

[0049] In the specific implementation, firstly, in the fault propagation tree, the root cause node has a depth of 0, and the direct downstream nodes of the root cause node have a depth of 1. For each edge traversed from the root cause node to the next level node, the depth of that node increases by 1. The depth reflects the propagation distance between the affected node and the root cause node. An affected node with a depth of 1 directly depends on the root cause node; the root cause node's fault will directly affect this node. Therefore, the fault propagation probability of this node is determined by the propagation probability of a single edge from the root cause node to this node. An affected node with a depth of 2 indirectly depends on the root cause node through an intermediate node. The root cause node's fault needs to propagate through two edges to reach this node; therefore, the fault propagation probability of this node is the product of the two edge probabilities. As the depth increases, the fault needs to propagate through more intermediate nodes to reach the target node. Each edge traversal requires multiplying by a new propagation probability value; therefore, the fault propagation probability decreases layer by layer. After traversing all affected nodes and calculating the corresponding fault propagation probability for each affected node, the fault propagation probability of each affected node is obtained. The extent of performance degradation is determined by two factors: the increase in response time and the increase in error rate. For each affected node, the average response time and error rate within the current time window are obtained. The average response time is compared to the baseline value under normal operating conditions to calculate the increase in response time; the error rate is compared to the baseline value under normal operating conditions to calculate the increase in error rate. The larger the increase in response time and the higher the increase in error rate, the greater the performance degradation. The affected traffic range is determined by the proportion of business requests handled by the affected node. The proportion of business requests is the ratio of the number of requests currently being processed by the affected node to the total number of requests in the information system. The larger the proportion, the larger the affected traffic range. Affected nodes with a higher probability of fault propagation typically have higher performance degradation and affected traffic range values; affected nodes with a lower probability of fault propagation typically have lower performance degradation and affected traffic range values. After traversing all affected nodes, the performance degradation and affected traffic range for each affected node are obtained. Finally, six preset thresholds are used as the judgment criteria, where the first preset threshold is greater than the third preset threshold, which is greater than the fifth preset threshold, and the second preset threshold is greater than the fourth preset threshold, which is greater than the sixth preset threshold.When the decline in service performance exceeds the first preset threshold and the affected traffic range exceeds the second preset threshold, the risk level is determined to be severe; when the decline in service performance exceeds the third preset threshold but does not exceed the first preset threshold, or the affected traffic range exceeds the fourth preset threshold but does not exceed the second preset threshold, the risk level is determined to be high; when the decline in service performance exceeds the fifth preset threshold but does not exceed the third preset threshold, or the affected traffic range exceeds the sixth preset threshold but does not exceed the fourth preset threshold, the risk level is determined to be medium; when the decline in service performance does not exceed the fifth preset threshold and the affected traffic range does not exceed the sixth preset threshold, the risk level is determined to be low. When the magnitude of the service performance degradation is inconsistent with the risk level corresponding to each of the affected traffic ranges, the final risk level is determined according to a preset priority rule: the risk level corresponding to the magnitude of the service performance degradation is used as the first reference level, and the risk level corresponding to the affected traffic range is used as the second reference level; the determination priority of the magnitude of the service performance degradation is higher than the determination priority of the affected traffic range; when the first reference level and the second reference level are inconsistent, the first reference level is used as the final risk level; when only one dimension of the determination result of the magnitude of the service performance degradation and the affected traffic range reaches the severity level while the other dimension does not, the severity level is used as the final risk level. The handling priority is determined according to the risk level, with severity level having the highest priority, followed by high level, and then medium and low levels in that order. After traversing all affected nodes, the risk level and handling priority of each node are obtained.

[0050] In step S5, the operating rule parameters of the information system are reset based on the risk handling result.

[0051] The risk handling results are obtained, including the matching results of actual abnormal events and topology node alarms, the validity verification results of each health score indicator, and the comparison results of root cause location results and actual root cause information. Based on the matching results, the updated value of the anomaly detection threshold of the information system is determined; based on the validity verification results, the updated value of the health score weight of the information system is determined; and based on the comparison results, the updated value of the root cause localization strategy of the information system is determined. The operating rule parameters of the information system are reset based on the updated values ​​of the anomaly detection threshold, the health score weight, and the root cause localization strategy.

[0052] It should be noted that the operation rule parameters refer to the judgment criteria and configuration items that the information system relies on during the execution of operation risk prevention and control; the root cause location strategy is a rule used to guide the tracing of the root cause node from the risk node down the inter-layer dependency relationship; the operation rule parameter reset value is a new parameter value used to write to the configuration center to guide the subsequent risk prevention and control process.

[0053] In specific implementation, firstly, records of abnormal events actually occurring within a preset time window after the risk handling operation are extracted from the alarm log of the information system. Then, alarm records triggered by topology nodes in the multi-dimensional topology within the same time window are extracted from the alarm database. Abnormal event records and alarm records are associated and matched according to occurrence time and node identifier. The proportion of successfully matched abnormal events to the total number of actual abnormal events is used as the alarm hit rate, and the proportion of successfully matched alarm records to the total number of alarm records is used as the alarm accuracy rate. The alarm hit rate and alarm accuracy rate are used as the matching results between actual abnormal events and topology node alarms. Before executing the risk handling operation, the health score of each topology node has been calculated based on various health score indicators, and risk nodes have been identified. After the risk handling operation is completed, the operating data of each topology node is re-collected, and the health score of each topology node is recalculated. The health score after handling is compared with the health score before handling. For each health score indicator, if the indicator after risk handling... If the health score indicator approaches the normal baseline value, it is considered valid; otherwise, it is considered invalid. The proportion of valid indicators to the total number of health score indicators is used as the effectiveness rate. The validity / invalidity determination results of each health score indicator and the effectiveness rate are used as the validity verification results of each health score indicator. The operation and maintenance logs and fault handling records of the root cause node before and after the risk disposal operation are obtained. The fault phenomenon, fault occurrence time, configuration change records before and after the fault, and status changes of upstream and downstream dependent nodes of the root cause node are comprehensively analyzed to determine the actual root cause of the fault of the root cause node. The located root cause node and its corresponding root cause type are compared with the actual root cause. If they match, the root cause location result is considered correct; otherwise, it is considered incorrect. The determination result is used as the comparison result between the root cause location result and the actual root cause information. The matching result, the validity verification result, and the comparison result are summarized to generate the risk disposal result.Secondly, based on the matching results, the updated value of the anomaly detection threshold of the information system is determined: The alarm hit rate and alarm accuracy are obtained from the matching results of actual abnormal events and topology node alarms. The alarm hit rate is compared with a preset hit rate threshold, and the alarm accuracy is compared with a preset accuracy threshold. If the alarm hit rate is lower than the preset hit rate threshold and the alarm accuracy is higher than the preset accuracy threshold, it indicates that the current anomaly detection threshold is set too high, causing the actual abnormal event to be undetected because it did not meet the alarm triggering conditions, i.e., a missed alarm is generated. In this case, the preset threshold is adjusted accordingly. The first adjustment step decreases the anomaly detection threshold; if the alarm hit rate is higher than the preset hit rate threshold and the alarm accuracy is lower than the preset accuracy threshold, it indicates that the current anomaly detection threshold is set too low, causing a large number of normal operating states to be misjudged as abnormal and generating false alarms. At this time, the anomaly detection threshold is increased according to the preset second adjustment step; if the alarm hit rate is not lower than the preset hit rate threshold and the alarm accuracy is not lower than the preset accuracy threshold, it indicates that the current anomaly detection threshold setting can balance missed alarms and false alarms and meet the system operation requirements. Therefore, the current anomaly detection threshold remains unchanged. The updated values ​​of the health score weights of the information system are determined based on the validity verification results: The validity / invalidity judgment results and effectiveness rates of each health score indicator are obtained from the validity verification results, and the effectiveness rate is compared with a preset effectiveness rate threshold. If the effectiveness rate is lower than the preset effectiveness rate threshold, it indicates that the current weight settings of each health score indicator are unreasonable, causing the health score results to fail to accurately reflect the actual operating status of each topology node. Therefore, the weights of each indicator are adjusted according to the validity / invalidity judgment results. For a health score indicator judged as valid, it indicates that the indicator's trend is consistent with the actual operating status of the node during this risk management process, effectively reflecting the node's health changes. The weight of this indicator is increased according to a preset weight increase step size. For a health score indicator judged as invalid, it indicates that the indicator failed to accurately reflect the actual operating status of the node during this risk management process. The weight of this indicator is decreased according to a preset weight decrease step size. After all health score indicator weights are adjusted, all adjusted weights are normalized so that the sum of the weights of each health score indicator equals 1. If the effectiveness rate is not lower than the preset effectiveness rate threshold, it indicates that the current weight settings of each health score indicator are reasonable. Therefore, the current weights of each health score indicator are kept unchanged.The update value of the root cause localization strategy of the information system is determined based on the comparison results: the correctness / incorrectness of the comparison results between the root cause localization results and the actual root cause information is determined; if the comparison results determine that the root cause localization results are incorrect, it indicates that the search capability of the current root cause localization strategy is insufficient to cover the root cause nodes of the actual fault, then the upper limit of the backtracking search layer is increased and the search node range is expanded, or when the current root cause localization strategy uses the root cause lookup tree for traversal search, the search threshold is increased by a preset step size, and the nodes on the root cause lookup tree are searched again; if the comparison results determine that the root cause localization results are correct, it indicates that the search depth and search range of the current root cause localization strategy can effectively locate the root cause nodes, then the parameter configuration of the current root cause localization strategy remains unchanged. Based on the updated values ​​of the anomaly detection threshold, the health score weights, and the root cause localization strategy, the operating rule parameters of the information system are reset: the updated anomaly detection threshold is written to the threshold configuration center of the information system, overriding the original anomaly detection threshold, and serves as the basis for subsequent anomaly detection judgment; the updated weights of each health score indicator are written to the health score calculation module of the information system, overriding the original weights of each health score indicator, and serve as the weight configuration for subsequent health score calculation; the updated search depth and search range parameters of the root cause localization strategy are written to the fault mode library and search rule configuration center of the information system, overriding the original root cause localization strategy parameters, and serve as the search configuration for subsequent root cause localization. In other words, the operating rule parameters are reset by replacing the corresponding current operating parameters in each configuration center or module with the updated values.

[0054] In this embodiment, Figure 2 The multi-dimensional stereoscopic topology diagram provided in this application is as follows: Figure 2As shown, the multi-dimensional topology comprises four layers from bottom to top: the physical resource layer, the cloud resource layer, the business service layer, and the front-end service layer. The physical resource layer includes hardware device nodes such as physical machines, disk arrays, and network switches; the cloud resource layer includes virtualized resource nodes such as virtual machines, container clusters, and cloud storage instances; the business service layer includes microservice modules such as business data analysis, performance monitoring and early warning, and resource status assessment, as well as monolithic application nodes such as traditional ERP / OA systems; and the front-end service layer includes access nodes for end users such as application gateways, API gateways, load balancers, and web applications. Each layer is connected by different types of edges: dependency edges carry call strength weights, representing call dependencies and upstream / downstream data dependencies between services; mapping edges represent the deployment correspondence between business services and cloud resources, as well as the attribution correspondence between cloud resources and physical resources; and network connection edges record the IP addresses, communication protocols, and link status information between each node. This multi-dimensional topology integrates the heterogeneous relationship data scattered across various operation and maintenance platforms in a hybrid architecture into a unified full-stack dependency graph, clearly presenting the complete call chain and resource attribution path from the user request entry point to the underlying physical infrastructure.

[0055] Therefore, this application, firstly, by acquiring multi-source heterogeneous data generated during the operation of the information system, integrates various relationships such as service call chains, resource deployment, host ownership, network associations, and data lineage to construct a multi-dimensional topology covering the front-end service layer, business service layer, cloud resource layer, and physical resource layer. This enables the heterogeneous data scattered across various operation and maintenance platforms under a hybrid architecture to be connected into a unified full-stack dependency graph, solving the problems of cross-layer dependencies not being intuitively displayed and fault propagation paths being unclear in existing technologies. It achieves a visual association between business applications and underlying infrastructure at the topology level. Secondly, by evaluating the health score of each topology node based on the operational indicators in the multi-source heterogeneous data and identifying risk nodes in an abnormal state, it can combine expert experience with real-time operation. By combining data, the system avoids generating a large number of false alarms during peak business periods using fixed threshold alarms, thus improving the accuracy of anomaly identification. Furthermore, by using risk nodes as the starting point and tracing back along the inter-layer dependencies of the multi-dimensional topology to locate the root cause node, the system can transform the root cause location of faults from relying on manual experience into an automated and quantifiable search process, significantly shortening the fault location time. Finally, by generating a fault propagation tree based on the root cause node, identifying the affected nodes and performing risk mitigation operations, and then resetting the operating rule parameters based on the mitigation results, the experience gained from each risk mitigation can be accumulated into continuous optimization of the system configuration, forming a complete risk prevention and control system. This enables efficient location and proactive prevention of cross-layer faults during the operation of hybrid architecture information systems.

[0056] In summary, the technical solution adopted in this application can perform cross-level quantitative analysis of the overall operational status of information systems, thereby improving the efficiency of fault root cause location.

[0057] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0058] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0059] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. A method for preventing and controlling operational risks in information systems using multidimensional stereotopology and fault propagation trees, characterized in that, Includes the following steps: Acquire multi-source heterogeneous data generated during the operation of the information system, and construct a multi-dimensional topology of the information system based on the multi-source heterogeneous data; Based on the operational metrics in the multi-source heterogeneous data, the health score of each topological node in the multi-dimensional topology is evaluated, and risk nodes are identified based on all health scores. Starting from the risk node, a cross-layer backtracking search is performed along the inter-layer dependencies of the multi-dimensional topology until the root cause node causing the current anomaly is located. A fault propagation tree is generated based on the root cause node and the multidimensional topology. The nodes in the fault propagation tree are fault nodes. Based on the propagation path of the fault nodes, the abnormal affected nodes are obtained. Risk handling operations are performed on the affected nodes to obtain the risk handling results. The operating rule parameters of the information system are reset based on the risk handling results.

2. The information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree as described in claim 1, characterized in that, Constructing the multi-dimensional topology of the information system based on the multi-source heterogeneous data specifically includes: The call intensity between each service node is determined based on the service call chain data in the multi-source heterogeneous data. Obtain the resource deployment relationships, host ownership relationships, network association relationships, and data lineage relationships in the information system; By performing topological fusion of all call intensities with the resource deployment relationship, the host affiliation relationship, the network association relationship, and the data lineage relationship, a multi-dimensional three-dimensional topology of the information system is obtained.

3. The information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree as described in claim 2, characterized in that, The multi-dimensional topology of the information system is obtained by topologically fusing all call intensities with the resource deployment relationships, host affiliation relationships, network association relationships, and data lineage relationships, specifically including: Determine the dependencies between each topology node based on the call intensity and the data lineage; The mapping relationship between each topology node is determined based on the resource deployment relationship and the host affiliation relationship; The network connection relationships between each topology node are determined based on the network association relationships. The dependencies, mappings, and network connections are associated to generate a multi-dimensional topology of the information system. The dependencies and mappings form inter-layer dependencies, and the dependencies, mappings, and network connections form topological dependencies.

4. The information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree as described in claim 1, characterized in that, Based on the operational metrics in the multi-source heterogeneous data, the health score of each topological node in the multidimensional topology is evaluated, and risk nodes in an abnormal state are identified based on all health scores, specifically including: The operational indicators in the multi-source heterogeneous data are normalized to obtain standard operational indicators; Determine the fusion weight of each evaluation indicator in the standard operation indicators, with each evaluation indicator corresponding to a fusion weight; The health score of the topological nodes in the multidimensional topology is determined based on the standard operating indicators and the fusion weights. The operational risk of all topology nodes is classified using the health score to obtain the operational risk level; Risk nodes are selected based on the stated operational risk level.

5. The information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree as described in claim 4, characterized in that, Determining the fusion weights of each evaluation indicator in the standard operating indicators specifically includes: Determine the subjective and objective weights of each evaluation indicator in the standard operating indicators; The subjective weights and objective weights of each evaluation indicator are combined to obtain the combined weights of each evaluation indicator.

6. The information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree as described in claim 1, characterized in that, Starting from the risk node, a cross-layer backtracking search is performed along the inter-layer dependencies of the multi-dimensional topology until the root cause node leading to the current anomaly is located. Specifically, this includes: Perform dual-mode anomaly detection on the risk nodes to obtain the anomaly category of the risk nodes; Based on the anomaly category, starting from the risk node, the process traces downwards layer by layer along the inter-layer dependencies of the multi-dimensional topology, checking the operational status of each topology node at each level for anomalies, until the root cause node causing the current anomaly is located. The inter-layer dependencies include the dependency relationship and the mapping relationship.

7. The information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree as described in claim 1, characterized in that, The generation of a fault propagation tree based on the root cause node and the multidimensional topology specifically includes: Starting from the root cause node, the topological dependencies of the multidimensional topology are expanded outward layer by layer to determine the upstream and downstream relationships and propagation paths between each topological node, forming a set of nodes and edges of the fault propagation tree. The topological dependencies include the dependency relationship, the mapping relationship, and the network connection relationship. The failure propagation probability and service impact range of each topology node are determined step by step. The failure propagation probability is used as the propagation probability label of each edge in the edge set, and the service impact range is used as the state label of each node in the node set. A fault propagation tree is generated based on the set of nodes, the set of edges, the propagation probability label, and the state label.

8. The information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree as described in claim 1, characterized in that, Based on the propagation path of the faulty node, the affected nodes are identified, and risk mitigation operations are performed on these affected nodes. The specific risk mitigation results include: Multiple affected nodes are identified based on the propagation path of the faulty node in the fault propagation tree; Based on the depth and scope of each affected node in the propagation path, determine the risk level and handling priority of each affected node; Based on the risk level and handling priority of each affected node, a corresponding preset handling strategy is matched. The preset handling strategy includes at least one of fault isolation, traffic switching, service degradation and node restart. The risk management operation is performed on each affected node using the preset management strategy to obtain the risk management result.

9. The information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree as described in claim 8, characterized in that, Based on the depth and scope of each affected node in the propagation path, the risk level and handling priority of each affected node are determined, specifically including: Obtain the propagation intensity of each edge along the propagation path from the root cause node to the affected node in the fault propagation tree. Based on the level depth of each affected node and the propagation intensity of each edge along the propagation path, obtain the fault propagation probability of each affected node. Obtain the current operating data, normal baseline value, and the proportion of service request volume carried by each affected node; assess the extent of service performance degradation based on the difference between the current operating data and the normal baseline value; and assess the range of affected traffic based on the proportion of service request volume. Based on the failure propagation probability, the extent of service performance degradation, and the range of affected traffic for each affected node, the risk level and handling priority of each affected node are determined.

10. The information system operation risk prevention and control method based on multi-dimensional topology and fault propagation tree as described in claim 1, characterized in that, Resetting the operating rule parameters of the information system based on the risk management results specifically includes: The risk handling results are obtained, including the matching results of actual abnormal events and topology node alarms, the validity verification results of each health score indicator, and the comparison results of root cause location results and actual root cause information. Based on the matching results, the updated value of the anomaly detection threshold of the information system is determined; based on the validity verification results, the updated value of the health score weight of the information system is determined; and based on the comparison results, the updated value of the root cause localization strategy of the information system is determined. The operating rule parameters of the information system are reset based on the updated values ​​of the anomaly detection threshold, the health score weight, and the root cause localization strategy.

Images