A machine room exception operation and maintenance processing method, device and electronic equipment

Abstract

The application provides a machine room exception operation and maintenance processing method and device and electronic equipment, and belongs to the technical field of machine room operation and maintenance. The application provides a machine room exception operation and maintenance processing method, which comprises the following steps: acquiring machine room operation data and computing power service state data; determining an exception severity and an exception resource object; performing topology processing to generate a resource object node set and a computing power service node set, and then constructing a computing power service dependency graph with a dependency weight; determining all computing power service nodes corresponding to the nodes of the exception resource object from the computing power service dependency graph, and then determining a computing power service service influence score for each computing power service node based on the exception severity, the dependency weight, a service level agreement (SLA) weight of the computing power service, and a computing power service exposure coefficient; and determining an exception operation and maintenance response action based on the service attribute and the service influence score of the computing power service. In summary, the computing power service affected by the machine room exception can be accurately identified, and various scenarios can be adapted.

Description

Technical Field

[0001] This application relates to the field of data center operation and maintenance technology, and in particular to a method, device and electronic equipment for handling data center anomalies. Background Technology

[0002] In computing data centers and intelligent computing hubs, heterogeneous resources such as GPU nodes, distributed storage, networks, power supply and distribution, and heat dissipation domains jointly support various computing services, including batch training, online inference, and streaming data processing. When hardware, storage, network, or power supply and heat dissipation anomalies occur in the data center, traditional operation and maintenance solutions mostly generate operation and maintenance work orders based on alarms from a single device, focusing only on the faulty hardware itself. It is difficult to identify which upper-layer computing services the fault will affect, nor can it quantify the actual business impact on different computing services. This can easily lead to problems such as alarm overload or decreased alarm accuracy, disordered handling priorities, and handling actions that are out of touch with the actual on-site environment, making it difficult to adapt to various application scenarios. Summary of the Invention

[0003] This application provides a method, device, and electronic equipment for handling data center anomalies, which can, to a certain extent, solve the problems of alarm overload or alarm accuracy degradation, disordered handling priorities, and handling actions that are out of touch with the actual on-site environment and difficult to adapt to various application scenarios.

[0004] According to one aspect of this application, a method for handling data center anomalies is provided. The method includes: acquiring data center operation data and computing power service status data; the data center operation data is used to indicate the real-time operation status of multiple resource objects within the data center; the computing power service status data is used to indicate the service attributes and operating conditions of computing power services running on each resource object; based on the data center operation data, determining the severity of the anomaly, and then determining the abnormal resource object; performing topology processing on the data center operation data and computing power service status data to generate a resource object node set and a computing power service node set, and then constructing a computing power service dependency graph with dependency weights; the dependency weights are determined based on declared dependency scores, operational association dependency scores, resource sharing dependency scores, and historical propagation dependency scores; one resource object node corresponds to multiple computing power service nodes; determining all computing power service nodes corresponding to the nodes of the abnormal resource object from the computing power service dependency graph, and then, for each computing power service node, determining the service impact score of the computing power service based on the severity of the anomaly, dependency weights, the service level agreement (SLA) weight of the computing power service, and the computing power service exposure coefficient; and determining the anomaly operation and maintenance response action based on the service attributes and service impact score of the computing power service.

[0005] Furthermore, according to one aspect of the method of this application, the data center operation data includes: monitoring data of data center hardware indicators, network indicators, storage indicators, power supply and distribution indicators, and heat dissipation domains; the service attributes of computing power services include: service type, SLA weight, and computing power service exposure coefficient; the operating conditions of computing power services include: service operation status, service deployment location, and current load status; acquiring data center operation data and computing power service status data includes: acquiring data center operation data from at least one of the data center hardware management unit and storage monitoring components; and acquiring computing power service status data from at least one of the network controller and computing power cluster scheduling interface.

[0006] Furthermore, according to one aspect of the method of this application, based on data center operation data, the severity of anomalies is determined, and then the abnormal resource objects are determined, including: for any resource object in the data center operation data, obtaining the indicator threshold and confidence level of the resource object; calculating the normalized severity of anomalies based on the data center operation data, indicator threshold and confidence level of the resource object; when the severity of anomalies is greater than or equal to a preset anomaly triggering threshold, determining the resource object corresponding to the severity of anomalies as an abnormal resource object.

[0007] Furthermore, according to one aspect of the method of this application, topology processing is performed on data center operation data and computing power service status data to generate a resource object node set and a computing power service node set, thereby constructing a computing power service dependency graph with dependency weights. This includes: performing topology preprocessing on the data center operation data and computing power service status data to obtain the resource object node set and the computing power service node set; the topology preprocessing includes: time alignment processing, identifier normalization processing, and topology deduplication processing; based on the resource object node set and the computing power service node set, generating candidate dependency edges between the resource object nodes and the computing power service node set, and calculating the declared dependency score, operational association dependency score, resource sharing dependency score, and historical propagation dependency score corresponding to each candidate dependency edge; weighting and fusing the declared dependency score, operational association dependency score, resource sharing dependency score, and historical propagation dependency score to obtain the dependency weight of the candidate dependency edge; and integrating all candidate dependency edges and their corresponding dependency weights to obtain the computing power service dependency graph.

[0008] Furthermore, according to one aspect of the method of this application, when the topology preprocessing is time alignment processing, a unified timestamp benchmark is applied to the data center operation data and computing power service status data, and time alignment processing is performed according to the data source frequency; when the topology preprocessing is identifier normalization processing, a resource identifier mapping table corresponding to the data center operation data and computing power service status data is maintained, and a globally unique identifier is generated; when the topology preprocessing is topology deduplication processing, data center operation data and computing power service status data with the same records are deleted.

[0009] Furthermore, according to one aspect of the method of this application, all computing power service nodes corresponding to the nodes of the abnormal resource objects are determined from the computing power service dependency graph. Then, for each computing power service node, the service impact score of the computing power service is determined based on the severity of the anomaly, the dependency weight, the service level agreement (SLA) weight of the computing power service, and the computing power service exposure coefficient. This includes: traversing the computing power service dependency graph to retrieve all computing power service nodes that have dependency edges with the nodes of the abnormal resource objects; for each computing power service node, obtaining the SLA weight and computing power service exposure coefficient from the computing power service status data, and multiplying the severity of the anomaly, the dependency weight, the SLA weight, and the computing power service exposure coefficient to obtain the service impact score of the corresponding computing power service node.

[0010] Furthermore, according to one aspect of the method of this application, abnormal operation and maintenance response actions are determined based on the business attributes and service impact scores of computing power services, including: obtaining the business type from the business attributes, and generating a set of candidate abnormal operation and maintenance response actions based on the business type and service impact scores; performing multi-dimensional feasibility verification on each candidate abnormal operation and maintenance response action to obtain an action score; the multi-dimensional feasibility verification includes at least: capacity verification, data location verification, task portability verification, checkpoint status verification, and execution risk verification; when the action score is greater than or equal to a preset action threshold, the candidate abnormal operation and maintenance response action corresponding to the action score is determined as an abnormal operation and maintenance response action.

[0011] Furthermore, according to one aspect of the method of this application, the method further includes: obtaining feedback recovery data of executing abnormal operation and maintenance response actions; and updating the abnormal operation and maintenance response actions based on the feedback recovery data.

[0012] According to another aspect of this application, a data center anomaly operation and maintenance processing device is provided. The device includes: an acquisition unit, used to acquire data center operation data and computing power service status data; the data center operation data is used to indicate the real-time operation status of multiple resource objects within the data center; the computing power service status data is used to indicate the service attributes and operating conditions of computing power services running on each resource object; a first determination unit, used to determine the severity of the anomaly based on the data center operation data, and then determine the abnormal resource object; and a construction unit, used to perform topology processing on the data center operation data and computing power service status data to generate a resource object node set and a computing power service node set, and then construct a topology processing device with... The system comprises: a computational power service dependency graph with weights; dependency weights determined based on declared dependency scores, runtime-related dependency scores, resource-sharing dependency scores, and historical propagation dependency scores; one resource object node corresponding to multiple computational power service nodes; a second determining unit, used to determine all computational power service nodes corresponding to the nodes of abnormal resource objects from the computational power service dependency graph, and then, for each computational power service node, determining the service impact score of the computational power service based on the severity of the anomaly, dependency weights, the service level agreement (SLA) weight of the computational power service, and the computational power service exposure coefficient; and an operation and maintenance unit, used to determine the abnormal operation and maintenance response actions based on the service attributes and service impact scores of the computational power service.

[0013] According to another aspect of this application, an electronic device is provided, comprising: a memory for storing computer-readable instructions; and a processor for executing the computer-readable instructions, causing the electronic device to perform the method as described in any embodiment of one aspect.

[0014] This application provides a method, apparatus, and electronic equipment for handling data center anomalies. This application discloses a method for handling data center anomalies, comprising: acquiring data center operation data and computing power service status data; the data center operation data indicating the real-time operation status of multiple resource objects within the data center; the computing power service status data indicating the service attributes and operating conditions of computing power services running on each resource object; determining the severity of the anomaly based on the data center operation data, and thus identifying the abnormal resource object; performing topology processing on the data center operation data and computing power service status data to generate a resource object node set and a computing power service node set, and then constructing a computing power service dependency graph with dependency weights; the dependency weights are determined based on declared dependency scores, operational dependency scores, resource sharing dependency scores, and historical propagation dependency scores; one resource object node corresponds to multiple computing power service nodes; determining all computing power service nodes corresponding to the abnormal resource object node from the computing power service dependency graph, and then, for each computing power service node, determining the service impact score of the computing power service based on the severity of the anomaly, dependency weights, the service level agreement (SLA) weight of the computing power service, and the computing power service exposure coefficient; and determining the anomaly operation and maintenance response action based on the service attributes and service impact score of the computing power service. In contrast to existing solutions that rely solely on device-level alarms for maintenance and cannot quantify the actual impact of anomalies on upper-layer computing services, this application first identifies the abnormal resource objects, then constructs a computing service dependency graph with dependency weights through topology processing, and uses this dependency graph to locate all computing service nodes affected by the abnormal resources. Furthermore, it calculates a service impact score by combining the severity of the anomaly, dependency weights, SLA weights, and computing service exposure coefficients, and then determines the corresponding anomaly maintenance response based on the computing service's own business attributes and the quantified service impact score. Through this layered and progressive processing, the anomalies of lower-level resources are propagated upwards to the upper-layer computing service dimension, solving the technical problem of traditional solutions that can only see hardware failures and cannot assess the extent of service damage. In summary, the technical solution provided by this application can accurately identify computing services affected by data center anomalies, avoiding problems such as alarm overload or decreased alarm accuracy, disordered handling priorities, and handling actions detached from the actual on-site environment, and can adapt to various application scenarios.

[0015] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0016] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 A flowchart illustrating a method for handling data center anomalies provided in this application embodiment; Figure 2 A structural block diagram of a data center anomaly handling device provided in this application embodiment; Figure 3 This is a hardware block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0019] Currently, in computing data centers and intelligent computing hubs, heterogeneous resources such as GPU nodes, distributed storage, networks, power supply and distribution, and heat dissipation domains jointly support various computing services, including batch training, online inference, and streaming data processing. When hardware, storage, network, or power supply and heat dissipation anomalies occur in the data center, traditional operation and maintenance solutions mostly generate operation and maintenance work orders directly based on alarms from a single device. They only focus on the faulty hardware itself, making it difficult to identify which upper-layer computing services the fault will affect, and also unable to quantify the actual business impact on different computing services. This can easily lead to problems such as alarm overload or decreased alarm accuracy, disordered handling priorities, and handling actions that are out of touch with the actual on-site environment, making it difficult to adapt to various application scenarios.

[0020] Therefore, to address the aforementioned issues, this application provides a method for handling data center anomalies. Compared to existing solutions that rely solely on device-level alarms for maintenance and cannot quantify the actual impact of anomalies on upper-layer computing services, this application first identifies the resource objects experiencing anomalies. Then, it constructs a computing service dependency graph with dependency weights through topology processing, using this graph to locate all computing service nodes affected by the anomaly. Furthermore, it calculates a service impact score by combining anomaly severity, dependency weights, SLA weights, and computing service exposure coefficients. Finally, it determines the corresponding anomaly maintenance response based on the computing service's own business attributes and the quantified service impact score. Through this layered and progressive processing, the underlying resource anomalies are propagated upwards to the upper-layer computing service dimension, solving the technical problem of traditional solutions that can only see hardware failures but cannot assess the extent of service damage. In summary, the technical solution provided by this application can accurately identify the computing services affected by data center anomalies, avoiding problems such as alarm overload or decreased alarm accuracy, disordered handling priorities, and handling actions detached from the actual on-site environment. It can adapt to various application scenarios.

[0021] This application provides a method for handling data center anomalies during operation and maintenance. Please refer to it. Figure 1 , Figure 1 This is a flowchart illustrating a method for handling data center anomalies, provided in an embodiment of this application. Figure 1 As shown, the method includes: In step S101, data center operation data and computing power service status data are acquired; data center operation data is used to indicate the real-time operation status of multiple resource objects in the data center; computing power service status data is used to indicate the service attributes and operating conditions of computing power services running on each resource object. In step S102, based on the data center operation data, the severity of the anomaly is determined, and then the abnormal resource object is identified; In step S103, topology processing is performed on the data center operation data and computing power service status data to generate a resource object node set and a computing power service node set, thereby constructing a computing power service dependency graph with dependency weights; the dependency weights are determined based on the declared dependency score, the operation association dependency score, the resource sharing dependency score, and the historical propagation dependency score; one resource object node corresponds to multiple computing power service nodes; In step S104, all computing power service nodes corresponding to the nodes of the abnormal resource objects are determined from the computing power service dependency graph. Then, for each computing power service node, the service impact score of the computing power service is determined based on the severity of the abnormality, the dependency weight, the service level agreement (SLA) weight of the computing power service, and the computing power service exposure coefficient. In step S105, based on the business attributes and service impact scores of the computing power service, the abnormal operation and maintenance response action is determined.

[0022] In this application, the resource object is a physical or logical object within the computer room that can be monitored, relied upon, and disposed of, including at least one of computing hardware, storage, network, power supply and distribution domain, and heat dissipation domain.

[0023] In this application, the data center operation data refers to the collected monitoring data that reflects the real-time operation of each resource object.

[0024] In this application, the computing power service refers to a business task running on a data center resource object, including at least one of training tasks, online inference services, and streaming data processing tasks.

[0025] In this application, the business attributes of the computing power service are inherent configuration information of the computing power service itself, including service type, SLA weight, and computing power service exposure coefficient, which will be elaborated on later. Here, SLA stands for Service Level Agreement. In the computing power data center scenario of this application, it can be understood as a pre-agreed service assurance indicator for the computing power service, used to define the acceptable availability, response performance, fault tolerance, and other requirements of the service. The SLA weight is a value quantified based on the importance of the service stipulated in the SLA. The higher the SLA level, the greater the corresponding SLA weight, and the higher the contribution of the service to the impact score during anomaly impact assessment.

[0026] In this application, the operating status of the computing power service refers to the dynamic status information generated during the actual operation of the computing power service, including the service operation status, service deployment location, and current load status, which will be described in detail later and will not be repeated here.

[0027] In this application, the resource object node is a graph structure vertex obtained after abstracting the resource object.

[0028] In this application, the computing power service node is a graph structure vertex obtained by abstracting the computing power service running on the resource object.

[0029] In this application, the resource object node set is a collection composed of all resource object nodes; the computing power service node set is a collection composed of all computing power service nodes.

[0030] In this application, the topology processing is a process of preprocessing data from the data center operation and computing power service status, and abstracting and generating graph nodes to provide basic data for constructing a computing power service dependency graph.

[0031] In this application, the computing power service dependency graph is a graph structure used to represent the mutual relationship between resource object nodes and computing power service nodes. The graph includes resource object nodes, computing power service nodes, and dependency edges connecting nodes.

[0032] In this application, the declared dependency score is obtained based on at least one explicit declared information such as the configuration, mounting relationship, and network policy of the computing power service, and is used to characterize the degree of dependency of the computing power service on the resource object at the configuration level. In this application, the runtime association dependency score is obtained based on at least one runtime monitoring data such as the runtime resource consumption, IO read / write, and service traffic of the computing power service, and is used to characterize the degree of dependency of the computing power service on the resource object during actual operation. In this application, the resource sharing dependency score is obtained based on the load ratio of multiple computing power services deployed on the same resource object, and is used to characterize the degree of indirect dependency caused by multiple services sharing the same resource. In this application, the historical propagation dependency score is obtained based on the statistical correlation between resource object anomalies and computing power service anomalies in historical failure events, and is used to characterize the degree of implicit dependency manifested by historical failures.

[0033] In this application, the dependency weight is calculated by fusing the declared dependency score, the runtime association dependency score, the resource sharing dependency score, and the historical propagation dependency score, and is used to characterize the degree of dependency between the resource object and the computing power service on a single dependency edge.

[0034] In this application, the severity of the anomaly is calculated based on data center operation data and is used to characterize the severity level of the anomaly occurring in the corresponding resource object.

[0035] In this application, the service impact score is calculated by comprehensively considering the severity of the anomaly, dependency weight, SLA weight, and computing power service exposure coefficient, and is used to quantify the magnitude of the business impact caused by resource object anomalies on computing power services.

[0036] In this application, the abnormal operation and maintenance response action is an operation and maintenance handling action generated in response to abnormal data center resources and used to handle computing power services and resource object failures.

[0037] In this embodiment, when executing the complete process S101-S105, the following steps are taken: First, data center operation data reflecting the status of resource objects and computing power service status data describing the upper-layer computing power services are acquired. Then, resource object anomalies are identified based on the data center operation data, the severity of the anomalies is calculated, and the resource objects that have anomalies are located. Next, topology processing is performed on the two sets of data to abstractly generate resource object node sets and computing power service node sets. Dependency weights are calculated by combining four types of dependency scores, thereby constructing a computing power service dependency graph to achieve unified modeling of the dependency relationships between heterogeneous resources and computing power services. Then, all computing power service nodes associated with the abnormal resource object nodes are retrieved from the computing power service dependency graph. For each computing power service node, the service impact score is quantitatively calculated by combining the severity of the resource anomaly, the dependency weight between the two, the service SLA weight, and the service exposure coefficient. Finally, based on the inherent business attributes of the computing power service itself and the service impact score representing the extent of business damage, an abnormal operation and maintenance response action adapted to the current business status is output. This solution establishes an assessment link from underlying hardware anomalies to upper-layer computing power services. Instead of viewing device alarms in isolation, it quantifies and assesses the actual impact of anomalies on computing power services, using this as a basis to determine operational and maintenance actions.

[0038] The following will elaborate on each of the steps mentioned above, including: First, step S101 is explained in detail, including: Data center operation data includes: monitoring data of data center hardware indicators, network indicators, storage indicators, power supply and distribution indicators, and heat dissipation areas; business attributes of computing power services include: service type, SLA weight, and computing power service exposure coefficient; operating conditions of computing power services include: service operation status, service deployment location, and current load status; acquiring data center operation data and computing power service status data includes: Data center operation data is obtained from at least one of the data center hardware management unit and storage monitoring component; Obtain computing power service status data from at least one of the network controller and the computing power cluster scheduling interface.

[0039] In this application, the data center hardware management unit is used to collect operational monitoring information of GPU nodes and hardware boards, and can be, for example, BMC or GPU-exporter.

[0040] In this application, the storage monitoring component is used to collect storage dimension monitoring information such as latency, throughput, and replica status of distributed storage volumes.

[0041] In this application, the network controller is used to collect network layer operation data such as network link packet loss, latency, and bandwidth usage, and can be exemplarily an SDN controller.

[0042] In this application, the computing power cluster scheduling interface is used to read the configuration information and real-time running information of computing power tasks within the cluster, and can be exemplarily a Kubernetes API.

[0043] In this application, the data center hardware metrics include hardware-level monitoring metrics such as hardware temperature, video memory usage, and node health status.

[0044] In this application, the network metrics include network operation metrics such as network packet loss rate, link latency, and bandwidth utilization.

[0045] In this application, the storage metrics include storage access latency, I / O throughput, and other storage operation metrics.

[0046] In this application, the power supply and distribution indicators include power supply and distribution domain monitoring indicators such as UPS load and power supply voltage.

[0047] In this application, the monitoring data of the heat dissipation area includes heat dissipation-related monitoring data such as cabinet temperature and humidity, and air conditioning operation status.

[0048] In this application, the business type is used to distinguish different computing power tasks, including batch training tasks, online inference services, and streaming data processing tasks.

[0049] In this application, the SLA weight is configured according to the service level agreement of the computing power service and is used to characterize the service value of different computing power services.

[0050] In this application, the computing power service exposure coefficient is calculated from the percentage of affected instances, real-time service load, and available redundant resources, and characterizes the degree to which computing power services are affected by resource anomalies.

[0051] In this application, the service operation status is used to record the current operation status identifiers of computing power service instances, such as running, paused, or abnormal.

[0052] In this application, the service deployment location is used to record the resource object node information corresponding to the actual operation of the computing power service instance.

[0053] In this application, the current load status is used to record real-time load information such as computing power usage and request traffic of computing power service instances.

[0054] In this embodiment of the application, when executing step S101, the data center operation data corresponding to the data center hardware, storage, power supply and distribution, and heat dissipation domains are collected through the data center hardware management unit and storage monitoring component; network dimension monitoring information is obtained through the network controller, and then the configuration and operation information of the computing power service is read by the computing power cluster scheduling interface to obtain computing power service status data; the collected data center operation data and computing power service status data provide the original data source for subsequent identification of resource anomalies, construction of computing power service dependency graph, and computing service impact analysis.

[0055] Secondly, step S102 is described in detail, including: Based on data center operational data, the severity of the anomaly is determined, and then the abnormal resource objects are identified, including: For any resource object in the data center operation data, obtain the resource object's indicator observation value, corresponding indicator threshold, and evidence confidence level; Based on the data center operation data, indicator thresholds, and evidence confidence levels of resource objects, the normalized severity of anomalies is calculated. When the severity of an anomaly is greater than or equal to the preset anomaly trigger threshold, the resource object corresponding to the severity of the anomaly is determined to be an abnormal resource object.

[0056] In this application, the observed index value is the real-time monitoring value of the resource object collected from the data center operation data.

[0057] In this application, the indicator threshold is a pre-configured critical value for the normal operation of the resource object, which can be determined from resource type configuration, vendor recommended value, business baseline or historical quantile confirmed manually, and there is no specific limitation.

[0058] In this application, the confidence level of the evidence is determined comprehensively based on the reliability of the data source, the completeness of the data collection, the freshness of the data, and the consistency of multi-source evidence, and is used to characterize the credibility of the monitoring data.

[0059] In this application, the preset anomaly triggering threshold is used to determine whether a resource object meets the anomaly judgment condition; if the anomaly is triggered by an indicator value being lower than the threshold, the indicator observation value and the indicator threshold are swapped in the calculation.

[0060] In this embodiment of the application, when executing step S102, the indicator observation value, preset indicator threshold, and evidence confidence level of the corresponding resource object are read; when the anomaly belongs to the scenario of the indicator exceeding the downward limit, the observation value and threshold are swapped; the anomaly severity in the 0-1 interval is obtained by substituting into the calculation formula; the anomaly severity is compared with the preset anomaly trigger threshold, and if the threshold is reached, the resource object is determined to be an abnormal resource object and anomaly evidence is generated, thus completing the location of the underlying fault resource, and outputting the anomaly severity and anomaly evidence for subsequent computing power service impact assessment.

[0061] For example, this application also provides a formula for calculating the severity of an anomaly:

[0062] In the formula, This represents the severity of the anomaly in resource object r. For the observed values ​​of the indicators, For the indicator threshold, The confidence level of the evidence; the normalized range of the calculated anomaly severity is 0-1.

[0063] Next, step S103 will be described in detail, including: Topology processing is performed on data center operation data and computing power service status data to generate resource object node sets and computing power service node sets, thereby constructing a computing power service dependency graph with dependency weights, including: Topology preprocessing is performed on the data center operation data and computing power service status data to obtain the resource object node set and the computing power service node set; topology preprocessing includes: time alignment processing, identifier normalization processing and topology deduplication processing. Based on the resource object node set and the computing power service node set, candidate dependency edges between resource object nodes and computing power service nodes are generated, and the declared dependency score, runtime association dependency score, resource sharing dependency score and historical propagation dependency score corresponding to the candidate dependency edges are calculated respectively. The dependency weights of candidate dependency edges are obtained by weighting and fusing the declared dependency score, the runtime dependency score, the resource sharing dependency score, and the historical propagation dependency score. By integrating all candidate dependency edges and their corresponding dependency weights, a computing power service dependency graph is obtained.

[0064] In this application, topology preprocessing is used to eliminate inconsistencies in time base, identification system, and topology records among multi-source data. Among them, time alignment processing is used to unify the time base of different data sources and perform window aggregation and missing value compensation on the data. Identifier normalization processing is used to mask the differences in the original identifiers of various monitoring components and assign globally unique identifiers to resource objects and computing power services. Topology deduplication processing is used to remove duplicate, expired, and redundant candidate dependency edges and purify the original graph data, which will be described in detail later.

[0065] In this application, the candidate dependency edges include computation dependency edges, storage dependency edges, network dependency edges, and domain association edges; computation dependency edges are generated by the deployment relationship between computing power services and running resource objects, storage dependency edges are generated by the storage mounting relationship of computing power services, network dependency edges are generated by the network paths accessed by computing power services, and domain association edges are generated by resource objects belonging to the same power supply and distribution domain or heat dissipation domain.

[0066] In this application, during weighted fusion, a preset non-negative weight coefficient is used to perform weighted calculations on the declared dependency score, runtime dependency score, resource sharing dependency score, and historical propagation dependency score. After interval truncation, the calculation results are limited to the 0-1 interval to obtain the final dependency weight of the candidate dependency edge. An edge retention threshold can also be set to discard candidate dependency edges with dependency weights lower than the edge retention threshold, so that they do not participate in the subsequent propagation of abnormal effects. When the resource object or computing power business status changes, incremental updates are only performed on the changed nodes and associated candidate dependency edges.

[0067] In this application, the computing power service dependency graph is an attribute graph composed of a set of resource object nodes, a set of computing power service nodes, and weighted candidate dependency edges; the computing power service dependency graph stores a graph version snapshot, which can read back the historical topology version according to the time of the anomaly; the computing power service dependency graph is used to express the strength of the dependency relationship between heterogeneous physical / logical resources and upper-layer computing power services, and provides a graph computing carrier for tracing the affected computing power service nodes from the abnormal resource objects.

[0068] In this embodiment, when executing step S103, the collected data center operation data and computing power service status data are first subjected to topology preprocessing, including time alignment, identifier normalization, and topology deduplication, to abstract the resource object node set and computing power service node set; then, candidate dependency edges are generated based on mounting relationships, topology affiliation, etc., and four types of dependency components are calculated respectively; the dependency weights are substituted into the dependency weight calculation formula, and the dependency weights of each candidate dependency edge are obtained through weighted fusion; candidate dependency edges with weights lower than the edge retention threshold are filtered out, and incremental updates are performed on the graph and version snapshots are saved to assemble a complete computing power service dependency graph; the unified modeling of the dependency relationship between heterogeneous resources and upper-layer computing power services is completed, providing a graph structure foundation for tracing the impact of abnormal services.

[0069] Furthermore, when topology preprocessing is time-aligned, a unified timestamp benchmark is used for data center operation data and computing power service status data, and time alignment is performed according to the data source frequency. Specifically, a unified timestamp is used as the benchmark to verify the clock deviation of the data reported by each data source. For high-frequency data sources, aggregation operations are performed within a set time window. Numerical indicators take the maximum or average value within the window, and status indicators take the latest status record within the window. For low-frequency data sources, they are directly assigned to the corresponding time window. When a data source in a window is missing data, forward filling is used to reuse existing valid records, while reducing the confidence of that data. The center point of the time window when the anomaly is first triggered is used as the anchor point to backtrack and extract the data center operation data and computing power service status data that have been aligned within the time window before the anchor point, as input for constructing the dependency graph.

[0070] When topology preprocessing involves identifier normalization, a resource identifier mapping table is maintained corresponding to the data center operation data and computing power service status data, generating globally unique identifiers. Specifically, the resource identifier mapping table records the globally unique identifiers allocated by the system, the original identifier dictionaries of each data source, the matching attribute set, and the mapping confidence. Multi-dimensional identifier mapping is completed sequentially through strong hardware identifier matching, management network IP matching, physical location code matching, naming rule matching, and runtime association matching. If a conflict occurs where the same original identifier matches multiple different globally unique identifiers, the conflicting entries are sent to a manual confirmation queue, and the mapping table is updated after manual confirmation. The computing power service generates a corresponding globally unique identifier based on the cluster namespace, combined with business tags and service registry entries. All subsequent association queries are completed based on this globally unique identifier.

[0071] When topology preprocessing is topology deduplication, duplicate records of data center operation data and computing power service status data are deleted. Specifically, for candidate dependency edges obtained from parsing data center operation data and computing power service status data, duplicate records are identified using computing power service identifiers, resource object identifiers, dependency types, and composite primary keys. Only candidate dependency edges with the latest timestamp and the highest data source confidence under the same composite primary key are retained. Candidate dependency edges that physically point to the same object but differ only in description type are semantically merged, and the higher value after merging is taken as the initial weight of the edge. Candidate dependency edges that have not received confirmation updates from the data source for multiple consecutive time windows are downgraded in weight. If they are still not confirmed after a preset lifespan, they are removed. At the same time, topology loop detection is performed to remove redundant domain association edges with low weights within the loop. After topology deduplication is completed, only the candidate dependency edges that have changed are written into the graph, and a graph version snapshot is generated for topology backtracking in subsequent abnormal scenarios.

[0072] For example, this application also provides a formula for calculating the dependent weights:

[0073] In the formula, This is a range cutoff function; the first input item inside the parentheses... This function outputs the weighted sum to be truncated, with the last two parameters 0 and 1 representing the lower and upper truncation limits, respectively. The function operates as follows: if the weighted sum to be truncated is less than 0, the output is 0; if the weighted sum to be truncated is greater than 1, the output is 1; if the weighted sum to be truncated is within the range 0-1, the output is the result. The dependency weights of candidate dependency edges. To declare dependencies, To run related dependencies, For resource sharing and dependency, The historical propagation dependence score is α, β, γ, and δ, which are preset non-negative weight coefficients, and α+β+γ+δ=1.

[0074] Next, step S104 will be explained in detail, including: From the computing power service dependency graph, identify all computing power service nodes corresponding to the nodes of the abnormal resource objects. Then, for each computing power service node, determine the service impact score of the computing power service based on the severity of the anomaly, dependency weight, service level agreement (SLA) weight of the computing power service, and computing power service exposure coefficient. This includes: Traverse the computing power service dependency graph and retrieve all computing power service nodes that have dependency edges with the nodes of the abnormal resource objects; For each computing power service node, the SLA weight and computing power service exposure coefficient are obtained from the computing power service status data. The anomaly severity, dependency weight, SLA weight and computing power service exposure coefficient are multiplied to obtain the service impact score of the corresponding computing power service node. It may also include: if the same computing power service node is affected by multiple abnormal resource objects, the service impact scores from multiple abnormal sources will be merged to prevent the merged score from exceeding the upper limit.

[0075] In this embodiment of the application, when executing step S104, the computing power service dependency graph is traversed to retrieve all computing power service nodes that have dependency edges with the abnormal resource object nodes; for each computing power service node, the single resource abnormality service impact score calculation formula is applied; when the service is affected by multiple abnormal resources at the same time, the final service impact score of the computing power service node is obtained by applying the combined calculation formula; the obtained service impact score quantitatively expresses the degree of damage suffered by each computing power service from this resource abnormality; when the service impact score reaches the preset impact threshold, the computing power service is marked as an affected service, providing a quantitative basis for determining the operation and maintenance response action.

[0076] For example, this application also provides a formula for calculating the service impact score under single resource anomaly:

[0077] In the formula, This represents the impact of the abnormal resource object r on the computing power service s. Indicates the severity of the abnormality. Represents the dependency weight. Represents SLA weights, This represents the exposure coefficient of computing power services.

[0078] For example, this application also provides a formula for calculating the combined service impact score under the combined effect of multiple abnormal resources:

[0079] In the formula, The impact on services after the final merger of computing power services (s) is analyzed. The impact of resource object r on computing power service s is analyzed. This indicates that a cumulative multiplication operation is performed on all abnormal resource objects r.

[0080] Finally, step S105 is described in detail, including: Based on the business attributes and service impact of computing power services, determine the abnormal operation and maintenance response actions, including: Obtain the business type from the business attributes, and generate a set of candidate abnormal operation and maintenance response actions based on the business type and service impact score; For each candidate abnormal operation and maintenance response action, a multi-dimensional feasibility verification is performed to obtain an action score; the multi-dimensional feasibility verification includes at least: capacity verification, data location verification, task portability verification, checkpoint status verification, and execution risk verification. When the action score is greater than or equal to the preset action recommendation threshold, the candidate abnormal operation and maintenance response action corresponding to the action score is determined as an abnormal operation and maintenance response action.

[0081] In this application, the candidate abnormal operation and maintenance response actions include at least one of the following: migration task, isolation node, rate limiting inference, switching storage replica, network bypass, pausing low-priority tasks, and manual work order.

[0082] In this application, the action score is used to comprehensively characterize the risk reduction, execution cost, SLA penalty, and feasibility gain of candidate abnormal operation and maintenance response actions.

[0083] In this application, capacity verification is used to verify whether the remaining computing power and storage capacity of the target resources meet the computing power service operation requirements; data location verification is used to verify whether the storage resources required by the service can be accessed by the target node; task portability verification is used to verify whether the service instance is allowed to be migrated; checkpoint status verification is used to verify whether the service has complete and usable checkpoints; execution risk verification is used to assess the risk of secondary failures and SLA defaults caused by the execution of actions.

[0084] In this embodiment of the application, when executing step S105, the service attributes of the computing power service are read, and candidate abnormal operation and maintenance response actions are generated by combining the service impact score output in step S104; multi-dimensional feasibility verification of capacity, data location, task portability, checkpoint status, and execution risk is carried out on each candidate action; the action score of each candidate action is obtained by substituting it into the action score calculation formula; candidate actions whose action scores meet the preset action recommendation threshold are selected and determined as abnormal operation and maintenance response actions; thereby realizing the transmission from the underlying resource anomaly to the business dimension, and outputting an operation and maintenance handling plan that has passed feasibility verification and is adapted to the current business situation.

[0085] For example, this application also provides a formula for calculating the action score:

[0086] In the formula, The action score representing candidate response action a To reduce the amount of risk, For execution costs, SLA penalty This is for feasibility gain.

[0087] In addition to the methods described above, the data center anomaly maintenance methods of this application also include: Obtain feedback and recovery data for actions taken to respond to abnormal operations and maintenance. Based on the feedback recovery data, update the abnormal operation and maintenance response actions.

[0088] In this application, the feedback recovery data refers to the status data collected after the execution of abnormal operation and maintenance response actions, including at least one of the following: operation and maintenance action execution status, fault recovery result, computing power service operation status, and manual review and confirmation information. Specifically, the execution status records whether the abnormal operation and maintenance response action was completed normally and whether any errors occurred during the execution process. The fault recovery result records whether the resource object anomaly has been eliminated and the change in the severity of the anomaly. The computing power service operation status records the business indicators of the affected computing power service, whether the SLA has been breached, and whether the business load has returned to normal. The manual review and confirmation information consists of the evaluation and correction opinions given by the operation and maintenance personnel for this handling, used to mark the action as effective, invalid, or with excessively high risk.

[0089] In this application, updating abnormal operation and maintenance response actions based on feedback recovery data may include updating dependency weights within the computing power service dependency graph, action scoring parameters corresponding to candidate response actions, and various policy thresholds; wherein, policy thresholds include abnormal trigger thresholds, edge retention thresholds, impact thresholds, and action recommendation thresholds.

[0090] In this embodiment, after the abnormal operation and maintenance response action is issued and executed, the system can collect feedback recovery data. This feedback recovery data includes the action execution status, fault recovery result, computing power service operation status, and manual review and confirmation information. Based on the feedback recovery data, on the one hand, the dependency weights of each candidate dependency edge in the computing power service dependency graph can be corrected, and on the other hand, the action scoring-related parameters and various policy thresholds can be updated. When the data center resource anomaly occurs again, the updated parameters and thresholds will participate in the calculation of subsequent anomaly assessment and response actions, realizing closed-loop processing of anomaly assessment, action execution, result feedback, and policy iteration, and continuously optimizing the accuracy and adaptability of data center anomaly operation and maintenance handling.

[0091] This application also provides a device for handling data center anomalies. Figure 2 A structural block diagram of a data center anomaly handling device provided in this application embodiment is shown below. Figure 2 As shown, the operation and maintenance handling device 200 for the computer room anomaly includes: The acquisition unit 201 is used to acquire data center operation data and computing power service status data; the data center operation data is used to indicate the real-time operation status of multiple resource objects in the data center; the computing power service status data is used to indicate the service attributes and operating conditions of computing power services running on each resource object. The first determining unit 202 is used to determine the severity of the anomaly based on the data center operation data, and then determine the abnormal resource object; Construction unit 203 is used to perform topology processing on data center operation data and computing power service status data to generate resource object node set and computing power service node set, and then construct computing power service dependency graph with dependency weights; the dependency weights are determined based on declared dependency score, operation association dependency score, resource sharing dependency score and historical propagation dependency score; one resource object node corresponds to multiple computing power service nodes; The second determining unit 204 is used to determine all computing power service nodes corresponding to the nodes of the abnormal resource objects from the computing power service dependency graph, and then, for each computing power service node, determine the service impact score of the computing power service based on the severity of the abnormality, the dependency weight, the service level agreement SLA weight of the computing power service and the computing power service exposure coefficient. Operation and maintenance unit 205 is used to determine abnormal operation and maintenance response actions based on the business attributes and service impact of computing power services.

[0092] In one exemplary embodiment, the acquisition unit 201 is specifically used for: acquiring data center operation data including: monitoring data of data center hardware indicators, network indicators, storage indicators, power supply and distribution indicators, and heat dissipation domains; acquiring service attributes of computing power services including: service type, SLA weight, and computing power service exposure coefficient; acquiring operating conditions of computing power services including: service operation status, service deployment location, and current load status; acquiring data center operation data and computing power service status data, including: acquiring data center operation data from at least one of the data center hardware management unit and storage monitoring component; and acquiring computing power service status data from at least one of the network controller and computing power cluster scheduling interface.

[0093] In one exemplary embodiment, the first determining unit 202 is specifically used to: for any resource object in the data center operation data, obtain the indicator threshold and confidence level of the resource object; calculate the normalized anomaly severity based on the data center operation data, indicator threshold and confidence level of the resource object; when the anomaly severity is greater than or equal to a preset anomaly triggering threshold, determine the resource object corresponding to the anomaly severity as an abnormal resource object.

[0094] In one exemplary embodiment, the construction unit 203 is specifically used to: perform topology preprocessing on the data center operation data and computing power service status data to obtain a resource object node set and a computing power service node set; the topology preprocessing includes: time alignment processing, identifier normalization processing, and topology deduplication processing; based on the resource object node set and the computing power service node set, generate candidate dependency edges between the resource object nodes and the computing power service node set, and calculate the declared dependency score, runtime association dependency score, resource sharing dependency score, and historical propagation dependency score corresponding to the candidate dependency edges respectively; perform weighted fusion of the declared dependency score, runtime association dependency score, resource sharing dependency score, and historical propagation dependency score to obtain the dependency weight of the candidate dependency edge; integrate all candidate dependency edges and their corresponding dependency weights to obtain the computing power service dependency graph.

[0095] In one exemplary embodiment, the construction unit 203 is specifically used to: when the topology preprocessing is time alignment processing, unify the timestamp benchmark for the data center operation data and the computing power service status data, and perform time alignment processing according to the data source frequency; when the topology preprocessing is identifier normalization processing, maintain the resource identifier mapping table corresponding to the data center operation data and the computing power service status data, and generate a globally unique identifier; when the topology preprocessing is topology deduplication processing, delete the data center operation data and computing power service status data with the same records.

[0096] In one exemplary embodiment, the second determining unit 204 is specifically used to: traverse the computing power service dependency graph, retrieve all computing power service nodes that have dependency edges with the nodes of the abnormal resource objects; for each computing power service node, obtain the SLA weight and computing power service exposure coefficient from the computing power service status data, and multiply the abnormality severity, dependency weight, SLA weight and computing power service exposure coefficient to obtain the service impact score of the corresponding computing power service node.

[0097] In one exemplary embodiment, the operation and maintenance unit 205 is specifically configured to: obtain the business type from the business attributes, and generate a set of candidate abnormal operation and maintenance response actions based on the business type and service impact score; perform multi-dimensional feasibility verification on each candidate abnormal operation and maintenance response action to obtain an action score; the multi-dimensional feasibility verification includes at least: capacity verification, data location verification, task portability verification, checkpoint status verification, and execution risk verification; when the action score is greater than or equal to a preset action threshold, the candidate abnormal operation and maintenance response action corresponding to the action score is determined as an abnormal operation and maintenance response action.

[0098] In one exemplary embodiment, the operation and maintenance unit 205 is further configured to: obtain feedback recovery data of executing abnormal operation and maintenance response actions; and update the abnormal operation and maintenance response actions based on the feedback recovery data.

[0099] Figure 3This is a hardware block diagram of an electronic device provided in an embodiment of this application. The electronic device 300 according to an embodiment of this application includes at least a processor and a memory for storing computer-readable instructions. When the computer-readable instructions are loaded and executed by the processor, the processor performs the data center anomaly handling method described in any of the preceding embodiments of this application.

[0100] Figure 3 The illustrated electronic device 300 specifically includes a CPU 301, a GPU 302, and a memory 303. These units are interconnected via a bus 304. The CPU 301 and / or GPU 302 can be used as the aforementioned processor, and the memory 303 can be used as the aforementioned memory for storing computer-readable instructions. Furthermore, the electronic device 300 may also include a communication unit 305, a storage unit 306, an output unit 307, an input unit 308, and an external device 309, all of which are also connected to the bus 304.

[0101] In summary, this application provides a method, apparatus, and electronic equipment for handling data center anomalies. This application discloses a method for handling data center anomalies, comprising: acquiring data center operation data and computing power service status data; the data center operation data indicating the real-time operation status of multiple resource objects within the data center; the computing power service status data indicating the service attributes and operating conditions of computing power services running on each resource object; determining the severity of the anomaly based on the data center operation data, and thus identifying the abnormal resource object; performing topology processing on the data center operation data and computing power service status data to generate a resource object node set and a computing power service node set, and then constructing a computing power service dependency graph with dependency weights; the dependency weights are determined based on declared dependency scores, operational dependency scores, resource sharing dependency scores, and historical propagation dependency scores; one resource object node corresponds to multiple computing power service nodes; determining all computing power service nodes corresponding to the abnormal resource object node from the computing power service dependency graph, and then, for each computing power service node, determining the service impact score of the computing power service based on the severity of the anomaly, dependency weights, the service level agreement (SLA) weight of the computing power service, and the computing power service exposure coefficient; and determining the anomaly operation and maintenance response action based on the service attributes and service impact score of the computing power service. In contrast to existing solutions that rely solely on device-level alarms for maintenance and cannot quantify the actual impact of anomalies on upper-layer computing services, this application first identifies the abnormal resource objects, then constructs a computing service dependency graph with dependency weights through topology processing, and uses this dependency graph to locate all computing service nodes affected by the abnormal resources. Furthermore, it calculates a service impact score by combining the severity of the anomaly, dependency weights, SLA weights, and computing service exposure coefficients, and then determines the corresponding anomaly maintenance response based on the computing service's own business attributes and the quantified service impact score. Through this layered and progressive processing, the anomalies of lower-level resources are propagated upwards to the upper-layer computing service dimension, solving the technical problem of traditional solutions that can only see hardware failures and cannot assess the extent of service damage. In summary, the technical solution provided by this application can accurately identify computing services affected by data center anomalies, avoiding problems such as alarm overload or decreased alarm accuracy, disordered handling priorities, and handling actions detached from the actual on-site environment, and can adapt to various application scenarios.

[0102] Those skilled in the art will recognize that the units and computing steps of the various examples described in conjunction with the embodiments claimed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0103] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details of the above application are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0104] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0105] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0106] It should also be noted that in the system and method of this application, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of this application.

[0107] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0108] The above description of the claimed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be applied within the widest scope consistent with the principles and novel features of this application.

[0109] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of this application to the forms described herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for handling operational and maintenance anomalies in a data center, characterized in that, The method includes: Acquire data center operation data and computing power service status data; the data center operation data is used to indicate the real-time operation status of multiple resource objects within the data center; the computing power service status data is used to indicate the service attributes and operating conditions of computing power services running on each of the resource objects; Based on the data center operation data, the severity of the anomaly is determined, and then the abnormal resource objects are identified. The data center operation data and the computing power service status data are subjected to topology processing to generate a resource object node set and a computing power service node set, thereby constructing a computing power service dependency graph with dependency weights; the dependency weights are determined based on declared dependency scores, operational association dependency scores, resource sharing dependency scores, and historical propagation dependency scores; one resource object node corresponds to multiple computing power service nodes; From the computing power service dependency graph, determine all computing power service nodes corresponding to the nodes of the abnormal resource objects, and then for each computing power service node, determine the service impact score of the computing power service based on the severity of the abnormality, the dependency weight, the service level agreement (SLA) weight of the computing power service, and the computing power service exposure coefficient. Based on the service attributes and service impact score of the computing power service, determine the abnormal operation and maintenance response action.

2. The method according to claim 1, characterized in that, The data center operation data includes: monitoring data of data center hardware indicators, network indicators, storage indicators, power supply and distribution indicators, and heat dissipation areas; the service attributes of the computing power service include: service type, SLA weight, and computing power service exposure coefficient; the operating conditions of the computing power service include: service operation status, service deployment location, and current load status; the acquisition of data center operation data and computing power service status data includes: The data center operation data is obtained from at least one of the data center hardware management unit and the storage monitoring component; The computing power service status data is obtained from at least one of the network controller and the computing power cluster scheduling interface.

3. The method according to claim 1, characterized in that, The process of determining the severity of the anomaly based on the data center's operational data, and then identifying the abnormal resource objects, includes: For any one of the resource objects in the data center operation data, obtain the indicator threshold and confidence level of the resource object; Based on the data center operation data of the resource object, the indicator threshold, and the confidence level, the normalized severity of the anomaly is calculated. When the severity of the anomaly is greater than or equal to a preset anomaly triggering threshold, the resource object corresponding to the severity of the anomaly is determined to be the resource object of the anomaly.

4. The method according to claim 1, characterized in that, The process of performing topology processing on the data center operation data and the computing power service status data to generate a resource object node set and a computing power service node set, and then constructing a computing power service dependency graph with dependency weights, includes: The data center operation data and the computing power service status data are subjected to topology preprocessing to obtain the resource object node set and the computing power service node set; the topology preprocessing includes: time alignment processing, identifier normalization processing and topology deduplication processing. Based on the resource object node set and the computing power service node set, candidate dependency edges between the resource object nodes and the computing power service node set are generated, and the declared dependency score, the runtime association dependency score, the resource sharing dependency score and the historical propagation dependency score corresponding to the candidate dependency edges are calculated respectively. The dependency weights of the candidate dependency edges are obtained by weighting and fusing the declared dependency score, the runtime association dependency score, the resource sharing dependency score, and the historical propagation dependency score. By integrating all the candidate dependency edges and their corresponding dependency weights, the computing power service dependency graph is obtained.

5. The method according to claim 4, characterized in that, When the topology preprocessing is the time alignment processing, the data center operation data and the computing power service status data are made to have a unified timestamp base, and time alignment processing is performed according to the data source frequency. When the topology preprocessing is the identifier normalization processing, maintain the resource identifier mapping table corresponding to the data center operation data and the computing power service status data, and generate a globally unique identifier; When the topology preprocessing is the topology deduplication process, duplicate records of the data center operation data and the computing power service status data are deleted.

6. The method according to claim 1, characterized in that, The process involves identifying all computing power service nodes corresponding to the nodes of the abnormal resource objects from the computing power service dependency graph, and then, for each computing power service node, determining the service impact score of the computing power service based on the severity of the anomaly, the dependency weight, the service level agreement (SLA) weight of the computing power service, and the computing power service exposure coefficient, including: Traverse the computing power service dependency graph to retrieve all computing power service nodes that have dependency edges with the nodes of the abnormal resource objects; For each computing power service node, the SLA weight and the computing power service exposure coefficient are obtained from the computing power service status data, and the anomaly severity, the dependency weight, the SLA weight and the computing power service exposure coefficient are multiplied together to obtain the service impact score of the corresponding computing power service node.

7. The method according to claim 1, characterized in that, The determination of abnormal operation and maintenance response actions based on the service attributes and service impact scores of the computing power service includes: Obtain the business type from the business attributes, and generate a set of candidate abnormal operation and maintenance response actions based on the business type and the service impact score; For each of the candidate abnormal operation and maintenance response actions, a multi-dimensional feasibility verification is performed to obtain an action score; the multi-dimensional feasibility verification includes at least: capacity verification, data location verification, task portability verification, checkpoint status verification, and execution risk verification. When the action score is greater than or equal to a preset action threshold, the candidate abnormal operation and maintenance response action corresponding to the action score is determined as the abnormal operation and maintenance response action.

8. The method according to claim 1, characterized in that, The method further includes: Obtain feedback recovery data from the execution of the abnormal operation and maintenance response action; Based on the feedback recovery data, update the abnormal operation and maintenance response action.

9. A data center anomaly handling device, characterized in that, The device includes: The acquisition unit is used to acquire data center operation data and computing power service status data; the data center operation data is used to indicate the real-time operation status of multiple resource objects in the data center; the computing power service status data is used to indicate the service attributes and operating conditions of the computing power services running on each of the resource objects. The first determining unit is used to determine the severity of the anomaly based on the data center operation data, and then determine the abnormal resource object. A construction unit is used to perform topology processing on the data center operation data and the computing power service status data to generate a resource object node set and a computing power service node set, and then construct a computing power service dependency graph with dependency weights; the dependency weights are determined based on declared dependency scores, operational association dependency scores, resource sharing dependency scores, and historical propagation dependency scores; one resource object node corresponds to multiple computing power service nodes; The second determining unit is used to determine all computing power service nodes corresponding to the nodes of the abnormal resource objects from the computing power service dependency graph, and then, for each computing power service node, to determine the service impact score of the computing power service based on the severity of the abnormality, the dependency weight, the service level agreement (SLA) weight of the computing power service, and the computing power service exposure coefficient. The operation and maintenance unit is used to determine abnormal operation and maintenance response actions based on the service attributes and service impact scores of the computing power service.

10. An electronic device, characterized in that, include: Memory, used to store computer-readable instructions; as well as A processor for executing the computer-readable instructions, causing the electronic device to perform the method as described in any one of claims 1-8.

Images