A computing power cluster monitoring, scheduling and management system applied to a data center

Abstract

The application relates to the technical field of data center computing power management, in particular to a computing power cluster monitoring and scheduling management system applied to a data center, which comprises a clustered management center, a node balance analysis module, a node property difference module, a scheduling task acquisition module, a node matching module, a scheduling verification module and an alarm feedback module; the application is used for calculating the difference degree of a node and a cluster through a Euclidean distance, avoiding the ambiguity of judging the imbalance only according to the load height, further locking a key difference event through node property difference analysis, enabling operation and management personnel to develop an optimization strategy, reducing the generation rate of deviation nodes from the root, ensuring that high-priority tasks are supported by resources in priority through the construction of a final priority score, and prompting operation and management personnel to adjust the strategy through comprehensive evaluation of the scheduling effect, forming a positioning-scheduling-verification-optimization whole-process closed loop, and guaranteeing the long-term balanced operation of the computing power cluster.

Description

Technical Field

[0001] This invention relates to the field of data center computing power management technology, and in particular to a computing power cluster monitoring and scheduling management system applied to data centers. Background Technology

[0002] With the rapid development of the digital economy, data centers, as the core carrier of computing infrastructure, are experiencing exponential growth in computing power clusters. The number of computing power nodes such as servers and virtual machines within the cluster can reach thousands or even tens of thousands. At the same time, the types of tasks carried by computing power clusters are becoming increasingly complex, covering diverse scenarios such as real-time transaction processing, high-frequency interactive computing, and large-scale offline data training.

[0003] Currently, most mainstream computing cluster monitoring and scheduling technologies focus on extensive optimization of "load migration," but these technologies still have three major shortcomings:

[0004] The location of deviation nodes is vague: it can only determine whether there is a load imbalance in a node, but it cannot accurately quantify the degree of difference between the node and the cluster as a whole, let alone pinpoint the specific root cause of the imbalance, resulting in a lack of targeted scheduling in the future.

[0005] Task scheduling lacks priority criteria: When faced with multiple types of tasks to be processed at deviation nodes, it is difficult to reasonably manage task priority scheduling.

[0006] The scheduling effect lacks closed-loop verification: After task migration, it is only possible to determine whether the node load index is "temporarily balanced" by re-collecting node load indicators, and it is impossible to confirm whether the scheduling has solved the imbalance problem from the root.

[0007] To address the aforementioned technical shortcomings, a solution is proposed. Summary of the Invention

[0008] The purpose of this invention is to provide a computing power cluster monitoring and scheduling management system for data centers to address the aforementioned technical deficiencies.

[0009] The objective of this invention can be achieved through the following technical solution: a computing power cluster monitoring and scheduling management system applied to data centers, including a clustered management center, a node balance analysis module, a node property difference module, a scheduling task acquisition module, a node matching module, a scheduling verification module, and an alarm feedback module;

[0010] The cluster management center is used to obtain the usage data of each device in the computing power cluster and send the usage data to the node load balancing analysis module;

[0011] The node balance analysis module is used to evaluate and analyze the node balance of the data, and to discriminate the obtained phase difference ratio XCi to identify the biased nodes.

[0012] The node property difference module is used to perform event difference locking and node trend classification analysis on the occurrence frequency of each process event type of the collected deviation nodes to obtain key difference events. Further analysis of key difference events yields a feedback list.

[0013] The task acquisition module is used to perform task key indicator assignment and fusion analysis on the task types of the tasks to be processed at each deviation node to obtain the final priority task queue.

[0014] The node matching module is used to analyze the collected CPU idle rate, memory idle rate, and GPU idle rate to determine the target node for task allocation.

[0015] The scheduling verification module is used to evaluate and analyze the balance improvement effect after the computing power cluster is scheduled, and to process the obtained balance improvement score to obtain alarm signals or compliance signals.

[0016] Preferably, the analysis process of the node balance analysis module is as follows:

[0017] The servers or virtual machines in the computing power cluster are labeled as computing power nodes Ji, i = 1, 2, 3, ..., m, where m is a natural number greater than zero;

[0018] Set a sampling period and obtain the usage data of each computing node Ji within the sampling period. The usage data represents the CPU utilization rate of the computing node. At the same time, sample each computing node Ji n times within the sampling period, where n is a natural number greater than zero, and record the CPU utilization rate Jin of each computing node Ji corresponding to the index. Jin represents the sampling data of the CPU utilization rate of the i-th computing node in the nth time.

[0019] The difference in CPU utilization between two nodes is calculated using the Euclidean distance method;

[0020] Calculate the average difference Pi between each computing node and other computing nodes, and then calculate the average of the average differences of all computing nodes to obtain the overall difference QP of the cluster.

[0021] By comparing the difference ratio XCi between the average difference Pi of computing power nodes and the average difference QP of the cluster;

[0022] The phase difference ratio XCi is processed for discrimination. If there is a phase difference ratio XCi greater than or equal to the preset phase difference ratio threshold, the corresponding computing power node is marked as a deviation node.

[0023] Preferably, the analysis process of the node property difference module is as follows:

[0024] The deviation node is marked as o, where o is a natural number greater than zero, and the occurrence count of each process event type of each deviation node within the sampling period is obtained;

[0025] Each process event type is set to h, where h is a natural number greater than zero. The occurrence frequency of each process event type of each deviation node is normalized to obtain the normalized value Goh of each process event type of each deviation node. At the same time, the average value GPh of the normalized value of each process event type h of all computing power nodes in the computing power cluster is obtained.

[0026] Calculate the absolute difference ∆Goh between the normalized value Goh and the average value GPh for each process event type at the deviation node, ∆Goh=∣Goh-GPh∣;

[0027] The absolute difference ∆Goh is processed to identify and classify the process event type corresponding to the absolute difference ∆Goh being greater than the preset absolute difference threshold as a critical difference event.

[0028] Preferably, the normalized value G of the deviation node corresponding to the key difference event is retrieved for 7 days, and the change trend curve of the normalized value G is constructed based on the time series. At the same time, the change curve of the average value GP is plotted in the coordinate system of the change trend curve of the normalized value G. The duration corresponding to the line segment of the change trend curve of the normalized value G above the change curve of the average value GP is obtained, and the duration is discriminated to obtain the persistent imbalance node and the sudden imbalance node.

[0029] Obtain basic information and key discrepancy events for the deviation nodes, and obtain feedback lists for persistent imbalance nodes + key discrepancy events and sudden imbalance nodes + key discrepancy events.

[0030] Preferably, the analysis process of the scheduling task acquisition module is as follows:

[0031] S1: Obtain the task type of each deviation node based on the feedback list, and assign a task score to the task type;

[0032] S2: Obtain the remaining processing time of the tasks to be processed at the deviation node, and obtain the urgency score corresponding to the remaining processing time being within the preset remaining processing time range;

[0033] S3: Obtain the task interruption impact score of the pending tasks at the deviation node;

[0034] S4: The final priority score is obtained by weighting and calculating the task score, urgency score, and task interruption impact score;

[0035] S5: Sort the priority tasks from high to low based on the final priority score to obtain the priority task queue;

[0036] S6: For tasks with the same final priority score in the priority task queue, adjust the sequence number according to the submission time of the tasks to be processed to obtain the final priority task queue.

[0037] Preferably, the analysis process of the node matching module is as follows:

[0038] The computing nodes in the computing cluster, excluding the deviation nodes, are marked as available nodes. The CPU idle rate, memory idle rate, and GPU idle rate of the available nodes are obtained. The average idle rate is calculated by averaging the CPU idle rate, memory idle rate, and GPU idle rate. The value obtained by normalizing the average idle rate is set as the resource idle score.

[0039] Obtain the real-time power consumption of available nodes, normalize the real-time power consumption, and set the value obtained from the normalization of real-time power consumption as the energy cost score.

[0040] Once the positive and load indicators of available nodes are obtained, the positive and load indicators are normalized. The sum of the product of positive indicator × corresponding weight coefficient of positive indicator + (1 - product of load indicator × corresponding weight coefficient of load indicator) = balance contribution score.

[0041] The value obtained by multiplying the resource idleness score by a1, adding (1 - energy cost score by a2) and the balance contribution score by a3, is set as the node adaptability score, where a1, a2, and a3 are all greater than zero. The node adaptability scores are sorted from largest to smallest. If the maximum value of the node adaptability score is unique, the available node with the maximum value of the node adaptability score is selected as the target node for task allocation. If the maximum value of the node adaptability score is not unique, the available node that is closest to the deviation node where the task to be processed is stored is selected as the target node for task allocation.

[0042] Preferably, the analysis process of the scheduling verification module is as follows:

[0043] After the task to be processed is scheduled, the deviation node elimination rate, key difference event mitigation rate and overall difference degree QP change rate of the computing power cluster are obtained. After normalization of the deviation node elimination rate, key difference event mitigation rate and overall difference degree QP change rate, the weighted sum is calculated to obtain the balance improvement score. The balance improvement score is then processed to obtain alarm signal or compliance signal.

[0044] Preferably, the critical difference event mitigation rate is calculated as follows: for each deviation node, the percentage decrease of the absolute difference ∆Goh between the normalized value Goh of the critical difference event after scheduling and the cluster mean GPh is calculated compared to before scheduling, and the average value of all deviation nodes is taken.

[0045] The beneficial effects of this invention are as follows:

[0046] (1) This invention calculates the difference in CPU utilization between nodes by Euclidean distance and combines the difference ratio XCi to accurately quantify the degree of difference between nodes and clusters. This avoids the ambiguity of judging imbalance solely by load level. Furthermore, it identifies key difference events by analyzing the differences in node properties and distinguishes between persistent / sudden imbalance nodes, enabling operation and management personnel to formulate targeted optimization strategies to reduce the occurrence rate of deviation nodes from the root.

[0047] (2) This invention generates an ordered task queue by constructing a final priority score, ensuring that high-priority tasks receive priority resource support, reducing the risk of core business interruption, improving the rationality and reliability of task processing, and prompting operation and management personnel to adjust strategies by comprehensively evaluating the scheduling effect, avoiding repeated scheduling and repeated imbalance of resources waste, forming a closed loop of positioning-scheduling-verification-optimization, and ensuring the long-term balanced operation of the computing power cluster. Attached Figure Description

[0048] The invention will now be further described with reference to the accompanying drawings;

[0049] Figure 1 This is a partial information flow diagram of the present invention;

[0050] Figure 2 This is a flowchart of the system of the present invention;

[0051] Figure 3 This is a partial analysis reference diagram of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments;

[0054] Example 1: Please refer to Figures 1 to 3As shown, the present invention is a computing power cluster monitoring and scheduling management system applied to data centers, including a clustered management center, a node balance analysis module, a node property difference module, a scheduling task acquisition module, a node matching module, a scheduling verification module, and an alarm feedback module.

[0055] The cluster management center and the node balance analysis module have a two-way communication connection. The cluster management center has a one-way communication connection with the node property difference module and the scheduling verification module. The node property difference module has a one-way communication connection with both the scheduling task acquisition module and the alarm feedback module. The scheduling task acquisition module has a one-way communication connection with the node matching module. The node matching module has a one-way communication connection with the alarm feedback module. The alarm feedback module has a one-way communication connection with the cluster management center. The scheduling verification module has a one-way communication connection with the alarm feedback module.

[0056] The cluster management center is used to obtain the usage data of each device in the computing power cluster and send the usage data to the node load balancing analysis module;

[0057] The node balance analysis module is used to evaluate and analyze the node balance of the data. The specific node balance evaluation and analysis process is as follows:

[0058] The servers or virtual machines in the computing power cluster are labeled as computing power nodes Ji, i = 1, 2, 3, ..., m, where m is a natural number greater than zero;

[0059] Set a sampling period and obtain the usage data of each computing node Ji within the sampling period. The usage data represents the CPU utilization rate of the computing node. At the same time, sample each computing node Ji n times within the sampling period, where n is a natural number greater than zero, and record the CPU utilization rate Jin of each computing node Ji corresponding to the index. Jin represents the sampling data of the CPU utilization rate of the i-th computing node in the nth time.

[0060] The difference in CPU utilization between two nodes is calculated using the Euclidean distance method;

[0061] For example, calculate the difference in CPU utilization (CCik) between the i-th computing node and the k-th (k > zero) computing node; Where JKn represents the sampled data of CPU utilization of the k-th computing node in the nth time;

[0062] Calculate the average difference Pi between each computing node and other computing nodes, and then calculate the average of the average differences of all computing nodes to obtain the overall difference QP of the cluster.

[0063] The computing resource balance of computing nodes is quantified by comparing the difference between the average difference Pi of computing power nodes and the average difference QP of the cluster (i.e., the absolute value of (average difference Pi - average difference QP of the cluster) / average difference QP of the cluster).

[0064] The phase difference ratio XCi is processed for discrimination. If there is a phase difference ratio XCi greater than or equal to the preset phase difference ratio threshold, the computing power cluster is determined to be unbalanced, and the corresponding computing power node is marked as a deviation node.

[0065] The deviation nodes are sent to the cluster management center for storage;

[0066] In this invention, identifying unbalanced nodes is a key step in load balancing analysis. The aim is to accurately identify nodes in the cluster whose load behavior differs significantly from that of other nodes, providing specific targets for subsequent optimization.

[0067] Example 2: To address the difficulty in understanding the nature and causes of imbalances at deviation nodes, which leads to a lack of targeted scheduling optimization and only achieves "extensive load migration" without resolving the root cause of the imbalance, the following method is used: The node nature difference module collects the occurrence frequency of each process event type at the deviation node to perform event difference locking and node trend segmentation analysis. The specific process for event difference locking and node trend segmentation analysis is as follows:

[0068] The deviation node is marked as o, where o is a natural number greater than zero;

[0069] The number of occurrences of each process event type at each deviation node within the sampling period is obtained. Each process event type includes system read operations, CPU context switching, etc.

[0070] Set the event type of each process to h, where h is a natural number greater than zero;

[0071] The occurrence frequency of each process event type of each deviation node is normalized to obtain the normalized value Goh of each process event type of each deviation node. At the same time, the average value GPh of the normalized value of each process event type h of all computing power nodes in the computing power cluster is obtained.

[0072] Calculate the absolute difference ∆Goh between the normalized value Goh and the average value GPh for each process event type at the deviation node, ∆Goh=∣Goh-GPh∣;

[0073] The absolute difference ∆Goh is processed to identify and classify the process event type corresponding to the absolute difference ∆Goh being greater than the preset absolute difference threshold as a critical difference event.

[0074] Retrieve the normalized value G of the deviation node corresponding to the key difference event over the past 7 days, construct the trend curve of the normalized value G based on the time series, and simultaneously plot the change curve of the average value GP in the coordinate system of the trend curve of the normalized value G. Obtain the duration corresponding to the line segment of the trend curve of the normalized value G above the change curve of the average value GP, and perform discrimination processing on the duration. If the duration is greater than the preset duration threshold, it is determined to be a persistent imbalance node; if the duration is less than or equal to the preset duration threshold, it is determined to be a sudden imbalance node.

[0075] Obtain basic information (such as ID, IP address, etc.) and key discrepancy events of the deviation nodes, and obtain feedback lists of persistent imbalance nodes + key discrepancy events and sudden imbalance nodes + key discrepancy events.

[0076] The alarm feedback module is used to respond to the feedback list and immediately display the feedback list so that operation and management personnel can manage deviation nodes rationally based on the feedback list, thereby reducing the occurrence rate of deviation nodes;

[0077] The task acquisition module is used to collect the task types of the tasks to be processed at each deviation node, and to perform task key indicator assignment and fusion analysis. The specific task key indicator assignment and fusion analysis process is as follows:

[0078] S1: Based on the feedback list, obtain the task type of each deviation node to be processed. The task types include real-time tasks, interactive tasks and offline tasks. Assign a task score to the task type: real-time tasks 10 points, interactive tasks 8 points and offline tasks 5 points.

[0079] S2: Obtain the remaining processing time of the task to be processed at the deviation node, and obtain the urgency score corresponding to the remaining processing time within the preset remaining processing time interval. It should be noted that the preset remaining processing time interval is set in ascending order, and the corresponding urgency score is set in ascending order, with urgency score (0-10).

[0080] S3: Obtain the task interruption impact score of the task to be processed at the deviation node. If the task to be processed is an intolerable interruption, assign a score of 10; if the task to be processed is an intolerable interruption, assign a score of 5.

[0081] S4: The final priority score is obtained by weighting and calculating the task score, urgency score, and task interruption impact score;

[0082] That is, the final priority score is calculated as: task score × preset task score weighting coefficient + urgency score × preset urgency score weighting coefficient + task interruption impact score × preset task interruption impact score weighting coefficient.

[0083] S5: Sort the priority tasks from high to low based on the final priority score to obtain the priority task queue;

[0084] S6: For tasks with the same final priority score in the priority task queue, adjust the sequence number according to the submission time of the tasks to be processed (the task submitted earlier has priority) to obtain the final priority task queue.

[0085] The alarm feedback module is used to respond to the final priority task queue and immediately display the final priority task queue, so that operation and management personnel can perform targeted retrieval and management of deviation nodes in the computing power cluster based on the feedback list and the final priority task queue, thereby improving the balance of the computing power cluster and the rationality of task scheduling.

[0086] Example 3: The node matching module is used to perform node performance and task matching analysis on the collected CPU idle rate, memory idle rate, and GPU idle rate. The specific node performance and task matching analysis process is as follows:

[0087] Mark all computing nodes in the computing power cluster, except for the deviation nodes, as available nodes;

[0088] The CPU idle rate, memory idle rate, and GPU idle rate of available nodes are obtained. The average idle rate is calculated by averaging the CPU idle rate, memory idle rate, and GPU idle rate. The average idle rate is then normalized and set as the resource idle score.

[0089] Obtain the real-time power consumption of available nodes, normalize the real-time power consumption, and set the value obtained from the normalization of real-time power consumption as the energy cost score.

[0090] Once the positive metrics (such as throughput) and load metrics (such as response time) of available nodes are obtained, the positive metrics and load metrics are normalized. The sum of the product of positive metrics × corresponding weight coefficients of positive metrics + (1 - product of load metrics × corresponding weight coefficients of load metrics) = balance contribution score.

[0091] The value obtained by multiplying the resource idleness score by a1 + (1 - energy cost score × a2) + balanced contribution score by a3 is set as the node adaptability score, where a1, a2, and a3 are all greater than zero. The node adaptability scores are sorted from largest to smallest. If the maximum value of the node adaptability score is unique, the available node with the maximum value of the node adaptability score is selected as the target node for task allocation. If the maximum value of the node adaptability score is not unique, the available node that is closest to the deviation node where the task to be processed is stored is selected as the target node for task allocation.

[0092] The alarm feedback module is used to respond to the task allocation target node and immediately display the task allocation target node so as to support the rational scheduling of tasks in the subsequent final priority task queue.

[0093] The scheduling verification module is used to evaluate and analyze the balance improvement effect after the computing power cluster is scheduled. The specific balance improvement effect evaluation and feedback analysis process is as follows:

[0094] After scheduling the tasks to be processed, obtain the deviation node elimination rate ((number of deviation nodes before scheduling - number of deviation nodes after scheduling) / number of deviation nodes before scheduling), the key difference event mitigation rate, and the overall difference degree QP change rate of the computing power cluster.

[0095] Key difference event mitigation rate: For each deviation node, calculate the percentage decrease of the absolute difference ∆Goh between the normalized value Goh of the key difference event after scheduling and the cluster mean GPh compared to before scheduling, and take the average of all deviation nodes.

[0096] After normalizing the deviation node elimination rate, key difference event mitigation rate, and overall cluster difference QP change rate, a weighted sum is calculated to obtain the balance improvement score. The balance improvement score is then judged. If the balance improvement score is less than the preset balance improvement score threshold, an alarm signal is generated. If the balance improvement score is greater than or equal to the preset balance improvement score threshold, a compliance signal is generated.

[0097] Among them, the sum of the normalized values ​​of the deviation node elimination rate, the key difference event mitigation rate, and the overall difference degree QP change rate of the cluster, multiplied by their respective pre-set weight coefficients, is set as the balance improvement score.

[0098] The alarm feedback module is used to respond to alarm signals or compliance signals and immediately display the preset warning text corresponding to the alarm signal or compliance signal, so as to intuitively understand the improvement of the balance of computing power cluster nodes and make reasonable responses based on the feedback information.

[0099] In summary, the difference in CPU utilization between nodes is calculated using Euclidean distance, and the difference ratio XCi is used to accurately quantify the degree of difference between nodes and the cluster. This avoids the ambiguity of judging imbalance solely based on load levels. Furthermore, the analysis of differences in node characteristics identifies key difference events and distinguishes between nodes with persistent / sudden imbalances. This allows operations and management personnel to develop targeted optimization strategies to reduce the occurrence rate of biased nodes at the root. At the same time, by constructing a final priority score, an ordered task queue is generated to ensure that high-priority tasks (such as real-time transaction tasks) receive priority resource support, reducing the risk of core business interruption and improving the rationality and reliability of task processing.

[0100] Furthermore, by comprehensively evaluating the scheduling effect, it prompts operation and management personnel to adjust strategies to avoid resource waste caused by repeated scheduling and imbalance. By combining the selected optimal task allocation nodes, it meets task requirements while prioritizing low-energy-consumption and high-stability nodes, which improves the utilization rate of computing resources and reduces data center energy consumption, in line with the development trend of green data centers. At the same time, it forms a closed loop of the entire process of positioning, scheduling, verification and optimization, ensuring the long-term balanced operation of the computing cluster.

[0101] The threshold is set for result comparison and analysis to determine whether it is good or bad. The value of the threshold is determined by a combination of large-scale model analysis of sample data and human experience. It can also be adjusted appropriately based on seasonal or common-sense influencing factors.

[0102] The size of the coefficient is a specific value obtained by quantifying each parameter to facilitate subsequent comparison. The size of the coefficient depends on the amount of sample data and the corresponding operating coefficient initially set by those skilled in the art for each set of sample data; as long as it does not affect the proportional relationship between the parameter and the quantified value.

[0103] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A computing power cluster monitoring and scheduling management system applied to data centers, characterized in that, It includes a clustered management center, a node load balancing analysis module, a node property difference module, a scheduling task acquisition module, a node matching module, a scheduling verification module, and an alarm feedback module; The cluster management center is used to obtain the usage data of each device in the computing power cluster and send the usage data to the node load balancing analysis module; The node balance analysis module is used to evaluate and analyze the node balance of the data, and to discriminate the obtained phase difference ratio XCi to identify the biased nodes. The difference ratio XCi = (average difference Pi - average difference QP of the cluster) / absolute value of the average difference QP of the cluster; The node property difference module is used to perform event difference locking and node trend classification analysis on the occurrence frequency of each process event type of the collected deviation nodes to obtain key difference events. Further analysis of key difference events yields a feedback list. The task acquisition module is used to perform task key indicator assignment and fusion analysis on the task types of the tasks to be processed at each deviation node to obtain the final priority task queue. The node matching module is used to analyze the collected CPU idle rate, memory idle rate, and GPU idle rate to determine the target node for task allocation. The scheduling verification module is used to evaluate and analyze the balance improvement effect after the computing power cluster is scheduled, and to process the obtained balance improvement score to obtain alarm signals or compliance signals. The analysis process of the node property difference module is as follows: The deviation node is marked as o, where o is a natural number greater than zero, and the occurrence count of each process event type of each deviation node within the sampling period is obtained; Each process event type is set to h, where h is a natural number greater than zero. The occurrence frequency of each process event type of each deviation node is normalized to obtain the normalized value Goh of each process event type of each deviation node. At the same time, the average value GPh of the normalized value of each process event type h of all computing power nodes in the computing power cluster is obtained. Calculate the absolute difference between the normalized value Goh and the average value GPh of the deviation node for each process event type. Goh, Goh=∣Goh-GPh∣; absolute difference Goh performs discriminative processing, determining the absolute differences. Process events with Goh values ​​greater than a preset absolute difference threshold are defined as critical difference events. Retrieve the normalized value G of the deviation node corresponding to the key difference event over the past 7 days, construct the change trend curve of the normalized value G based on the time series, and simultaneously plot the change curve of the average value GP in the coordinate system of the change trend curve of the normalized value G. Obtain the duration corresponding to the line segment of the change trend curve of the normalized value G above the change curve of the average value GP, and perform discrimination processing on the duration to obtain the persistent imbalance node and the sudden imbalance node. Obtain basic information and key discrepancy events for the deviation nodes, and obtain feedback lists for persistent imbalance nodes + key discrepancy events and sudden imbalance nodes + key discrepancy events. The analysis process of the scheduling task acquisition module is as follows: S1: Obtain the task type of each deviation node based on the feedback list, and assign a task score to the task type; S2: Obtain the remaining processing time of the tasks to be processed at the deviation node, and obtain the urgency score corresponding to the remaining processing time being within the preset remaining processing time range; S3: Obtain the task interruption impact score of the pending tasks at the deviation node; S4: The final priority score is obtained by weighting and calculating the task score, urgency score, and task interruption impact score; S5: Sort the priority tasks from high to low based on the final priority score to obtain the priority task queue; S6: For tasks with the same final priority score in the priority task queue, adjust the sequence number according to the submission time of the tasks to be processed to obtain the final priority task queue. The analysis process of the node matching module is as follows: The computing nodes in the computing cluster, excluding the deviation nodes, are marked as available nodes. The CPU idle rate, memory idle rate, and GPU idle rate of the available nodes are obtained. The average idle rate is calculated by averaging the CPU idle rate, memory idle rate, and GPU idle rate. The value obtained by normalizing the average idle rate is set as the resource idle score. Obtain the real-time power consumption of available nodes, normalize the real-time power consumption, and set the value obtained from the normalization of real-time power consumption as the energy cost score. Once the positive and load indicators of available nodes are obtained, the positive and load indicators are normalized. The sum of the product of positive indicator × corresponding weight coefficient of positive indicator + (1 - product of load indicator × corresponding weight coefficient of load indicator) = balance contribution score. The value obtained by multiplying the resource idleness score by a1, adding (1 - energy cost score by a2) and the balance contribution score by a3, is set as the node adaptability score, where a1, a2, and a3 are all greater than zero. The node adaptability scores are sorted from largest to smallest. If the maximum value of the node adaptability score is unique, the available node with the maximum value of the node adaptability score is selected as the target node for task allocation. If the maximum value of the node adaptability score is not unique, the available node that is closest to the deviation node where the task to be processed is stored is selected as the target node for task allocation.

2. The computing power cluster monitoring and scheduling management system for data centers according to claim 1, characterized in that, The analysis process of the node balance analysis module is as follows: The servers or virtual machines in the computing power cluster are labeled as computing power nodes Ji, i = 1, 2, 3, ..., m, where m is a natural number greater than zero; Set a sampling period and obtain the usage data of each computing node Ji within the sampling period. The usage data represents the CPU utilization rate of the computing node. At the same time, sample each computing node Ji n times within the sampling period, where n is a natural number greater than zero, and record the CPU utilization rate Jin of each computing node Ji corresponding to the index. Jin represents the sampling data of the CPU utilization rate of the i-th computing node in the nth time. The difference in CPU utilization between two nodes is calculated using the Euclidean distance method; Calculate the average difference Pi between each computing node and other computing nodes, and then calculate the average of the average differences of all computing nodes to obtain the overall difference QP of the cluster. By comparing the difference ratio XCi between the average difference Pi of computing power nodes and the average difference QP of the cluster; The phase difference ratio XCi is processed for discrimination. If there is a phase difference ratio XCi greater than or equal to the preset phase difference ratio threshold, the corresponding computing power node is marked as a deviation node.

3. The computing power cluster monitoring and scheduling management system for data centers according to claim 2, characterized in that, The analysis process of the scheduling verification module is as follows: After the task to be processed is scheduled, the deviation node elimination rate, key difference event mitigation rate and overall difference degree QP change rate of the computing power cluster are obtained. After normalization of the deviation node elimination rate, key difference event mitigation rate and overall difference degree QP change rate, the weighted sum is calculated to obtain the balance improvement score. The balance improvement score is then processed to obtain alarm signal or compliance signal.

4. The computing power cluster monitoring and scheduling management system for data centers according to claim 3, characterized in that, Critical Difference Event Mitigation Rate: For each deviation node, calculate the absolute difference between the normalized value Goh of the critical difference event after scheduling and the cluster mean GPh. The percentage decrease in Goh compared to before scheduling is the average of all deviation nodes.

Images