A hierarchical architecture heterogeneous cluster power consumption monitoring and control system and method

Abstract

The application discloses a layered architecture heterogeneous cluster power consumption monitoring and control system and method, including a node agent module deployed on each computing node, a unified scheduling hardware component object, collecting hardware state data and splicing into a unified time sequence index, while exposing monitoring and control endpoints; a time sequence database module actively pulls and stores time sequence samples at a preset period; a heterogeneous sensing intelligent decision engine module sequentially performs power consumption trend prediction, differentiated scoring and hysteresis decision of load sensing, power consumption-temperature combined anomaly classification correction, and cluster energy efficiency sensing power consumption quota allocation on the time sequence samples, and finally generates control parameters; a control front-end module encapsulates and issues the parameters to the node agent module for execution, forming an automatic closed loop of collection-prediction-decision-execution-feedback. The advantage is that through unified hardware abstraction, prediction-driven decision, anomaly classification correction and cluster collaborative quota management, the timeliness, stability and energy efficiency adaptation capability of heterogeneous cluster power consumption control are improved.

Description

Technical Field

[0001] This invention relates to the fields of high-performance computing and data center operation and maintenance technology, and in particular to a hierarchical architecture heterogeneous cluster power consumption monitoring and control system and method. Background Technology

[0002] As the scale of high-performance computing and artificial intelligence businesses continues to grow, the complexity of power management for heterogeneous components such as central processing units (CPUs), graphics processing units (GPUs), and baseboard management controllers (BMCs) in clusters is constantly increasing. Existing power monitoring and control solutions typically rely on different hardware interfaces such as sysfs, NVML, and REDFISH for data acquisition or control operations. Overall, there is a lack of unified interface abstraction and standardized management models, resulting in high management and expansion costs across hardware platforms.

[0003] A search revealed that Chinese patent CN121455696A discloses a heterogeneous GPU resource management and scheduling method, which uses a device plugin combined with a Prometheus and Grafana stack to collect GPU power consumption metrics. While this method can achieve a certain degree of power consumption data visualization, it lacks the ability to predict power consumption trends and an automated power consumption decision-making mechanism, still requiring manual intervention for judgment and control. US patent US7725572B1 discloses a central monitoring architecture based on an MBean server. This architecture is geared towards general management scenarios and fails to provide differentiated control strategies for the differences in power consumption response rate, nonlinear characteristics, and thermal inertia of heterogeneous hardware. Chinese patent CN112291346A discloses a mimicry application deployment and management system for heterogeneous node clusters, which adopts a heterogeneous node monitoring service timed reporting mechanism. However, its function is mainly limited to node-level status monitoring and fails to build cluster-level predictive power quota management capabilities. Based on the above comparative literature, it can be seen that existing solutions generally lack the ability to predict power consumption trends, the ability to make automated power consumption decisions, the mechanism for differentiated processing of heterogeneous hardware, the mechanism for automatic detection and classification of anomalies, and the ability to dynamically manage power consumption quotas based on prediction within the cluster.

[0004] In actual system deployments, existing technologies still have several architectural shortcomings: inconsistent hardware management interfaces (such as sysfs, NVML, REDFISH, etc.) lead to high costs for cross-hardware expansion; monitoring and control links are disconnected, and power consumption control decisions mostly rely on manual judgment, making it difficult to form an automated closed-loop process from perception to control; in multi-node cluster scenarios, the dependencies and data flows between components lack clear definitions, making it difficult to effectively implement system-level coordinated power consumption control; in addition, existing solutions fail to fully perceive and adapt to the essential differences in power consumption change rates, nonlinear characteristics, and thermal inertia of different hardware types such as central processing units, graphics processing units, and baseboard management controllers, thus restricting the refinement of power consumption control systems and the overall energy efficiency optimization effect. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a hierarchical architecture for monitoring and controlling the power consumption of heterogeneous clusters, so as to solve the problems of inconsistent hardware interfaces, fragmented monitoring and control, reliance on human experience for power consumption decisions, and lack of predictive allocation of cluster power consumption budget in heterogeneous clusters.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows: a hierarchical architecture heterogeneous cluster power consumption monitoring and control system, including a node agent module, a time-series database module, a heterogeneous perception intelligent decision engine module and a control front-end module; The node proxy module is deployed in each computing node of the heterogeneous cluster and includes multiple hardware component objects. The node proxy module is used to uniformly schedule the data update interface of each hardware component object and concatenate the hardware status data returned by each hardware component object into a time-series indicator in Prometheus text format, while exposing HTTP monitoring endpoints and HTTP control endpoints. The time-series database module is used to actively pull time-series indicators exposed by the HTTP monitoring endpoint according to a preset collection period, and perform timestamp alignment and deduplication storage operations on the time-series indicators in sequence to obtain time-series samples. The heterogeneous perception intelligent decision engine module includes an adaptive prediction submodule, a heterogeneous scoring decision submodule, a multivariate anomaly detection submodule, and a prediction-driven collaborative allocation submodule. The adaptive prediction submodule is used to predict time series samples to obtain prediction results including the predicted power consumption value for the next k steps, the predicted first-order increment, and the predicted second-order gradient. The heterogeneous scoring decision submodule is used to identify the load type based on hardware status data and select the corresponding weight configuration according to the identified load type; perform differentiated normalization on different hardware status data, and perform weighted summation of the normalized hardware status data with the predicted first-order increment and predicted second-order gradient according to the weight configuration to obtain a comprehensive score; then form a hysteresis dead zone through preset upgrade threshold and downgrade threshold, and determine the control mode and generate candidate control parameters based on the relationship between the comprehensive score and the hysteresis dead zone; The aforementioned multivariate anomaly detection submodule is used to correct candidate control parameters based on power consumption-temperature anomaly categories to obtain response parameters; The prediction-driven collaborative allocation submodule is used to fine-tune the response parameters according to the cluster budget to obtain the final control parameters; The control front-end module is used to encapsulate the final control parameters into an HTTP / API request and send it to the HTTP control endpoint of the target node; The node proxy module is also used to parse HTTP requests received through the HTTP control endpoint, route the final control parameters to the corresponding hardware component object, and execute control through the hardware control interface.

[0007] Compared with existing technologies, the advantages of this invention lie in deploying a node agent module on each computing node. This module uniformly schedules the data update interfaces of various hardware components and concatenates the status data returned by different hardware into a unified Prometheus text-format time-series indicator. Simultaneously, it exposes independent HTTP monitoring and control endpoints. This achieves standardized abstraction of heterogeneous hardware interfaces such as sysfs, NVML, and REDFISH, reducing the management and expansion costs across hardware platforms. Furthermore, it clarifies the physical boundaries and interaction protocols of the monitoring data acquisition link and the control command issuance link, thus eliminating the disconnect between monitoring and control. This provides the architectural foundation for subsequent automated closed-loop decision-making. The time-series database module actively pulls monitoring endpoint data according to a preset collection cycle and performs timestamp alignment and deduplication storage, providing high-quality time-series samples for the adaptive prediction submodule. Based on the time-series samples, the adaptive prediction submodule outputs multi-dimensional prediction results, including predicted power consumption values ​​for the next k steps, predicted first-order increments, and predicted second-order gradients. This enables the system to perceive power consumption change trends in advance, providing a forward-looking basis for subsequent decisions and avoiding the problem of lagging regulation caused by relying solely on current instantaneous values ​​in traditional solutions. The heterogeneous scoring and decision-making submodule can identify the current load type based on hardware status data and target different load types accordingly. For the same load type, corresponding weight configurations are selected, and then differentiated normalization processing is performed on the data of different hardware states. The normalized data is then weighted and summed with the predicted first-order increment and predicted second-order gradient to obtain a comprehensive score. Then, using the hysteresis dead zone formed by the preset upgrade threshold and downgrade threshold, the control mode is determined and candidate control parameters are generated based on the relative relationship between the comprehensive score and the dead zone. This process fully considers the essential differences in power consumption response rate, nonlinear characteristics, and thermal inertia of heterogeneous hardware such as central processing unit, graphics processing unit, and board management controller. Through load-aware weight adaptation and hysteresis dead zone design, the control parameters are effectively avoided near the critical point. The frequent oscillations improve the stability of control decisions and hardware adaptability; the multivariate anomaly detection submodule corrects candidate control parameters based on power consumption-temperature anomaly categories to obtain response parameters, enabling timely adjustment of control strategies when anomalies are detected, preventing improper control under abnormal operating conditions, and enhancing the system's safety and robustness; the prediction-driven collaborative allocation submodule further fine-tunes the response parameters based on the cluster's global budget to obtain the final control parameters, so that the power consumption adjustment of each node is not only based on its own prediction and state, but also collaboratively meets the power consumption quota constraints of the entire cluster, realizing the unification from node-level fine-grained control to cluster-level global optimization;The control front-end module encapsulates the final control parameters into HTTP / API requests and sends them to the control endpoint of the target node. The node proxy module parses the request and routes the parameters to the corresponding hardware component object, executing the actual control operation through the hardware control interface. This completes the entire closed-loop process of "data acquisition—time-series storage—predictive analysis—differentiated decision-making—anomaly correction—cluster collaboration—command execution," requiring no manual intervention and significantly reducing the operational burden. Furthermore, through a comprehensive mechanism of predictive-driven, heterogeneous perception, and global collaboration, it effectively improves the power consumption control accuracy, response speed, and overall energy efficiency of heterogeneous clusters under dynamic loads.

[0008] The hardware component objects include at least a central processing unit (CPU) component, a graphics processing unit (GPU) component, and a baseboard management controller (BMC) component in the cluster; wherein, the CPU component is used to collect hardware status data including CPU frequency, CPU utilization, and frequency modulation strategy; the GPU component is used to collect hardware status data including GPU power consumption, temperature, GPU utilization, and power consumption limits; and the BMC component is used to collect hardware status data including overall power consumption, fan status, and sensor status.

[0009] The specific operation of the adaptive prediction submodule is as follows: For each hardware component object, maintain an independent Holt two-parameter exponential smoothing prediction model, and recursively calculate the power consumption prediction value for the next k steps according to the horizontal component and the trend component. Obtain the normalized prediction error and perform parameter self-tuning updates on the horizontal smoothing coefficient and the trend smoothing coefficient; Obtain the first-order predicted increment and the second-order predicted gradient, where the first-order predicted increment is the difference between the predicted power consumption value in the next step and the current power consumption observation value, and the second-order predicted gradient is the difference between the current first-order predicted increment and the previous first-order predicted increment. The system calculates the confidence interval for the k-step power consumption prediction. When the upper bound of the confidence interval exceeds the budget threshold, it triggers the heterogeneous scoring decision submodule or the prediction-driven collaborative allocation submodule to execute a decision in advance. The prediction models of each hardware component can dynamically adjust parameters according to their own power consumption change characteristics, effectively adapting to the differences in power consumption response rate and nonlinear characteristics among the central processing unit, graphics processing unit, and baseboard management controller. By calculating the first-order increment and the second-order gradient of the prediction, the system can quantify the rate of change and acceleration trend of power consumption, providing richer dynamic characteristics for subsequent comprehensive scoring than a single prediction value. At the same time, by calculating the confidence interval for the k-step prediction value and triggering the decision module to execute an operation in advance when the upper bound of the confidence interval exceeds the budget threshold, the system can respond before the actual power consumption exceeds the limit, realizing a shift from passive alarm to proactive prevention, and significantly improving the foresight and security of cluster power consumption control.

[0010] In the heterogeneous scoring decision submodule, the upgrade threshold is greater than the downgrade threshold. When the overall score exceeds the upgrade threshold, the control mode switches to a more conservative mode, generating more aggressive power-reduction candidate control parameters. When the overall score is lower than the downgrade threshold, the control mode switches to a more aggressive mode, allowing performance improvement. When the overall score is between the two, the current control mode is maintained. This three-stage decision mechanism enables the system to respond quickly to power consumption risks while fully utilizing hardware performance within a safe range, achieving a reasonable dynamic balance between power consumption constraints and computational efficiency.

[0011] The multi-dimensional anomaly detection submodule calculates the power consumption Z-score and temperature Z-score respectively, and distinguishes between real overload, sensor anomaly, hardware failure and heat dissipation anomaly based on the joint relationship between the two, thereby determining the power consumption-temperature anomaly category; wherein, when it is determined to be a real overload or heat dissipation anomaly, protective power reduction or fan strategy adjustment is performed on the candidate control parameters; when it is determined to be a sensor anomaly or hardware failure, only an alarm is triggered without modifying the candidate control parameters. The system calculates power consumption Z-score and temperature Z-score separately, and classifies anomalies into four categories based on their combined relationship: actual overload, sensor anomaly, hardware failure, and heat dissipation anomaly. This joint discrimination method overcomes the shortcomings of single threshold detection, which is prone to false alarms or missed alarms. It can accurately locate the root cause of the anomaly rather than just sensing the abnormal phenomenon. When it is determined to be an actual overload or heat dissipation anomaly, it means that the increase in power consumption is indeed due to the actual load or deterioration of heat dissipation conditions. At this time, protective power reduction or fan strategy adjustment of candidate control parameters can effectively curb the further increase in power consumption or improve the heat dissipation environment, playing a substantial protective role. When it is determined to be a sensor anomaly or hardware failure, since the data itself is unreliable or the hardware has failed, only an alarm is triggered without correcting the control parameters. This avoids generating inappropriate control commands based on erroneous data, prevents ineffective control or secondary damage caused by misjudgment, and enhances the system's fault tolerance and operational safety under abnormal conditions.

[0012] The prediction-driven collaborative allocation submodule calculates the power consumption quota based on the predicted power consumption, current power consumption, and comprehensive energy efficiency ratio of each node. When the current power consumption of a node exceeds the corresponding quota, the response parameter is adjusted to a control parameter for power reduction. The comprehensive energy efficiency ratio is defined as the ratio of the node's performance proxy value to its current power consumption. The prediction-driven collaborative allocation submodule does not only determine the quota based on the current power consumption, but also calculates the power consumption quota of each node by comprehensively considering the predicted power consumption, current power consumption, and comprehensive energy efficiency ratio (the ratio of performance proxy value to current power consumption). The predicted power consumption reflects the power consumption trend of the node in the short term, the current power consumption reflects the real-time state, and the comprehensive energy efficiency ratio characterizes the performance value that the node can contribute per unit of power consumption. The comprehensive evaluation of the three factors ensures that the quota allocation not only focuses on the current and future energy consumption constraints, but also prioritizes the allocation of power consumption budget to nodes with higher energy efficiency ratios. Thus, under the premise of limited total power consumption budget of the cluster, the power consumption quota of each node is allocated with a more reasonable strategy, thereby improving the overall energy efficiency of the cluster. When the current power consumption of a node exceeds the corresponding quota, the response parameter is adjusted to the control parameter in the direction of power reduction. This mechanism enables the actual power consumption of each node to converge towards the quota target, realizing dynamic power consumption quota management based on prediction and energy efficiency priority within the cluster.

[0013] It also includes a visualization module, which is used to read time-series samples and form a monitoring view; The system supports automatic mode, supervised mode, and manual takeover mode. In automatic mode, the heterogeneous perception intelligent decision engine module automatically generates and distributes the final control parameters. In supervised mode, the final control parameters are confirmed by the user through the visualization module before being distributed by the control front-end module. In manual takeover mode, the user directly sets the control parameters through the control front-end module. The visualization module provides users with an intuitive presentation of key information such as power consumption trends and abnormal states. The system supports three operating modes: automatic, supervised, and manual takeover. In automatic mode, the intelligent decision engine automatically generates and distributes final control parameters, achieving fully automated closed-loop control without human intervention, suitable for efficient operation and maintenance of large-scale clusters under normal operating conditions. In supervised mode, final parameters are confirmed by the user through the visualization module before distribution, balancing automation efficiency with the security of manual review, suitable for cautious deployment scenarios in the initial stages of critical business or strategy adjustments. In manual takeover mode, users directly set control parameters, providing the highest-level manual intervention for extreme abnormal situations or special operational needs. This hierarchical design of the three modes allows the system to flexibly switch between automation and manual controllability, adapting to diverse operational needs under different application scenarios, management strategies, and security levels.

[0014] A method for monitoring and controlling the power consumption of a heterogeneous cluster using the hierarchical architecture described above includes the following steps: Step 1: Obtain the hardware status data collected by each hardware component object and concatenate them into a time-series index in Prometheus text format. Step 2: Perform timestamp alignment and deduplication storage operations on the time series indicators according to the preset collection period to obtain time series samples; Step 3-1: Predict the time series samples to obtain prediction results including the predicted power consumption value for the next k steps, the predicted first-order increment, and the predicted second-order gradient. Step 3-2: Identify the load type based on the hardware status data, and select the corresponding weight configuration based on the identified load type; perform differentiated normalization on different hardware status data, and sum the normalized hardware status data with the predicted first-order increment and predicted second-order gradient according to the weight configuration to obtain a comprehensive score; then form a hysteresis dead zone through preset upgrade threshold and downgrade threshold, and determine the control mode and generate candidate control parameters based on the relationship between the comprehensive score and the hysteresis dead zone. Step 3-3: Correct the candidate control parameters according to the power consumption-temperature anomaly category to obtain the response parameters; Steps 3-4: Adjust the response parameters according to the cluster budget to obtain the final control parameters; Step 4: The final control parameters obtained in Steps 3-4 are sent to each computing node, and each computing node routes them to the corresponding hardware component object to execute the control. Step 5: After executing the control, return to Step 1 to re-acquire hardware status data and continue execution, forming a feedback loop. The timing logic and data flow relationships between the steps clearly define the complete closed-loop path of data acquisition, prediction and decision-making, control execution, and feedback updates. This enables the entire system to automatically complete the iterative process of "perception—prediction—decision-execution—feedback" in each acquisition cycle, thereby ensuring that the cluster power consumption status can be continuously tracked, dynamically evaluated, and adjusted in real time, ultimately achieving the goal of stable, continuous, and adaptive power management under long-term operating conditions. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall system architecture of the present invention; Figure 2 UML class diagram for hardware components; Figure 3 This is a diagram of the time-series data structure. Figure 4 A diagram showing the relationships between Prometheus components; Figure 5 For the overall information dashboard of the cluster; Figure 6This is the interface for an integrated monitoring and control platform. Figure 7 For node and general hardware information dashboards; Figure 8 This is a dashboard for GPU hardware information. Detailed Implementation

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0017] like Figure 1 As shown, a hierarchical architecture heterogeneous cluster power consumption monitoring and control system includes a node agent module, a time-series database module, a visualization module, a heterogeneous perception intelligent decision engine module, and a control front-end module. The node agent module is deployed across the compute nodes of the heterogeneous cluster. Each node agent module contains multiple hardware component objects, including at least the CPU, GPU, and baseboard management controller components. The node agent module uniformly schedules the data update interfaces of each hardware component object and concatenates the hardware status data returned by each hardware component object into a Prometheus text-formatted time-series metric. It also exposes HTTP monitoring and control endpoints. Specifically, the CPU component collects hardware status data including CPU frequency, CPU utilization, and frequency modulation policies; the GPU component collects hardware status data including GPU power consumption, temperature, GPU utilization, and power limits; and the baseboard management controller component collects hardware status data including overall system power consumption, fan status, and sensor status. Figure 2 The UML class diagram further illustrates the inheritance relationship between CPU components, GPU components, and BMC components that implement the standard hardware abstraction interface based on the abstract base class, as well as the composition relationship between node proxy modules and hardware component objects; Figure 3 This provides the organization method for indicator names, labels, and timestamp samples in time series data.

[0018] Power consumption variables used in prediction, scoring, anomaly detection, and collaborative allocation are uniformly converted to watts (W). GPU power consumption is obtained by dividing the nvgpu_power milliwatt value collected by NVML by 1000; CPU power consumption is preferentially based on the CurrentCPUPowerWatts and other sub-items collected by the BMC component. If the target server does not provide CPU sub-items power consumption, a CPU power consumption proxy value is constructed from CPU utilization, current frequency, maximum frequency, and historical node total power consumption; the CPU power consumption proxy value can be calculated as P_cpu = P_idle + (u_cpu)(f_cur / f_max)(P_node - P_idle - P_gpu - P_other), where u_cpu is the CPU utilization, f_cur / f_max is the current frequency ratio, and P_idle can be obtained from the average of the historical low-load window; if this estimated value is less than 0, it is set to 0; the node's current power consumption P_i^{current} is preferentially based on the BMC total power consumption or power input power consumption. If it is missing, it is obtained by summing the GPU power consumption, CPU power consumption proxy value, and other component power consumption estimates. It should be noted that the above proxy values ​​are only used for prediction, scoring, and quota allocation, and do not change the existing CPU / GPU / BMC control interface of the node proxy module.

[0019] The time-series database module is used to actively pull time-series metrics exposed by HTTP monitoring endpoints according to a preset collection period (5-15s). The time-series metrics are then aligned with timestamps and deduplicated for storage to obtain time-series samples. A time-series sample can be represented as m=(name, labels, ts, value, unit), where name is the metric name, labels are the labels, including at least node id, component id, and device id, ts is the sampling timestamp, value is the sampled value, and unit is the unit. During timestamp alignment, the collection period tau is used as the bucket width, and samples with the same name and label that fall into the same bucket are grouped into the same sampling time t. If duplicate samples exist in the same bucket, the one with the latest timestamp is retained. Figure 4 This demonstrates the proactive retrieval relationship between a time-series database module (such as Prometheus) and the HTTP monitoring endpoint of the node agent module. This proactive retrieval method ensures the reliable collection of monitoring data.

[0020] The heterogeneous perception intelligent decision engine module reads time-series samples and processes them sequentially through its four sub-modules. The heterogeneous perception intelligent decision engine module includes an adaptive prediction sub-module, a heterogeneous scoring decision sub-module, a multivariate anomaly detection sub-module, and a prediction-driven collaborative allocation sub-module. The adaptive prediction submodule is used to predict time-series samples, obtaining prediction results that include power consumption predictions for the next k steps, predictions of the first-order increment, and predictions of the second-order gradient. Its specific operation process is as follows: Obtain the horizontal component at the current sampling time. and trend components : , ,in, Indicates the horizontal component, Indicates trend components, Indicates the current sampling time. and These represent the horizontal smoothing coefficient and the trend smoothing coefficient, respectively, both ranging from (0,1). The power consumption observation at the current sampling moment (provided by the value in the time series sample); Get the future at the current sampling time Step power consumption prediction value Used to predict in advance whether future power consumption is close to or exceeds the budget: , ,in, Indicates the prediction step size. This indicates the maximum number of prediction steps, typically ranging from 3 to 12, corresponding to an advance of approximately 30 seconds to 3 minutes. The confidence interval can be expressed as Used to provide prediction uncertainty, when a single node or single component... If the upper bound of the confidence level exceeds the node budget threshold, a heterogeneous scoring decision is triggered first; if the sum of the predicted demands of multiple nodes exceeds the cluster budget, a prediction-driven collaborative allocation is triggered. This reflects that the uncertainty of multi-step forecasting increases with the step size; During model initialization, it can be set to , If there is only one initial sample, then let Predicted residual standard deviation Based on the most recent M errors calculate; Obtaining normalized prediction error , ,in, The current sampling time The predicted power consumption of a single-step backtest. To prevent positive numbers with a denominator of zero, we can take 10. -6 Or a positive number not less than the sampling precision; After parameter self-tuning, the horizontal smoothing coefficient and trend smoothing coefficient are updated as follows: , ,in, The `clip` function is used to restrict arguments to a given range of upper and lower bounds. and These represent the learning rates for parameter self-tuning. This represents the average normalized prediction error over the past M steps. This represents the average trend change over the past M steps, where M is the length of the parameter self-tuning sliding window, representing the most recent M sampling periods. M is typically between 3 and 20. and These represent the upper and lower bounds of the horizontal smoothing coefficient, which can range from 0.05 to 0.95. and These represent the upper and lower bounds of the trend smoothing coefficient, which can range from 0.05 to 0.95; the parameters are in self-tuning. Update item adopts and To mitigate the deviation and avoid underestimation of volatility intensity due to the cancellation of positive and negative trend changes; Obtain the first-order increment of the prediction This indicates the magnitude of the increase or decrease in power consumption relative to the current level in the next step: ,in, This represents the actual power consumption value at the current sampling moment; Obtain the predicted second gradient Indicate whether the upward or downward trend in power consumption is accelerating: .

[0021] The heterogeneous scoring decision submodule first identifies the current load type based on hardware status data, specifically by constructing a load fingerprint vector. Used to save snapshots of current CPU / GPU usage, power consumption, temperature, and other status parameters. ,in, Indicates CPU utilization. Indicates GPU utilization. This represents the ratio of CPU utilization to GPU utilization. , This prevents positive numbers with a denominator of zero. This is the normalized value for temperature risk. This is the normalized value for power consumption; if and If the load type at the current sampling time is idle, then... and If the load type at the current sampling time is CPU intensive; and If the load type at the current sampling time is GPU intensive, then the load type at the current sampling time is GPU intensive; otherwise, the load type at the current sampling time is hybrid computing. The threshold for determining idle time. The threshold for CPU-intensive detection. The threshold for determining GPU density; in this embodiment, , , ; Then, select the corresponding weight configuration based on the identified load type. The weight configuration is set in the order of indicators (i.e., CPU utilization, GPU utilization, temperature risk, and power consumption risk), for example, idle type (0.20, 0.20, 0.20, 0.40), CPU intensive type (0.40, 0.15, 0.20, 0.25), GPU intensive type (0.15, 0.40, 0.20, 0.25), and mixed computing type (0.25, 0.25, 0.25, 0.25). Different normalization functions are used for different hardware metrics to map CPU utilization, GPU utilization, temperature and power consumption to a unified dimension that can be compared in a weighted manner: CPU utilization and power consumption are normalized linearly, GPU utilization is normalized by power law to enhance sensitivity in high load ranges, and temperature is normalized by Sigmoid to highlight the risks when approaching the temperature threshold. CPU utilization normalized to ,in, This indicates the current CPU utilization. This represents the minimum CPU utilization, which can be 0. This represents the maximum CPU utilization, which can be 1 or 100%. If the original CPU utilization is recorded as a percentage, first divide by 100 to convert it to 0 to 1. GPU utilization normalized to ,in, This indicates the current GPU utilization rate. This represents the minimum GPU utilization. This represents the maximum GPU utilization. This represents the GPU power-law shape factor, typically ranging from 1 to 3, used to enhance the impact of high GPU utilization intervals on the score. Temperature risk normalization ,in, Indicates the Sigmoid slope. Indicates the original temperature. Indicates the inflection point of temperature risk; Power consumption normalized to ,in, Current power consumption, , It can be determined by the node's historical window, the lower limit of rated power consumption, and the upper limit of power consumption; Before calculating the overall score, the predicted trend term is normalized to prevent the watt dimension from directly entering the 0-1 scoring range. , The overall score is defined as follows: ,in, , Indicates the load type at the current sampling time. Indicates the index under the load type at the current sampling time. Dynamic weights, and These represent coefficients, both ranging from 0 to 1, and can be 0.15 and 0.10 respectively. Next, through the preset upgrade threshold and downgrade threshold A hysteresis dead zone is formed when the upgrade threshold exceeds the downgrade threshold. Based on a comparison of the comprehensive score and the threshold, control modes are determined according to conservatism from low to high (performance → balance → energy saving → emergency), and candidate control parameters are generated. , Indicates the CPU operating mode or target frequency. Indicates the GPU power consumption limit value. The BMC fan control strategy is represented by the following mapping rules: when and At this time, the control mode is performance. Set to performance-priority frequency tuning strategy / hold frequency. Set to maximum or nominal power consumption. Set to automatic fan policy; when At that time, the control mode is balanced. Set the scheduler to an adaptive frequency modulation strategy. Set as nominal power consumption. Set to automatic fan policy; when or At that time, the control mode is energy saving. Set to energy-saving priority frequency regulation strategy. Set to reduce the power consumption limit. Set to automatic fan slowdown policy; when and At this time, the control mode is emergency. Set to energy-saving priority frequency regulation strategy + frequency reduction. Set to minimum safe power consumption. Set to manual maximum speed; If the previous cycle pattern and the current score fall into the dead zone (i.e.) If it is, then the pattern of the previous cycle will be maintained.

[0022] The multivariate anomaly detection submodule calculates the power consumption Z-score and temperature Z-score respectively, and distinguishes between true overload, sensor malfunction, hardware failure, and heat dissipation anomaly based on their joint relationship, thereby determining the power consumption-temperature anomaly category, specifically including: First, store the most recent W sampled values ​​for each node or hardware component, where W is typically between 6 and 60; add the current sampled value at the start of each new sampling period. or Remove the oldest sample, then follow the formula. and Calculate the mean and standard deviation, and then calculate the power consumption Z-score. and temperature Z-score , , ,in, This represents the current power consumption observation. and Let represent the mean and standard deviation of power consumption within the sliding window, respectively. This represents the current temperature observation value. and These represent the mean and standard deviation of the temperature within the sliding window, respectively. Then, based on the power consumption Z-score and temperature Z-score, a multivariate cross-correlation judgment rule is used to determine the current operating state category: when and At this time, the current operating state category is true overload, allowing protective power reduction control to be executed. This represents the abnormal threshold, typically ranging from 2 to 4. when and When the current operating status is classified as sensor malfunction or data error, only an alarm is issued and frequency reduction is not implemented; when and Within the normal range When the current operating status is classified as hardware failure or power failure, alarms and status isolation are executed. when and Within the normal range When the current operating status is classified as abnormal heat dissipation, the priority should be to adjust the BMC fan strategy rather than reduce computing performance. Next, update the exponential decay persistence counter. This is used to accumulate abnormal states to determine whether the abnormality is persistent, and thus decide whether to enter the control and correction phase. The update formula is: ,in, This represents the attenuation factor, with a value ranging from 0 to 1, typically between 0.7 and 0.95. Indicates an indicator function, when or The value is 1 if the condition is met, and 0 otherwise. This indicates the abnormal threshold; the exponentially decaying persistence counter increments by 1 when the abnormality disappears. Exponential decay does not immediately reset to zero, thus effectively capturing intermittent recurring anomalies; when hour( The threshold for persistence (which can be 2 to 5) indicates that the current anomaly is considered persistent, at which point the control and correction phase begins. If the current operating status is classified as sensor malfunction or data error, hardware failure, or power failure, only an alarm will be issued and frequency reduction will not be implemented. If the current operating status is classified as true overload, the exception type will be further subdivided according to the following rules, and then the candidate control parameters will be adjusted based on the type. Obtain response parameters : If the current operating status is classified as true overload, then first classify the anomaly type, and then correct the candidate control parameters: when or More than within a single sampling window Furthermore, when the CPU frequency quickly drops back to the normal range in the next sampling window, the anomaly type is a sudden real overload. The corrective measures are: to temporarily reduce the CPU frequency by one level or reduce the GPU power limit by one preset step, and then restore it after the Z-score drops. In this embodiment, the critical threshold is represented. ; When the mean of the most recent M-step Z-score is greater than And the maximum value was not reached. When the anomaly type is drift overload, the corrective measure is to upgrade the control mode to energy saving priority and reduce the power consumption limit. In this embodiment, the drift detection threshold is represented. ; when At that time, the anomaly type is step true overload, where, This represents the mean of the most recent window. This represents the mean of the previous window. The step decision coefficient is represented in this embodiment. The corrective measure is to persistently reduce the frequency and trigger coordinated quota reallocation. If the current operating status is classified as abnormal heat dissipation, the corrective measure is to increase the fan duty cycle to 70% to 100% according to the temperature risk, and restore automatic fan operation after the temperature drops back to the normal range.

[0023] The prediction-driven collaborative allocation submodule is used to fine-tune the response parameters according to the cluster budget to obtain the final control parameters. Specifically, it calculates the power consumption quota based on the predicted power consumption, current power consumption and comprehensive energy efficiency ratio of each node, and adjusts the response parameters to control parameters in the direction of power reduction when the current power consumption of a node exceeds the corresponding quota. The comprehensive energy efficiency ratio is defined as the ratio of the node performance proxy value to the current power consumption.

[0024] The prediction-driven collaborative allocation submodule calculates power consumption quotas based on the predicted power consumption, current power consumption, and overall energy efficiency ratio of each node. When a node's current power consumption exceeds its corresponding quota, the response parameters are adjusted to control parameters that reduce power consumption. The overall energy efficiency ratio is defined as the ratio of a node's performance proxy value to its current power consumption, specifically including: Let the response parameter of the nth node be denoted as , ,in, This represents the response parameters of the CPU on the nth node. This represents the response parameters of the GPU on the nth node. This represents the response parameters of the fan on the nth node; Get the node energy efficiency ratio of the nth node. , ,in, This represents the overall performance proxy value, which is primarily obtained by weighting indicators such as CPU utilization, GPU utilization, and frequency status collected by PowerAll, with a weighting coefficient of 0.5. If the system incorporates application throughput metrics, then throughput can be included in this proxy value. Represents a node Current power consumption, n=1,2,...,N, where N represents the total number of nodes in the cluster; Obtain the effective power consumption requirement of the nth node , and in When the prior quota reallocation is triggered, Indicates the redistribution trigger coefficient. This represents the cluster budget, which is the maximum total power consumption limit for the cluster set by the administrator or allowed by the power supply capacity. Obtain the unnormalized energy efficiency sensing weights of the nth node: ,in, This represents the energy efficiency preference factor, with a value ranging from 0 to 2; Obtain the normalized energy efficiency sensing weights for the nth node: ; Get the power consumption quota of the nth node And allocate, ,in, This represents the minimum safe power consumption of node n. This represents the maximum allowable power consumption of node n, with the following constraint: and ; Calculate the quota satisfaction rate of the nth node after quota allocation. , And based on quota satisfaction Adjusting the final control parameters : when At that time, the node was severely over-quotaed, and the final control parameters were... The settings include switching the CPU to a power-saving priority frequency adjustment strategy or a user-space frequency adjustment strategy to reduce the frequency, reducing the GPU to near the minimum safe power consumption, and increasing the speed of the BMC fan; when At that time, the node slightly exceeded the quota, and the final control parameters were adjusted. The settings include CPU switching scheduler adaptive frequency adjustment strategy or slight frequency reduction, GPU reducing power consumption limit by excess ratio, and BMC keeping the fan on automatic and increasing the speed when there is a temperature risk. when At the same time, maintain response parameters Alternatively, a performance mode can be enabled, where the CPU uses a performance-priority frequency adjustment strategy or maintains its frequency, the GPU uses maximum or nominal power consumption, and the BMC uses automatic fan.

[0025] If the total quota after constraints If the cluster budget is still exceeded, then the power consumption quota of the nth node will be adjusted. Updated to , Reduce the quota for low-energy-efficiency nodes, making them the priority for power reduction adjustments until the requirements are met. Or achieve the minimum safe power consumption of each node.

[0026] The control front-end module is used to encapsulate the final control parameters into an HTTP / API request, which is sent to the HTTP control endpoint of the target node in JSON format (e.g., {node_id, cpu:{governor,target_freq}, gpu:{power_limit_w}, bmc:{fan_mode,fan_percent}, mode, reason}). The control front-end is only responsible for serialization and sending, and does not participate in decision-making.

[0027] Upon receiving a request, the node proxy module parses and routes the control parameters to the corresponding hardware component object: the CPU component writes `scaling_governor` or `scaling_setspeed` to sysfs, the GPU component sets `power_limit` via NVML, and the BMC component sets the fan mode or duty cycle via REDFISH or IPMI. If execution fails, it returns the reason for the failure and retains the valid control parameters from the previous cycle.

[0028] In this embodiment, the system also includes a visualization module for reading time-series samples and generating a monitoring view, such as... Figure 5 , Figure 6 , Figure 7 and Figure 8 As shown, the visualization module provides a dashboard for overall cluster information, an integrated monitoring and control platform interface, dashboards for node and general hardware information, and a dashboard for GPU hardware information. It can display key information such as total cluster power consumption, node-level CPU / GPU timing metrics, and linked configuration of control parameters. It should be noted that this module is for monitoring, display, and manual confirmation support; it does not participate in the necessary calculation chain for the final control parameters. The automatic closed-loop calculation chain is: time series sample → adaptive prediction → heterogeneous scoring decision → multivariate anomaly detection → prediction-driven collaborative allocation → final control parameters.

[0029] The system supports automatic mode, supervised mode, and manual takeover mode. In automatic mode, the heterogeneous perception intelligent decision engine module automatically generates and distributes the final control parameters. In supervised mode, the final control parameters are confirmed by the user through the visualization module before being distributed by the control front-end module. In manual takeover mode, the user directly sets the control parameters through the control front-end module.

[0030] The method for monitoring and controlling the power consumption of a heterogeneous cluster using the above-mentioned hierarchical architecture includes the following steps: Step 1: Obtain the hardware status data collected by each hardware component object and concatenate them into a time-series index in Prometheus text format. At the start of each acquisition cycle, the node agent module deployed on each computing node uniformly schedules the data update interfaces of its included central processing unit, graphics processing unit, and baseboard management controller components. Each hardware component object reads its own hardware status data, such as CPU frequency / utilization / frequency modulation strategy, GPU power consumption / temperature / utilization / power limit, overall power consumption / fan status / sensor status, through low-level interfaces such as sysfs, NVML, and REDFISH. The node agent module concatenates the above heterogeneous data into time-series indicators in Prometheus text format and makes them available for upstream retrieval through exposed HTTP monitoring endpoints. The node agent module, as a unified entry point, completes the standardized serialization of heterogeneous hardware status.

[0031] Step 2: Perform timestamp alignment and deduplication storage operations on the time series indicators according to the preset collection period to obtain time series samples; The time-series database module actively pulls the HTTP monitoring endpoints exposed in step 1 according to a preset collection period (usually 5-15 seconds) to obtain time-series metrics in Prometheus text format. Then, the pulled raw data undergoes timestamp alignment: samples with the same name and label falling within the same time window are grouped into the same sampling time, using the collection period as the bucket width; if duplicate samples appear within the same bucket, the entry with the latest timestamp is retained, completing deduplication; finally, structured time-series samples are obtained, each sample containing the metric name, label (including at least node ID, component ID, and device ID), sampling timestamp, sample value, and unit; the organization of time-series samples is as follows: Figure 3 As shown, the relationship between Prometheus actively pulling node proxy monitoring endpoints is as follows: Figure 4 As shown, this step provides a high-quality, consistent data foundation for subsequent forecasting and decision-making.

[0032] Step 3-1: Predict the time series samples to obtain prediction results including the predicted power consumption value for the next k steps, the predicted first-order increment, and the predicted second-order gradient. The adaptive prediction submodule in the heterogeneous perception intelligent decision engine module reads the time-series samples generated in step 2 and maintains an independent Holt two-parameter exponential smoothing prediction model for each hardware component object; the model recursively calculates the future based on the horizontal and trend components. step( , Typically, a power consumption prediction value of 3 to 12 is taken. Simultaneously, a parameter self-tuning mechanism driven by normalized prediction error is introduced to dynamically adjust the horizontal smoothing coefficient. and trend smoothing coefficient To adapt to the power consumption variations of different hardware; based on the prediction results, the first-order prediction increment is further calculated. (The magnitude of the increase or decrease in power consumption relative to the current level in the next step) and the predicted second-order gradient (Power consumption acceleration trend); Simultaneous computation The confidence interval of the predicted value is determined. When the upper bound of the confidence interval exceeds the budget threshold, the subsequent decision-making sub-module is triggered in advance to perform operations, realizing the transformation from passive alarm to proactive prevention.

[0033] Step 3-2: Identify the load type based on the hardware status data, and select the corresponding weight configuration based on the identified load type; perform differentiated normalization on different hardware status data, and sum the normalized hardware status data with the predicted first-order increment and predicted second-order gradient according to the weight configuration to obtain a comprehensive score; then form a hysteresis dead zone through preset upgrade threshold and downgrade threshold, and determine the control mode and generate candidate control parameters based on the relationship between the comprehensive score and the hysteresis dead zone. The heterogeneous scoring decision submodule first constructs the load fingerprint vector. Based on preset thresholds, the current load type is categorized into idle, CPU-intensive, GPU-intensive, or hybrid computing types, and a corresponding dynamic weight vector is selected according to the load type. Subsequently, differentiated normalization is performed on different hardware metrics such as CPU utilization, GPU utilization, temperature, and power consumption—CPU utilization and power consumption are linearly normalized, GPU utilization is power-law normalized to enhance sensitivity in high-load ranges, and temperature is normalized using Sigmoid normalization to highlight the risk when approaching thresholds. The normalized metrics are then weighted and summed with the predicted first-order increment and predicted second-order gradient according to the weight configuration corresponding to the load type to obtain a comprehensive score. Next, based on the preset upgrade threshold... and downgrade threshold A hysteresis dead zone is formed. Based on the relative relationship between the comprehensive score and the dead zone, the current control mode is determined in the order of performance → balance → energy saving → emergency, and corresponding candidate control parameters are generated. This step, through load-aware weight adaptation and hysteresis design, fully considers the differences in power consumption response rate and thermal inertia of heterogeneous hardware such as CPU, GPU, and BMC, effectively avoiding frequent oscillations of control parameters near the upgrade / downgrade threshold.

[0034] Step 3-3: Correct the candidate control parameters according to the power consumption-temperature anomaly category to obtain the response parameters; The multivariate anomaly detection submodules calculate the Z-score of the current power consumption. Z-score of temperature Based on the combined relationship between the two, it is divided into four categories of power consumption-temperature anomalies: actual overload, sensor malfunction, hardware failure, and heat dissipation anomaly; the specific judgment rule is: if both power consumption and temperature Z-score exceed the threshold ( If the Z-score is 2-4, it is considered a true overload; if the power consumption Z-score exceeds the threshold while the temperature Z-score is below the threshold... If the power consumption Z-score is extremely low while the temperature is normal, it is judged as a sensor malfunction; if the temperature Z-score exceeds the threshold while the power consumption is normal, it is judged as a heat dissipation malfunction; at the same time, an exponential decay persistence counter is introduced. Only when the anomaly is persistent ( The system enters the control correction phase; when a true overload or abnormal heat dissipation is detected, protective power reduction or fan strategy adjustment measures are taken respectively to obtain response parameters. When a sensor malfunction or hardware failure is detected, only an alarm is triggered without correcting the candidate control parameters, thus avoiding the generation of inappropriate control commands based on erroneous data.

[0035] Steps 3-4: Adjust the response parameters according to the cluster budget to obtain the final control parameters; The prediction-driven collaborative allocation submodule integrates the predicted power consumption of each node. Current power consumption and overall energy efficiency ratio (Defined as the ratio of node performance proxy value to current power consumption) Calculate the power consumption quota for each node. Specifically, first calculate the effective power consumption requirement. When the sum of predicted demand from multiple nodes exceeds the cluster budget trigger threshold, prior quota reallocation is initiated; based on energy efficiency perception weights. Total cluster budget Distributed to each node, resulting in And the total amount does not exceed the cluster budget; based on quota satisfaction. Determine if a node has exceeded its quota: When When the quota is severely exceeded, power consumption is forcibly reduced; when Slight over-quota usage, moderate frequency reduction; when The system maintains or allows performance mode; if the total quota still exceeds the budget after constraints, the quota for low-efficiency nodes is reduced by the inverse of the energy efficiency ratio; the final output includes control parameters such as CPU frequency / power consumption limits, GPU power limits, and BMC fan policies. It enables dynamic power consumption quota management based on prediction and energy efficiency priority within the cluster.

[0036] Step 4: The final control parameters obtained in Steps 3-4 are sent to each computing node, and each computing node routes them to the corresponding hardware component object to execute the control. The control front-end module encapsulates the final control parameters generated in steps 3-4 into an HTTP / API request (JSON format, including fields such as node_id, cpu / governor / target_freq, gpu / power_limit_w, bmc / fan_mode / fan_percent, mode, reason, etc.) and sends it to the HTTP control endpoint of the target node. The node proxy module parses the received HTTP request, routes the control parameters to the corresponding hardware component object, and performs the actual operation through the hardware control interface: the CPU component writes scaling_governor or scaling_setspeed to the sysfs to switch the frequency scaling strategy or set the fixed frequency; the GPU component sets power_limit to adjust the power consumption limit through the NVML interface; and the BMC component sets the fan mode or duty cycle through the REDFISH or IPMI protocol. When the execution fails, the failure reason is returned and the valid control parameters of the previous cycle are maintained to ensure system stability.

[0037] Step 5: After executing the control, return to step 1 to collect hardware status data again and continue execution to form a feedback loop; After completing the control execution in step 4, the system immediately enters the next acquisition cycle, returning to step 1 to re-invoke the data update interface of each hardware component object to acquire the latest hardware status data after the control execution. This data then sequentially goes through the timing storage in step 2, the prediction and decision-making in steps 3-1 to 3-4, and the control issuance in step 4, forming a complete closed-loop chain of "acquisition → prediction → decision-making → execution → feedback" (the closed-loop relationship is as follows...). Figure 4 (As shown by the arrows in the diagram, from the time-series database to the decision engine and then to the control front end). This closed-loop structure enables the cluster power consumption status to be continuously tracked, dynamically evaluated, and adjusted in real time during each decision cycle, achieving the goal of stable, continuous, and adaptive power consumption management under long-term operating conditions.

[0038] Thus, through the iterative process of steps 1 to 5, the system can automatically monitor, predict, make decisions, control, and provide feedback on the power consumption of heterogeneous clusters without human intervention, significantly reducing the operational burden. Furthermore, through a comprehensive mechanism of prediction-driven, heterogeneous perception, and global collaboration, it effectively improves the power consumption control accuracy, response speed, and overall energy efficiency of heterogeneous clusters under dynamic loads.

[0039] To verify the technical effect of the present invention, tests were conducted in the following experimental environment: a heterogeneous computing cluster consisting of two Inspur NF5280M6 dual-socket rack servers (each node is configured with: Intel Xeon Gold 5320 CPU ×4, NVIDIA Tesla A100 80G GPU ×4, BMC supporting REDFISH 1.8 protocol), the total power consumption budget of the cluster was set to 1.8kW, and the time-series database retrieval cycle was 5 seconds.

[0040] In Linpack HPL compute-intensive load testing, existing manual control methods reduced cluster power consumption from approximately 2.1kW to approximately 1.8kW (a reduction of approximately 14%), but floating-point performance decreased from approximately 60 TFlops to approximately 54.5 TFlops (a performance loss of approximately 9%), with a control response latency of 30 to 60 seconds. With the automatic mode of this invention, the control response latency is reduced to 5 to 15 seconds, and the adaptive prediction submodule detects the power consumption increase trend 3 to 10 steps (15 to 50 seconds) in advance. During the power consumption acceleration phase (ΔP'>0), the second-order gradient coupling term enables the scoring decision to trigger power-saving switching approximately 10 to 15 seconds earlier than simply looking at the first-order increment. The hysteresis anti-oscillation mechanism eliminates approximately 40% to 60% of unnecessary control mode switching. Prediction-driven prior quota redistribution allows the cluster to begin adjusting the quotas of each node when power consumption reaches 90% of the budget.

[0041] In vLLM large model inference load testing, existing manual methods control cluster power consumption to approximately 1.8kW, increasing runtime by about 10% (from 53 seconds to 58 seconds). With the system of this invention, load fingerprinting automatically determines the current load as GPU-intensive, automatically switching to a high-weight configuration with high GPU utilization. Power consumption control is prioritized by adjusting GPU power limits, with the expected runtime increase to be kept within 5% to 7%.

[0042] In the anomaly detection test, by injecting artificial anomaly patterns into historical power consumption data, the multivariate cross-correlation analysis was verified to effectively distinguish between real overload and sensor failure, avoiding about 80% of the sensor failure erroneous control operations in the univariate method; the exponential decay persistence counter has a significantly better detection rate for intermittent anomaly patterns than the simple binary counter.

Claims

1. A hierarchical architecture heterogeneous cluster power consumption monitoring and control system, characterized in that... It includes a node agent module, a time-series database module, a heterogeneous perception intelligent decision engine module, and a control front-end module; The node proxy module is deployed in each computing node of the heterogeneous cluster and includes multiple hardware component objects. The node proxy module is used to uniformly schedule the data update interface of each hardware component object and concatenate the hardware status data returned by each hardware component object into a time-series indicator in Prometheus text format, while exposing HTTP monitoring endpoints and HTTP control endpoints. The time-series database module is used to actively pull time-series indicators exposed by the HTTP monitoring endpoint according to a preset collection period, and perform timestamp alignment and deduplication storage operations on the time-series indicators in sequence to obtain time-series samples. The heterogeneous perception intelligent decision engine module includes an adaptive prediction submodule, a heterogeneous scoring decision submodule, a multivariate anomaly detection submodule, and a prediction-driven collaborative allocation submodule. The adaptive prediction submodule is used to predict time series samples to obtain prediction results including the predicted power consumption value for the next k steps, the predicted first-order increment, and the predicted second-order gradient. The heterogeneous scoring decision submodule is used to identify the load type based on hardware status data and select the corresponding weight configuration according to the identified load type; perform differentiated normalization on different hardware status data, and perform weighted summation of the normalized hardware status data with the predicted first-order increment and predicted second-order gradient according to the weight configuration to obtain a comprehensive score; then form a hysteresis dead zone through preset upgrade threshold and downgrade threshold, and determine the control mode and generate candidate control parameters based on the relationship between the comprehensive score and the hysteresis dead zone; The aforementioned multivariate anomaly detection submodule is used to correct candidate control parameters based on power consumption-temperature anomaly categories to obtain response parameters; The prediction-driven collaborative allocation submodule is used to fine-tune the response parameters according to the cluster budget to obtain the final control parameters; The control front-end module is used to encapsulate the final control parameters into an HTTP / API request and send it to the HTTP control endpoint of the target node; The node proxy module is also used to parse HTTP requests received through the HTTP control endpoint, route the final control parameters to the corresponding hardware component object, and execute control through the hardware control interface.

2. The hierarchical architecture heterogeneous cluster power consumption monitoring and control system according to claim 1, characterized in that... The hardware component objects include at least a central processing unit (CPU) component, a graphics processing unit (GPU) component, and a baseboard management controller (BMC) component in the cluster; wherein, the CPU component is used to collect hardware status data including CPU frequency, CPU utilization, and frequency modulation strategy; the GPU component is used to collect hardware status data including GPU power consumption, temperature, GPU utilization, and power consumption limits; and the BMC component is used to collect hardware status data including overall power consumption, fan status, and sensor status.

3. The hierarchical architecture heterogeneous cluster power consumption monitoring and control system according to claim 1, characterized in that... The specific operation of the adaptive prediction submodule is as follows: For each hardware component object, maintain an independent Holt two-parameter exponential smoothing prediction model, and recursively calculate the power consumption prediction value for the next k steps according to the horizontal component and the trend component. Obtain the normalized prediction error and perform parameter self-tuning updates on the horizontal smoothing coefficient and the trend smoothing coefficient; Obtain the first-order predicted increment and the second-order predicted gradient, where the first-order predicted increment is the difference between the predicted power consumption value in the next step and the current power consumption observation value, and the second-order predicted gradient is the difference between the current first-order predicted increment and the previous first-order predicted increment. Calculate the confidence interval of the k-step power consumption prediction value. When the upper bound of the confidence interval exceeds the budget threshold, trigger the heterogeneous scoring decision submodule or the prediction-driven collaborative allocation submodule to make a decision in advance.

4. A hierarchical architecture heterogeneous cluster power consumption monitoring and control system according to claim 1, characterized in that... In the heterogeneous scoring decision submodule, the upgrade threshold is greater than the downgrade threshold. When the overall score exceeds the upgrade threshold, the control mode switches to a more conservative mode and generates more aggressive power-reduction candidate control parameters. When the overall score is lower than the downgrade threshold, the control mode switches to a more aggressive mode, allowing performance improvement. When the overall score is between the two, the current control mode is maintained.

5. A hierarchical architecture heterogeneous cluster power consumption monitoring and control system according to claim 1, characterized in that... The multi-dimensional anomaly detection submodule calculates the power consumption Z-score and temperature Z-score respectively, and distinguishes between real overload, sensor anomaly, hardware failure and heat dissipation anomaly based on the joint relationship between the two, thereby determining the power consumption-temperature anomaly category; wherein, when it is determined to be a real overload or heat dissipation anomaly, protective power reduction or fan strategy adjustment is performed on the candidate control parameters; when it is determined to be a sensor anomaly or hardware failure, only an alarm is triggered without modifying the candidate control parameters.

6. A hierarchical architecture heterogeneous cluster power consumption monitoring and control system according to claim 1, characterized in that... The prediction-driven collaborative allocation submodule calculates the power consumption quota based on the predicted power consumption, current power consumption, and comprehensive energy efficiency ratio of each node. When the current power consumption of a node exceeds the corresponding quota, the response parameter is adjusted to a control parameter for power reduction. The comprehensive energy efficiency ratio is defined as the ratio of the node's performance proxy value to its current power consumption.

7. A hierarchical architecture heterogeneous cluster power consumption monitoring and control system according to claim 1, characterized in that... It also includes a visualization module, which is used to read time-series samples and form a monitoring view; The system supports automatic mode, supervised mode, and manual takeover mode. In automatic mode, the heterogeneous perception intelligent decision engine module automatically generates and distributes the final control parameters. In supervised mode, the final control parameters are confirmed by the user through the visualization module and then distributed by the control front-end module. In manual takeover mode, the user directly sets the control parameters through the control front-end module.

8. A method for monitoring and controlling the power consumption of a heterogeneous cluster using a hierarchical architecture as described in claim 1, characterized in that... Includes the following steps: Step 1: Obtain the hardware status data collected by each hardware component object and concatenate them into a time-series index in Prometheus text format. Step 2: Perform timestamp alignment and deduplication storage operations on the time series indicators according to the preset collection period to obtain time series samples; Step 3-1: Predict the time series samples to obtain prediction results including the predicted power consumption value for the next k steps, the predicted first-order increment, and the predicted second-order gradient. Step 3-2: Identify the load type based on the hardware status data, and select the corresponding weight configuration based on the identified load type; perform differentiated normalization on different hardware status data, and sum the normalized hardware status data with the predicted first-order increment and predicted second-order gradient according to the weight configuration to obtain a comprehensive score; then form a hysteresis dead zone through preset upgrade threshold and downgrade threshold, and determine the control mode and generate candidate control parameters based on the relationship between the comprehensive score and the hysteresis dead zone. Step 3-3: Correct the candidate control parameters according to the power consumption-temperature anomaly category to obtain the response parameters; Steps 3-4: Adjust the response parameters according to the cluster budget to obtain the final control parameters; Step 4: The final control parameters obtained in Steps 3-4 are sent to each computing node, and each computing node routes them to the corresponding hardware component object to execute the control. Step 5: After executing the control, return to step 1 to collect hardware status data again and continue execution, forming a feedback loop.

Images