A method and system for detecting and processing energy efficiency data in a data center

Abstract

This invention discloses a method and system for detecting and processing energy efficiency data in data centers, comprising: deploying sensors and a distributed acquisition network to collect energy efficiency data through energy efficiency change rate calculation and adaptive sampling frequency adjustment; employing adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling, and multi-level prediction fusion to achieve dynamic energy efficiency prediction; introducing energy efficiency-aware load assessment indicators to maximize energy efficiency; constructing an intelligent group decision-making model to achieve global allocation of computing resources through dynamic trade-off scheduling optimization, task allocation stability analysis, and adaptive convergence adjustment; and based on an energy efficiency autonomous learning framework, using imitation learning for initial policy training, reinforcement learning to optimize scheduling strategies, and adaptive tuning and dynamic adjustment to achieve adaptive optimization. This invention can accurately, quickly, and dynamically adjust the allocation of computing resources to achieve optimal energy efficiency in the face of rapidly changing data center business conditions.

Description

Technical Field

[0001] This invention relates to the field of data detection technology, and in particular to a method and system for detecting and processing energy efficiency data in a data center. Background Technology

[0002] With the rapid development of technologies such as cloud computing, big data, and artificial intelligence, data centers have become a crucial infrastructure supporting the global digital economy. However, the energy consumption of data centers is becoming increasingly prominent, accounting for a significant proportion of global electricity consumption and directly impacting operating costs and environmental sustainability. Therefore, improving the energy efficiency of data centers has become one of the key challenges in current research and engineering practice.

[0003] Data center energy efficiency management involves multiple layers, including server energy consumption optimization, network device power consumption management, cooling system optimization, and load balancing and resource scheduling. Among these, load balancing and resource scheduling play a crucial role in energy efficiency optimization, determining not only the utilization rate of computing resources but also influencing overall energy consumption and system performance. An ideal load balancing and resource scheduling solution should be able to dynamically adapt to changes in business needs, achieving the lowest possible energy consumption while ensuring performance. However, current load balancing and resource scheduling methods generally face numerous challenges.

[0004] Traditional load balancing strategies often rely on fixed rules or historical statistics, making it difficult to adapt to the dynamic environment of data centers in real time. This leads to problems such as uneven distribution of computing resources, server overload, or wasted idle resources. Meanwhile, existing resource scheduling algorithms are mainly based on static optimization or heuristic methods, which suffer from slow response times and insufficient scheduling accuracy when handling complex business loads. They struggle to effectively cope with sudden surges in business traffic, dynamic energy consumption changes, and resource contention in multi-tenant environments. Furthermore, the energy consumption characteristics of servers, storage devices, and network nodes within a data center vary. How to achieve the optimal balance between energy efficiency and performance—while meeting Quality of Service (QoS) requirements—by comprehensively considering computing resource utilization, power consumption characteristics, and cooling energy consumption remains a pressing technical challenge.

[0005] Therefore, in order to adapt to the rapidly changing business needs of data centers, improve resource utilization, and reduce overall energy consumption, there is an urgent need to develop an adaptive load balancing and resource scheduling strategy that can dynamically allocate resources based on real-time business load and energy consumption data, thereby optimizing energy efficiency while ensuring computing performance. Summary of the Invention

[0006] To address the above problems, this invention provides a method and system for detecting and processing energy efficiency data in data centers.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] On one hand, this invention discloses a method for detecting and processing energy efficiency data in a data center, comprising:

[0009] Step 1: Deploy sensors and a distributed acquisition network, and achieve energy efficiency data acquisition by calculating the rate of change of energy efficiency and adjusting the adaptive sampling frequency, combined with edge computing preprocessing;

[0010] Step 2: Adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling, and multi-level prediction fusion are used to achieve dynamic energy efficiency prediction;

[0011] Step 3: Introduce energy efficiency-aware load assessment metrics, and combine them with task migration decisions, dynamic weight adjustments, and global load balancing optimization to maximize energy efficiency;

[0012] Step 4: Construct an intelligent group decision-making model, and realize the global allocation of computing resources through dynamic trade-off scheduling optimization, task allocation stability analysis and adaptive convergence adjustment;

[0013] Step 5: Based on the energy efficiency autonomous learning framework, use imitation learning to train the initial strategy, reinforcement learning to optimize the scheduling strategy, adaptive tuning and dynamic adjustment to achieve adaptive optimization.

[0014] Furthermore: Step 1 includes:

[0015] Deploy energy efficiency sensors in critical data center equipment to build a distributed data acquisition network, periodically collect power, load, and timestamp data from the equipment, and send them to edge nodes;

[0016] Calculate the rate of change in energy efficiency and determine whether the energy consumption status of the data center fluctuates drastically by setting a threshold.

[0017] The sampling frequency is dynamically adjusted based on the rate of change of energy efficiency, the sampling interval is controlled by a nonlinear adjustment factor, and the sampling period is smoothed by an exponential weighted moving average to ensure that the sampling period adapts to load changes.

[0018] Data preprocessing is performed at edge nodes, outlier data is filtered and removed using local anomaly factors, and data is smoothed using sliding window mean filtering. Data is only uploaded to the central server after the confidence level is met, thereby reducing communication and computing overhead.

[0019] Furthermore: Step 2 includes:

[0020] The size of the forecast time window is dynamically adjusted according to the rate of change in energy efficiency, and the impact of short-term fluctuations on the window is suppressed by the exponential smoothing method to ensure that the window size adapts to the fluctuation of energy efficiency.

[0021] Short-term and long-term error feedback mechanisms are introduced to dynamically adjust the parameters of the prediction model. Short-term errors are used for fine-tuning the model parameters, while long-term errors are used to correct trend predictions.

[0022] We construct a feature vector based on the rate of change of energy efficiency, use a nonlinear autoregressive model for trend modeling, and introduce a periodic factor to adjust the model output to adapt to the periodic characteristics of data center energy consumption.

[0023] By integrating short-term and long-term forecast results and dynamically adjusting the weights, the final forecast output is optimized based on short-term and long-term errors, thereby achieving dynamic forecasting of energy efficiency data.

[0024] Furthermore: Step 3 includes:

[0025] Define energy efficiency-aware load assessment metrics, which integrate the CPU / GPU utilization of computing nodes, current power consumption, and task latency. Standardize each metric through a normalization method to achieve load measurement.

[0026] Based on energy efficiency optimization, the task migration decision is calculated, the global energy efficiency optimization objective function is calculated, the principle of minimum incremental energy consumption is followed when selecting migration tasks, and constraints are set to ensure that the target node is not overloaded.

[0027] Dynamic weight adaptive adjustment: The weight parameters are dynamically updated based on historical error feedback. When the system load is unbalanced, the load balancing weight is increased, and when energy consumption exceeds the standard, the energy consumption optimization weight is increased.

[0028] Global load balancing optimization employs gradient descent to adjust task allocation strategies, minimizing load variance and ensuring optimal overall energy efficiency for the data center.

[0029] Furthermore, step 4 includes:

[0030] A smart group decision-making model is constructed, consisting of multiple smart agents. Each agent makes resource scheduling decisions based on local observations and global feedback. The goal is to minimize the total energy consumption of the system and maximize the overall computational efficiency.

[0031] Implement scheduling optimization based on dynamic trade-offs, define the objective function for resource scheduling optimization, introduce a global energy efficiency feedback adjustment strategy, and achieve resource competition and global optimal scheduling through the game mechanism among intelligent agents;

[0032] Perform task allocation stability analysis, calculate task migration stability index, and if it exceeds the threshold, use gradient descent method to adjust task scheduling to enhance stability.

[0033] The algorithm performs adaptive convergence adjustment, dynamically calculates the convergence speed of the global energy efficiency target, and adjusts the decision step size of the intelligent agent according to the convergence situation to ensure that the algorithm is highly adaptable under different load conditions and realizes intelligent scheduling and optimization of data center computing resources.

[0034] Furthermore: Step 5 includes:

[0035] An energy efficiency autonomous learning framework is constructed, defining the state space of the data center, including business load, energy consumption, resource scheduling and overall energy efficiency targets, and adopting a strategy based on deep reinforcement learning to generate a network for adaptive optimization;

[0036] Imitation learning is used for initial policy training, and supervised learning methods are used to adjust the policy network parameters in combination with historical best scheduling data to improve the policy accuracy in the initial stage.

[0037] Reinforcement learning is used for autonomous optimization, the policy gradient method is used to dynamically adjust the policy network, and the experience replay mechanism is combined to improve the policy convergence speed and ensure that the energy efficiency scheduling policy adapts to real-time environmental changes.

[0038] Implement adaptive tuning and dynamic energy efficiency adjustment, dynamically adjust the learning rate based on target deviation, adapt to changes in business load in real time, and achieve long-term self-evolution optimization of the data center.

[0039] On the other hand, this invention discloses an energy efficiency data detection and processing system for a data center, comprising:

[0040] Dynamic self-calibration energy efficiency data acquisition module: Deploy sensors and a distributed acquisition network, and realize energy efficiency data acquisition by calculating the rate of change of energy efficiency and adjusting the adaptive sampling frequency, combined with edge computing preprocessing;

[0041] Dynamic prediction module: It adopts adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling and multi-level prediction fusion to realize dynamic energy efficiency prediction;

[0042] Dynamic load balancing module: Introduces energy-efficiency-aware load assessment metrics, combined with task migration decisions, dynamic weight adjustments, and global load balancing optimization, to maximize energy efficiency;

[0043] Resource scheduling optimization module: Constructs an intelligent group decision-making model to achieve global allocation of computing resources through dynamic trade-off scheduling optimization, task allocation stability analysis, and adaptive convergence adjustment;

[0044] Adaptive optimization module: Based on the energy efficiency autonomous learning framework, it uses imitation learning for initial policy training, reinforcement learning to optimize scheduling strategies, and adaptive tuning and dynamic adjustment to achieve adaptive optimization.

[0045] The technological advancements achieved by this invention compared to existing technologies are as follows:

[0046] Traditional data centers suffer from noise interference and inaccurate equipment energy consumption models during energy efficiency data collection, making it difficult to achieve ideal results in subsequent optimization. This invention adopts a dynamic self-calibration mechanism, combined with time series analysis, outlier detection, and adaptive calibration technology, to achieve high-precision energy efficiency data collection. This provides reliable energy efficiency data support for load balancing and resource scheduling, fundamentally improving the accuracy of decision-making.

[0047] Existing load balancing and resource scheduling algorithms often rely on historical statistical data when making adjustments, which cannot accurately predict future load and energy consumption trends, resulting in slow response speeds. The dynamic prediction based on energy efficiency feedback proposed in this invention uses an adaptive attention mechanism and multimodal fusion learning to combine historical business load, temperature, power consumption, and other data to achieve high-precision prediction of energy efficiency trends. This enables the system to perceive load changes in advance, optimize resource scheduling proactively, and thus improve scheduling efficiency and response speed.

[0048] Existing load balancing algorithms struggle to adapt to complex energy efficiency requirements, resulting in low utilization of computing resources and excessive server energy consumption. This invention presents a dynamic load balancing algorithm with an energy efficiency trade-off. By constructing an energy efficiency optimization objective function, it balances performance requirements with power consumption overhead and dynamically adjusts task allocation based on reinforcement learning, enabling computing nodes to maximize energy efficiency gains while ensuring Quality of Service (QoS).

[0049] Traditional resource scheduling strategies struggle to adapt to sudden load changes in data centers, often employing static or heuristic methods and lacking global optimization capabilities. This invention presents an intelligent swarm decision-making resource scheduling optimization algorithm. By introducing multi-agent reinforcement learning and game theory-driven intelligent swarm decision-making, multiple computing nodes can collaboratively adjust resource allocation, dynamically optimizing scheduling strategies under sudden load conditions, improving computing resource utilization, and reducing power waste.

[0050] Existing energy efficiency optimization methods typically rely on manually set thresholds and rules, making it difficult to adapt to environmental changes during long-term operation. This invention introduces an energy efficiency autonomous learning optimization mechanism, combining reinforcement learning and imitation learning, enabling the system to continuously learn historical best strategies during operation and adaptively adjust energy efficiency optimization schemes according to current business needs, achieving long-term self-optimization of the data center.

[0051] Traditional energy efficiency optimization methods often focus excessively on reducing energy consumption while neglecting system performance, leading to a decline in user experience. This invention comprehensively considers the trade-off between energy efficiency and computing performance at every stage, ensuring that business quality is not affected while optimizing energy consumption, and ultimately achieving a dynamic balance between optimal energy efficiency and optimal performance.

[0052] In summary, this invention innovates on multiple levels, including data acquisition, load prediction, resource scheduling, and autonomous optimization, overcoming the bottlenecks of traditional load balancing and resource scheduling in dynamic adjustments, such as slow response speed and insufficient scheduling accuracy. Through innovative technologies such as real-time high-precision energy efficiency data acquisition, intelligent prediction, reinforcement learning optimization, intelligent group decision-making, and autonomous learning, it achieves intelligent, automated, and adaptive energy efficiency management for data centers. Ultimately, this invention can accurately, quickly, and dynamically adjust computing resource allocation in the face of rapidly changing data center business, achieving an optimal balance between energy efficiency and performance, significantly reducing overall data center energy consumption, and improving operational efficiency. Attached Figure Description

[0053] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0054] In the attached diagram:

[0055] Figure 1 This is a flowchart of the present invention;

[0056] Figure 2 This is a system structure diagram of the present invention. Detailed Implementation

[0057] The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0058] Example 1

[0059] like Figure 1 As shown, the present invention discloses a method for detecting and processing energy efficiency data in a data center, including:

[0060] Specifically, step 1 includes:

[0061] To address the trade-off between data lag and computational overhead in traditional energy efficiency data acquisition methods, this step employs a dynamic self-calibration mechanism. This mechanism adjusts the sampling frequency in real-time based on energy efficiency fluctuations to ensure data accuracy and timeliness while reducing unnecessary computational costs. Dynamic self-calibration energy efficiency data acquisition mainly includes four key sub-steps: sensor data acquisition, energy efficiency change rate calculation, adaptive sampling frequency adjustment, and edge preprocessing. The detailed implementation process is described below.

[0062] Sub-step 1.1: Sensor data acquisition

[0063] High-precision energy efficiency sensors are deployed at key locations in data center servers, storage devices, network switches, and cooling systems, and a distributed data acquisition network is built to collect energy efficiency data from different types of equipment.

[0064] Define the device set S = {s1, s2, ..., s} n}, where s i This represents the i-th device in the data center.

[0065] For each device i Deploy an energy efficiency sensor P i The sensor periodically measures the power consumption of the device. and running load And record the timestamp t.

[0066] Define the collected data vector for:

[0067]

[0068] in, This represents the power consumption data at the current moment. This is the current CPU load data. This is timestamp information.

[0069] The data acquisition module is responsible for periodically reading sensor data and sending the data to the local edge node for real-time preprocessing.

[0070] Sub-step 1.2: Calculation of energy efficiency change rate

[0071] To determine whether the energy efficiency status of the current data center has changed drastically, it is necessary to calculate the rate of change of energy efficiency data, thereby providing a basis for subsequent adaptive sampling frequency adjustments.

[0072] Set the time window size to Δt, and calculate the rate of change of energy efficiency within two adjacent time windows.

[0073]

[0074] in, This represents the energy consumption value at the current time t. This represents the energy consumption value for the previous time window.

[0075] Calculate the overall global energy efficiency change rate R of the data center. t As a global adjustment parameter:

[0076]

[0077] Where N is the total number of devices in the data center.

[0078] By setting a threshold for the rate of change R th When R t >R th When R indicates that the data center's energy consumption is fluctuating drastically, it is necessary to increase the sampling frequency; when R t <R th In such cases, the sampling frequency can be reduced to decrease data transmission and computational burden.

[0079] Sub-step 1.3: Adaptive sampling frequency adjustment

[0080] Based on the calculated rate of change in energy efficiency, the sampling frequency is dynamically adjusted so that the sampling period can be automatically optimized according to the load changes of the data center, thereby improving the timeliness and accuracy of the data.

[0081] Set the initial sampling interval T0 (i.e., the default data sampling period in the initial state of the system).

[0082] The sampling period is dynamically adjusted using a nonlinear adjustment factor α, and the sampling interval T at the next time step is calculated. t :

[0083]

[0084] Where α is an adaptive adjustment coefficient, which ensures that the sampling interval is shortened when the data fluctuates drastically and increased when the data is stable.

[0085] Set upper and lower limits T for sampling interval min and T max To ensure that the sampling period does not exceed a reasonable range:

[0086] T t =max(T) min ,min(T max ,T t ))

[0087] Historical data is used to smooth the sampling interval using an exponentially weighted moving average to avoid sharp fluctuations.

[0088] T t =λT t +(1-λ)T t-1

[0089] Wherein, λ is the smoothing coefficient, used to control the degree of influence of historical data on the current adjustment.

[0090] Sub-step 1.4: Edge computing preprocessing

[0091] Since data center energy efficiency data collection involves a large amount of sensor data, directly transmitting it to the central server would cause significant computing and communication overhead. Therefore, data preprocessing is required on edge devices to reduce the computing burden on the main server.

[0092] A data anomaly detection module is deployed on the edge computing node, and an algorithm based on the Local Outlier Factor (LOF) is used to filter out abnormal data and remove extreme values.

[0093] The outlier score for each data point is calculated using the outlier detection formula:

[0094]

[0095] in, Let k be the local reachability density, and k be the number of neighboring points.

[0096] Set an exception threshold LOF th ,when At that time, the data point was marked as an anomaly and discarded.

[0097] A sliding window-based mean filter is used to smooth the data in order to reduce the impact of noise.

[0098]

[0099] Where W is the size of the sliding window. This is the smoothed energy efficiency data.

[0100] Processed data Uploads are only sent to the central server after a certain level of confidence is met, thereby reducing invalid data transmission and improving system response speed.

[0101] Dynamic self-calibration energy efficiency data acquisition achieves efficient and accurate data collection of data centers through four sub-steps: sensor data acquisition, energy efficiency change rate calculation, adaptive sampling frequency adjustment, and edge computing preprocessing. This ensures the timeliness of the data and optimizes computing costs, while providing high-quality input data for subsequent energy efficiency prediction, load balancing, and resource scheduling.

[0102] Specifically, step 2 includes:

[0103] To fully utilize the high-precision energy efficiency data collected in Step 1 and address the low accuracy and insufficient adaptability of existing energy consumption prediction models due to fixed time windows and static parameter adjustments, this step designs a dynamic prediction model based on energy efficiency feedback. This model combines energy efficiency change rate, short-term and long-term error feedback, and a dynamic window adjustment strategy to achieve high-precision prediction of data center energy efficiency. The dynamic prediction based on energy efficiency feedback mainly includes four core sub-steps: adaptive time window adjustment, feedback-based error correction, energy efficiency trend modeling, and multi-level prediction fusion. The detailed implementation process is as follows.

[0104] Sub-step 2.1: Adaptive time window adjustment

[0105] Since the energy efficiency fluctuations in data centers are not uniform, to avoid the loss of critical information during periods of severe fluctuation or overcomputation during periods of stable conditions using fixed-window forecasting methods, it is necessary to base forecasting on the energy efficiency change rate R. t The prediction window size is dynamically adjusted to ensure the optimal time span selection.

[0106] The initial time window size W0 is usually set to the default number of time steps, such as 10 time steps.

[0107] Calculate the adaptive time window W t Its rate of change with energy efficiency R t Dynamic adjustment:

[0108] W t =W0+γR t

[0109] Wherein, γ is an adjustment coefficient used to control the range of window changes, so that the window size increases as energy efficiency fluctuations increase, in order to adapt to drastic changes.

[0110] Set upper and lower limits for window size W min and W max Ensure the window is neither too small nor too large:

[0111] W t =max(W min ,min(W max W t ))

[0112] Exponential smoothing is used to smooth the window size to avoid drastic fluctuations in window size.

[0113] W t =λW t +(1-λ)W t-1

[0114] Where λ is the smoothing coefficient, used to suppress the impact of short-term data fluctuations on the window size.

[0115] Sub-step 2.2: Feedback-based error correction

[0116] Traditional prediction models use fixed parameters during training, but data center energy efficiency is affected by complex factors, and a single static model cannot adapt to dynamic changes. Therefore, an error correction mechanism based on energy efficiency feedback is introduced to improve prediction accuracy through tiered adjustments of short-term and long-term errors.

[0117] Calculate the short-term error ΔS t Used to dynamically adjust prediction model parameters:

[0118]

[0119] Among them, E t This represents the actual energy consumption value. This is the predicted value from the previous moment.

[0120] Calculate the long-term error ΔL t Used to correct trend forecasts:

[0121]

[0122] Where N is the long-term error calculation window.

[0123] The prediction model parameter set Θ is dynamically adjusted, and corrections are made based on both short-term and long-term errors.

[0124] Θ t =Θ t-1 +η1ΔS t +η2ΔL t

[0125] Where η1 and η2 are adjustment coefficients used to control the impact of short-term and long-term errors on model parameters.

[0126] Sub-step 2.3: Energy efficiency trend modeling

[0127] To improve the adaptability of prediction models to energy efficiency change trends, it is necessary to model the energy efficiency changes and build a time series prediction model by combining historical data.

[0128] Construct a feature vector based on the rate of change of energy efficiency:

[0129]

[0130] Among them, W t For adaptive time windows, R t This represents the current rate of change in energy efficiency.

[0131] A nonlinear autoregressive model is used for trend modeling, and its prediction formula is as follows:

[0132]

[0133] Where f(·) is a nonlinear mapping function, Θ t These are the model parameters optimized based on error feedback.

[0134] A periodic factor is introduced to adjust the model output, taking into account the periodic characteristics of data center energy consumption:

[0135]

[0136] Where ω is the periodicity influence coefficient and T is the period length.

[0137] Sub-step 2.4: Multi-level prediction fusion

[0138] Since data center energy consumption is affected by a variety of factors, a single forecasting method is difficult to adapt to all situations. Therefore, a method that combines short-term and long-term forecasts is adopted to improve the overall forecasting accuracy.

[0139] Constructing a short-term forecasting model F s Used for rapid response to energy efficiency fluctuations:

[0140]

[0141] Using LSTM (Long Short-Term Memory) network for short-term energy consumption prediction is suitable for handling sudden fluctuations.

[0142] Constructing a long-term prediction model F l Used to capture overall energy efficiency trends:

[0143]

[0144] The autoregressive integral moving average model is used for long-term energy consumption trend prediction and is suitable for capturing periodic changes.

[0145] By combining short-term and long-term forecast results, a weighted method is used to produce the final forecast output:

[0146]

[0147] Among them, w s and w l To dynamically adjust the weights, optimization is performed based on short-term and long-term errors:

[0148]

[0149]

[0150] To achieve adaptive optimization, long-term predictions should be prioritized when short-term errors are large, and short-term predictions should be prioritized when long-term errors are large.

[0151] Dynamic prediction based on energy efficiency feedback achieves high-precision dynamic prediction of energy efficiency data through four key sub-steps: adaptive time window adjustment, feedback-based error correction, energy efficiency trend modeling, and multi-level prediction fusion. This invention can improve prediction accuracy while reducing computational costs when data center energy consumption changes rapidly, and provides high-quality prediction data input for subsequent load balancing and resource scheduling.

[0152] Specifically, step 3 includes:

[0153] To fully utilize the high-precision energy efficiency data collected in Step 1 and the predicted future energy consumption trends in Step 2, and to address the issues of slow response speed and insufficient energy efficiency optimization in existing load balancing strategies during dynamic adjustment, this step designs a dynamic load balancing algorithm based on energy efficiency trade-offs. This algorithm combines real-time energy consumption data, predicted energy consumption trends, and computing resource utilization to optimize overall energy consumption while ensuring service performance, achieving efficient allocation of data center computing resources. The energy efficiency-trade-based dynamic load balancing mainly includes four core sub-steps: energy efficiency-aware load assessment, energy efficiency-optimized task migration decision-making, dynamic weight adaptive adjustment, and global load balancing optimization. The detailed implementation process is as follows.

[0154] Sub-step 3.1: Energy efficiency-aware load assessment

[0155] Traditional load balancing methods typically rely solely on CPU utilization or task queue length for load assessment, neglecting energy efficiency optimization goals. Therefore, an energy efficiency-aware load assessment metric is introduced. It achieves accurate load measurement by comprehensively calculating resource usage and energy consumption factors.

[0156] Define the energy efficiency-aware load of computing node i at time t. for:

[0157]

[0158] in:

[0159] The CPU / GPU utilization of compute node i reflects the computing resource load.

[0160] The current power consumption of node i is calculated using the energy efficiency data collected in step 1;

[0161] To calculate the task waiting time at node i and measure the congestion level of the task queue;

[0162] α, β, and γ are adjustable weighting coefficients to balance the influence of different factors.

[0163] Normalization methods are used to standardize indicators across different dimensions to eliminate dimensional differences:

[0164]

[0165] Where X is any load index (e.g., ... ), X max and X min These represent the maximum and minimum values ​​of historical data, respectively.

[0166] Sub-step 3.2: Task migration decision based on energy efficiency optimization

[0167] When the load of a certain computing node Exceeding the set threshold L max When this happens, some of the tasks need to be migrated to reduce overall energy consumption and balance the computing load. To this end, a task migration decision model is introduced to achieve dynamic task scheduling.

[0168] Calculate the global energy efficiency optimization objective function for the current system:

[0169]

[0170] Where N is the total number of computing nodes in the data center, and ω1 and ω2 are the weights for adjusting energy efficiency and load balancing.

[0171] When selecting migration tasks, a task selection strategy based on the principle of minimum incremental energy consumption is adopted:

[0172]

[0173] Where j is the task to be migrated. The power consumption of the current node. The power consumption of the target migration node after the task migration.

[0174] Set task migration constraints to ensure that the migration task does not cause the target node to become overloaded:

[0175]

[0176] Where k is the set of all possible target nodes.

[0177] Sub-step 3.3: Dynamic weight adaptive adjustment

[0178] Since data center load changes dynamically over time, load balancing algorithms need to adaptively adjust energy efficiency weights to flexibly optimize performance and energy consumption under different business needs. To this end, a weight adaptive adjustment strategy based on feedback regulation is designed.

[0179] Calculate historical error feedback to dynamically adjust weights:

[0180]

[0181] in, and η represents the average values ​​of the global load balancing target and the energy consumption target, respectively, and η is the learning rate.

[0182] Update weight parameters to ensure the system dynamically adapts to load changes:

[0183]

[0184] When the system load is unbalanced, ω1 increases to increase the target weight of load balancing;

[0185] When the system energy consumption exceeds expectations, ω2 increases to strengthen the energy consumption optimization target.

[0186] Sub-step 3.4: Global load balancing optimization

[0187] After the task migration decision is made, global load optimization adjustments are needed to ensure optimal global load balancing in the data center.

[0188] Define a global load balancing optimization objective function to minimize load variance:

[0189]

[0190] in, This represents the average load across all compute nodes.

[0191] A gradient descent-based method is used for load balancing optimization iterations to adjust the task allocation strategy.

[0192]

[0193] Where η is the learning rate. The gradient of the global optimization target with respect to the load.

[0194] Perform final load adjustments to ensure optimal overall energy efficiency for the data center:

[0195] After the task migration is complete, the global load balancing status is recalculated;

[0196] If the global load variance is still higher than the threshold, the task allocation is iteratively adjusted until a stable state is reached.

[0197] Dynamic load balancing with energy efficiency trade-offs achieves dynamic load balancing in data centers through four key sub-steps: energy efficiency-aware load assessment, energy efficiency-optimized task migration decision-making, dynamic weight adaptive adjustment, and global load balancing optimization. This algorithm maximizes energy efficiency while meeting business needs, providing reliable support for subsequent resource scheduling strategies.

[0198] Specifically, step 4 includes:

[0199] Step 1 involves accurately collecting energy efficiency data through a dynamic self-calibration mechanism. Step 2 builds a dynamic prediction model based on an energy efficiency feedback mechanism. Step 3 utilizes an energy efficiency trade-off method to achieve dynamic load balancing. Step 4 further designs a resource scheduling optimization algorithm based on intelligent group decision-making to dynamically optimize the allocation of computing resources globally, enabling the data center to achieve optimal scheduling decisions under different business loads and energy efficiency constraints. This algorithm comprehensively considers historical energy consumption trends, real-time load changes, and future business needs, and introduces an intelligent group decision-making model, enabling multiple intelligent agents to make collaborative decisions to achieve optimal resource utilization and energy efficiency ratio. Its core includes four key sub-steps: intelligent group decision-making modeling, scheduling optimization based on dynamic trade-offs, task allocation stability analysis, and adaptive convergence adjustment. The detailed implementation process is as follows.

[0200] Sub-step 4.1: Intelligent group decision-making modeling

[0201] To construct an efficient resource scheduling decision-making mechanism, an intelligent swarm decision-making model is introduced. This model consists of multiple autonomous intelligent agents (IAs), each of which makes a joint decision on the resource scheduling scheme based on its local information and global feedback. The core definition of the intelligent swarm decision-making model is as follows:

[0202] Define a set of intelligent agents: Suppose a data center contains M computing nodes, and each node i is controlled by an intelligent agent IA. i The decision variable for management is a resource allocation vector:

[0203]

[0204] in, Indicates computing resources (CPU / GPU), Indicates memory resources, This indicates storage resources.

[0205] The goal of intelligent agents is to optimize resource scheduling to minimize total system energy consumption. And maximize overall computational efficiency

[0206]

[0207] Wherein, λ is an energy efficiency trade-off parameter that can be dynamically adjusted to adapt to different business needs.

[0208] Each intelligent agent makes resource scheduling decisions based on local observations (local load, energy consumption, task execution status) and global feedback information (overall system load balancing status, future business forecast data).

[0209] Sub-step 4.2: Scheduling optimization based on dynamic trade-offs

[0210] To ensure the rational allocation of computing resources, the intelligent agent needs to comprehensively consider load balancing, task execution efficiency, and energy consumption optimization goals during the decision-making process, and adopt a scheduling optimization strategy based on dynamic trade-offs.

[0211] Define a resource scheduling optimization objective function to minimize resource allocation imbalance and maximize computational efficiency:

[0212]

[0213] in, This indicates the average resource allocation across all computing nodes.

[0214] Introducing a dynamic weight adjustment strategy based on global energy efficiency feedback makes resource scheduling more flexible:

[0215]

[0216] Where Φ is the overall energy efficiency optimization target of the system, and η is the adjustment step size, so that resource scheduling can adapt to the real-time load changes of the system.

[0217] Intelligent agents compete for resources through a game-theoretic mechanism to ensure optimal global scheduling.

[0218] Sub-step 4.3: Task allocation stability analysis

[0219] Due to the nonlinear nature of intelligent group decision-making, task allocation may experience oscillations or instability, thus requiring stability analysis and the design of stability enhancement mechanisms.

[0220] Calculate the task migration stability index

[0221]

[0222] like If the threshold is exceeded, it indicates that there are significant fluctuations in task allocation, and adjustments are needed.

[0223] Gradient descent is used to converge and adjust task scheduling.

[0224]

[0225] Where η is the convergence factor, ensuring that resource scheduling gradually tends to a stable state.

[0226] Sub-step 4.4: Adaptive convergence adjustment

[0227] To improve the algorithm's adaptability and enable resource scheduling to be automatically optimized under different load conditions, an adaptive convergence adjustment strategy is introduced.

[0228] Calculate the convergence rate δΦ of the global energy efficiency target t :

[0229]

[0230] If δΦ t If δΦ is too large, it indicates that the scheduling adjustment is too drastic and the adjustment step size needs to be reduced; t If it is too small, then more adjustments are needed.

[0231] Dynamically adjust the decision step size α of the intelligent agent:

[0232] α t+1 =α t ×(1-k·δΦ t )

[0233] Where k is the convergence adjustment coefficient, which ensures the adaptability of the algorithm under different load conditions.

[0234] The intelligent group decision-making resource scheduling optimization achieves intelligent scheduling optimization of data center computing resources through four key sub-steps: intelligent group decision-making modeling, scheduling optimization based on dynamic trade-offs, task allocation stability analysis, and adaptive convergence adjustment. This invention effectively combines historical energy consumption trends, real-time load data, and future business needs, and utilizes an intelligent agent collaborative decision-making mechanism to enable data centers to achieve optimal energy efficiency while ensuring high computing performance.

[0235] Specifically, step 5 includes:

[0236] Step 1 involves accurately collecting energy efficiency data through a dynamic self-calibration mechanism. Step 2 involves constructing a dynamic prediction model based on an energy efficiency feedback mechanism. Step 3 involves achieving dynamic load balancing using an energy efficiency trade-off method. Step 4 involves optimizing resource scheduling using an intelligent group decision-making strategy. Step 5 further introduces an optimization mechanism based on autonomous energy efficiency learning to improve the data center's energy efficiency adaptability, resource scheduling accuracy, and business load adaptability during long-term operation. This mechanism relies on an adaptive training framework combining reinforcement learning and imitation learning, enabling the system to continuously optimize its energy efficiency management strategy in complex dynamic environments, ultimately achieving self-evolutionary optimization of the data center.

[0237] Sub-step 5.1: Construction of the Energy Efficiency Autonomous Learning Framework

[0238] To enable the system to autonomously optimize energy efficiency management strategies based on historical data and real-time feedback, an autonomous learning framework is introduced, which consists of three parts: energy efficiency state space construction, energy efficiency strategy generation network, and decision-making reward mechanism.

[0239] Define the energy efficiency state space S, and let the state of the data center at time t be represented as:

[0240] S t ={L t ,P t ,R t ,Φ t}

[0241] Among them, L t P represents the service load status. t In energy consumption state, R t For resource scheduling status, Φ t The overall energy efficiency target of the system.

[0242] An energy efficiency policy generation network is constructed, and adaptive optimization is performed using a policy network based on deep reinforcement learning. Its decision function is:

[0243] π θ (A t |S t )=P(A t |S t ,θ)

[0244] Among them, A t This represents the scheduling action, θ is the policy network parameter, and π is the digit. θ This represents the policy function.

[0245] During training, EPN continuously adjusts θ through a reinforcement learning framework to optimize policy decisions and maximize the system's energy efficiency benefits.

[0246] Sub-step 5.2: Initial policy training based on imitation learning

[0247] Since reinforcement learning often faces problems such as low sample efficiency and high exploration costs in the initial stage, imitation learning methods are introduced to use the historical best scheduling scheme for initial policy training in order to improve the efficiency of autonomous learning.

[0248] Using historical best energy efficiency scheduling data D expert Training the initial policy network

[0249]

[0250] in, This is the best scheduling strategy in history.

[0251] By adjusting the parameters of the policy network through supervised learning, it can approximate the optimal energy-efficient scheduling scheme in the initial stage, thereby reducing exploration costs.

[0252] Sub-step 5.3: Autonomous Optimization Based on Reinforcement Learning

[0253] After completing the initial strategy training, the system continuously optimizes the energy efficiency scheduling strategy in a real-time environment through reinforcement learning, making it adaptable to different business loads and energy efficiency constraints.

[0254] Optimize energy efficiency strategy using policy gradient method:

[0255]

[0256] Where α is the learning rate, and J(θ) is the energy efficiency optimization objective:

[0257]

[0258] Where γ is the discount factor, R t The reward function measures the energy efficiency gains of the current scheduling strategy.

[0259] Improve policy convergence speed and reduce policy oscillations by using an experience replay-based approach.

[0260] Sub-step 5.4: Adaptive optimization and dynamic energy efficiency adjustment

[0261] To enable the autonomous learning mechanism to adapt to different business needs and environmental changes, an adaptive optimization strategy is designed to allow the energy efficiency optimization strategy to be dynamically adjusted.

[0262] Adaptive adjustment of the learning rate based on target bias:

[0263] α t+1 =α t ×(1-k|J(θ t )-J(θ t-1 )|)

[0264] Where k is the adaptive adjustment coefficient, which reduces the learning rate to stabilize the optimization process when the strategy optimization returns fluctuate greatly.

[0265] A dynamic energy efficiency adjustment mechanism is introduced to enable scheduling strategies to adapt to different business scenarios in real time:

[0266] If the workload suddenly increases, increase the resource scheduling response speed.

[0267] If the energy consumption exceeds the set threshold, the scheduling strategy will be automatically adjusted to put the computing node into a low-power mode.

[0268] The optimization mechanism of energy efficiency autonomous learning is achieved by constructing an energy efficiency autonomous learning framework, using imitation learning for initial strategy training, adopting reinforcement learning methods to optimize energy efficiency scheduling strategies, and introducing adaptive tuning and dynamic energy efficiency adjustment mechanisms. This enables data centers to continuously optimize their energy efficiency management strategies during long-term operation, improve resource scheduling accuracy, adapt to different business load requirements, and achieve true self-evolutionary optimization.

[0269] Example 2

[0270] Dynamic self-calibration energy efficiency data acquisition module: Deploy sensors and a distributed acquisition network, and realize energy efficiency data acquisition by calculating the rate of change of energy efficiency and adjusting the adaptive sampling frequency, combined with edge computing preprocessing;

[0271] Dynamic prediction module: It adopts adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling and multi-level prediction fusion to realize dynamic energy efficiency prediction;

[0272] Dynamic load balancing module: Introduces energy-efficiency-aware load assessment metrics, combined with task migration decisions, dynamic weight adjustments, and global load balancing optimization, to maximize energy efficiency;

[0273] Resource scheduling optimization module: Constructs an intelligent group decision-making model to achieve global allocation of computing resources through dynamic trade-off scheduling optimization, task allocation stability analysis, and adaptive convergence adjustment;

[0274] Adaptive optimization module: Based on the energy efficiency autonomous learning framework, it uses imitation learning for initial policy training, reinforcement learning to optimize scheduling strategies, and adaptive tuning and dynamic adjustment to achieve adaptive optimization.

[0275] The modules in Embodiment 2 are used to implement the functions in Embodiment 1. This embodiment can be implemented by a system including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement the data center energy efficiency data detection and processing method according to Embodiment 1 of this application. The system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface. Their settings and functions are known in the art and will not be described in detail here.

[0276] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting and processing energy efficiency data in a data center, characterized in that, include: Step 1: Deploy sensors and a distributed acquisition network, and achieve energy efficiency data acquisition by calculating the rate of change of energy efficiency and adjusting the adaptive sampling frequency, combined with edge computing preprocessing; Step 2: Adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling, and multi-level prediction fusion are employed to achieve dynamic energy efficiency prediction; among which, short-term and long-term error feedback correction includes: Calculate the short-term error ΔS t Used to dynamically adjust prediction model parameters: Among them, E t This represents the actual energy consumption value. This is the predicted value from the previous moment; Calculate the long-term error ΔL t Used to correct trend forecasts: Where N is the long-term error calculation window; The prediction model parameter set Θ is dynamically adjusted, and corrections are made based on both short-term and long-term errors. I t =Θ t-1 +η1ΔS t +η2ΔL t Where η1 and η2 are adjustment coefficients used to control the impact of short-term and long-term errors on model parameters; Step 3: Introduce energy efficiency-aware load assessment metrics, and combine them with task migration decisions, dynamic weight adjustments, and global load balancing optimization to maximize energy efficiency; Step 4: Construct an intelligent group decision-making model, and realize the global allocation of computing resources through dynamic trade-off scheduling optimization, task allocation stability analysis and adaptive convergence adjustment; Step 5: Based on the energy efficiency autonomous learning framework, use imitation learning to train the initial strategy, reinforcement learning to optimize the scheduling strategy, adaptive tuning and dynamic adjustment to achieve adaptive optimization.

2. The method for detecting and processing energy efficiency data in a data center according to claim 1, characterized in that, Step 1 includes: Deploy energy efficiency sensors in critical data center equipment to build a distributed data acquisition network, periodically collect power, load, and timestamp data from the equipment, and send them to edge nodes; Calculate the rate of change in energy efficiency and determine whether the energy consumption status of the data center fluctuates drastically by setting a threshold. The sampling frequency is dynamically adjusted based on the rate of change of energy efficiency, the sampling interval is controlled by a nonlinear adjustment factor, and the sampling period is smoothed by an exponential weighted moving average to ensure that the sampling period adapts to load changes. Data preprocessing is performed at edge nodes, outlier data is filtered and removed using local anomaly factors, and data is smoothed using sliding window mean filtering. Data is only uploaded to the central server after the confidence level is met, thereby reducing communication and computing overhead.

3. The method for detecting and processing energy efficiency data in a data center according to claim 2, characterized in that, Step 2 includes: The size of the forecast time window is dynamically adjusted according to the rate of change in energy efficiency, and the impact of short-term fluctuations on the window is suppressed by the exponential smoothing method to ensure that the window size adapts to the fluctuation of energy efficiency. Short-term and long-term error feedback mechanisms are introduced to dynamically adjust the parameters of the prediction model. Short-term errors are used for fine-tuning the model parameters, while long-term errors are used to correct trend predictions. We construct a feature vector based on the rate of change of energy efficiency, use a nonlinear autoregressive model for trend modeling, and introduce a periodic factor to adjust the model output to adapt to the periodic characteristics of data center energy consumption. By integrating short-term and long-term forecast results and dynamically adjusting the weights, the final forecast output is optimized based on short-term and long-term errors, thereby achieving dynamic forecasting of energy efficiency data.

4. The method for detecting and processing energy efficiency data in a data center according to claim 3, characterized in that, Step 3 includes: Define energy efficiency-aware load assessment metrics, which integrate the CPU / GPU utilization of computing nodes, current power consumption, and task latency. Standardize each metric through a normalization method to achieve load measurement. Based on energy efficiency optimization, the task migration decision is calculated, the global energy efficiency optimization objective function is calculated, the principle of minimum incremental energy consumption is followed when selecting migration tasks, and constraints are set to ensure that the target node is not overloaded. Dynamic weight adaptive adjustment: The weight parameters are dynamically updated based on historical error feedback. When the system load is unbalanced, the load balancing weight is increased, and when energy consumption exceeds the standard, the energy consumption optimization weight is increased. Global load balancing optimization employs gradient descent to adjust task allocation strategies, minimizing load variance and ensuring optimal overall energy efficiency for the data center.

5. The method for detecting and processing energy efficiency data in a data center according to claim 4, characterized in that, Step 4 includes: A smart group decision-making model is constructed, consisting of multiple smart agents. Each agent makes resource scheduling decisions based on local observations and global feedback. The goal is to minimize the total energy consumption of the system and maximize the overall computational efficiency. Implement scheduling optimization based on dynamic trade-offs, define the objective function for resource scheduling optimization, introduce a global energy efficiency feedback adjustment strategy, and achieve resource competition and global optimal scheduling through the game mechanism among intelligent agents; Perform task allocation stability analysis, calculate task migration stability index, and if it exceeds the threshold, use gradient descent method to adjust task scheduling to enhance stability. The algorithm performs adaptive convergence adjustment, dynamically calculates the convergence speed of the global energy efficiency target, and adjusts the decision step size of the intelligent agent according to the convergence situation to ensure that the algorithm is highly adaptable under different load conditions and realizes intelligent scheduling and optimization of data center computing resources.

6. The method for detecting and processing energy efficiency data in a data center according to claim 5, characterized in that, Step 5 includes: An energy efficiency autonomous learning framework is constructed, defining the state space of the data center, including business load, energy consumption, resource scheduling and overall energy efficiency targets, and adopting a strategy based on deep reinforcement learning to generate a network for adaptive optimization; Imitation learning is used for initial policy training, and supervised learning methods are used to adjust the policy network parameters in combination with historical best scheduling data to improve the policy accuracy in the initial stage. Reinforcement learning is used for autonomous optimization, the policy gradient method is used to dynamically adjust the policy network, and the experience replay mechanism is combined to improve the policy convergence speed and ensure that the energy efficiency scheduling policy adapts to real-time environmental changes. Implement adaptive tuning and dynamic energy efficiency adjustment, dynamically adjust the learning rate based on target deviation, adapt to changes in business load in real time, and achieve long-term self-evolution optimization of the data center.

7. A data center energy efficiency data detection and processing system, characterized in that, include: Dynamic self-calibration energy efficiency data acquisition module: Deploy sensors and a distributed acquisition network, and realize energy efficiency data acquisition by calculating the rate of change of energy efficiency and adjusting the adaptive sampling frequency, combined with edge computing preprocessing; Dynamic prediction module: Employing adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling, and multi-level prediction fusion, it achieves dynamic energy efficiency prediction; among which, short-term and long-term error feedback correction includes: Calculate the short-term error ΔS t Used to dynamically adjust prediction model parameters: Among them, E t This represents the actual energy consumption value. This is the predicted value from the previous moment; Calculate the long-term error ΔL t Used to correct trend forecasts: Where N is the long-term error calculation window; The prediction model parameter set Θ is dynamically adjusted, and corrections are made based on both short-term and long-term errors. I t =Θ t-1 +η1ΔS t +η2ΔL t Where η1 and η2 are adjustment coefficients used to control the impact of short-term and long-term errors on model parameters; Dynamic load balancing module: Introduces energy-efficiency-aware load assessment metrics, combined with task migration decisions, dynamic weight adjustments, and global load balancing optimization, to maximize energy efficiency; Resource scheduling optimization module: Constructs an intelligent group decision-making model to achieve global allocation of computing resources through dynamic trade-off scheduling optimization, task allocation stability analysis, and adaptive convergence adjustment; Adaptive optimization module: Based on the energy efficiency autonomous learning framework, it uses imitation learning for initial policy training, reinforcement learning to optimize scheduling strategies, and adaptive tuning and dynamic adjustment to achieve adaptive optimization.

Images