Energy efficiency data detection processing method and system of data center

Abstract

The invention discloses an energy efficiency data detection processing method and system for a data center, and the method comprises the steps: deploying a sensor and a distributed collection network, and achieving the collection of energy efficiency data through the calculation of an energy efficiency change rate and the adjustment of a self-adaptive sampling frequency; dynamic energy efficiency prediction is realized by adopting adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling and multi-level prediction fusion; an energy efficiency perception load evaluation index is introduced to realize energy efficiency maximization; an intelligent group decision model is constructed, and global allocation of computing resources is realized through dynamic balance scheduling optimization, task allocation stability analysis and adaptive convergence adjustment; based on an energy efficiency autonomous learning framework, initial strategy training, reinforcement learning optimization scheduling strategy, adaptive tuning and dynamic adjustment are carried out by utilizing imitation learning, and adaptive optimization is realized. According to the invention, under the condition that the business of the data center changes rapidly, the allocation of the computing resources can be adjusted accurately, rapidly and dynamically, and the optimal energy efficiency is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of data detection, and particularly to an energy efficiency data detection and processing method and system for a data center. Background Art

[0002] With the rapid development of technologies such as cloud computing, big data, and artificial intelligence, data centers have become important infrastructure to support the global digital economy. However, the energy consumption problem of data centers has become increasingly prominent, and their energy consumption has accounted for a significant proportion of the global total electricity consumption, directly affecting operating costs and environmental sustainability. Therefore, improving the energy efficiency of data centers has become one of the key challenges in current research and engineering practice.

[0003] The energy efficiency management of data centers involves multiple aspects, including server energy consumption optimization, network device power consumption management, cooling system optimization, and load balancing and resource scheduling, etc. Among them, load balancing and resource scheduling play a crucial role in energy efficiency optimization. They not only determine the utilization rate of computing resources but also affect the overall energy consumption and system performance. An ideal load balancing and resource scheduling scheme should be able to dynamically adapt to changes in business requirements and achieve the lowest energy consumption while ensuring performance. However, current load balancing and resource scheduling methods generally face many challenges.

[0004] Traditional load balancing strategies often rely on fixed rules or historical statistical information and are difficult to adapt to the dynamic environment of data centers in real time, resulting in problems such as uneven distribution of computing resources, server overload, or waste of idle resources. At the same time, existing resource scheduling algorithms are mainly based on static optimization or heuristic methods, and have problems such as slow response speed and insufficient scheduling accuracy when dealing with complex business loads, and are difficult to effectively handle sudden business traffic, dynamic energy consumption changes, and resource competition in a multi-tenant environment. In addition, the energy consumption characteristics of servers, storage devices, and network nodes inside data centers are different. How to comprehensively consider the utilization rate of computing resources, power consumption characteristics, and cooling energy consumption while meeting the requirements of service quality (QoS) and achieve the optimal balance between energy efficiency and performance is still a technical problem to be solved urgently.

[0005] Therefore, in order to adapt to the rapidly changing business requirements of data centers, improve resource utilization rate, and reduce overall energy consumption, it is urgent to develop an adaptive load balancing and resource scheduling strategy that can perform dynamic resource allocation according to real-time business load and energy consumption data, so as to achieve energy efficiency optimization while ensuring computing performance.. Summary of the Invention

[0006] To solve the above problems, the present invention provides an energy efficiency data detection and processing method and system for a data center.

[0007] To achieve the above object, the technical solutions adopted by the present invention are as follows:

[0008] On the one hand, the present invention discloses an energy efficiency data detection and processing method for a data center, including:

[0009] Step 1: Deploy sensors and a distributed acquisition network. Through energy efficiency change rate calculation and adaptive sampling frequency adjustment, combined with edge computing preprocessing, realize energy efficiency data acquisition;

[0010] Step 2: Adopt adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling, and multi-level prediction fusion to realize energy efficiency dynamic prediction;

[0011] Step 3: Introduce an energy efficiency-aware load evaluation index. Combine task migration decision-making, dynamic weight adjustment, and global load balancing optimization to maximize energy efficiency;

[0012] Step 4: Build an intelligent group decision-making model. Through dynamic trade-off scheduling optimization, task allocation stability analysis, and adaptive convergence adjustment, realize global allocation of computing resources;

[0013] Step 5: Based on the energy efficiency autonomous learning framework, use imitation learning for initial policy training, reinforcement learning to optimize the scheduling policy, and adaptive tuning and dynamic adjustment to realize adaptive optimization.

[0014] Further: The said Step 1 includes:

[0015] Deploy energy efficiency sensors on key devices in the data center, build a distributed acquisition network, periodically collect power, load, and timestamp data of the devices and send them to the edge nodes;

[0016] Calculate the energy efficiency change rate, and judge whether the energy consumption state of the data center fluctuates violently by setting a threshold;

[0017] Dynamically adjust the sampling frequency based on the energy efficiency change rate, use a non-linear adjustment factor to control the sampling interval, and perform exponential weighted moving average smoothing to ensure that the sampling period adapts to the load change;

[0018] Perform data preprocessing at the edge node, use the local outlier factor to screen and eliminate abnormal data, use a sliding window mean filter to smooth the data, and only upload it to the central server after meeting the confidence level to reduce communication and computing overhead.

[0019] Further: The said Step 2 includes:

[0020] Dynamically adjust the size of the prediction time window according to the energy efficiency change rate, and suppress the influence of short-term fluctuations on the window through the exponential smoothing method to ensure that the window size adapts to the energy efficiency fluctuation situation;

[0021] Introduce short-term and long-term error feedback mechanisms to dynamically adjust the parameters of the prediction model. Combine short-term errors for fine-tuning the model parameters and long-term errors for correcting trend predictions.

[0022] Construct a feature vector based on the energy efficiency change rate, use a non-linear autoregressive model for trend modeling, and introduce a periodic factor to adjust the model output to adapt to the periodic characteristics of the data center energy consumption.

[0023] Fuse the short-term and long-term prediction results, dynamically adjust the weights, and optimize the final prediction output according to the short-term and long-term errors, so as to achieve dynamic prediction of energy efficiency data.

[0024] Further: The step 3 includes:

[0025] Define the load evaluation index for energy efficiency awareness, comprehensively calculate the CPU / GPU utilization rate, current power consumption, and task waiting time of the node, and standardize each index through a normalization method to achieve load measurement.

[0026] Based on the task migration decision for energy efficiency optimization, calculate the global energy efficiency optimization objective function, follow the principle of minimum incremental energy consumption when selecting migration tasks, and set constraints to ensure that the target node is not overloaded.

[0027] Dynamically adaptively adjust the weights, dynamically update the weight parameters according to the historical error feedback, increase the load balancing weight when the system load is unbalanced, and increase the energy consumption optimization weight when the energy consumption exceeds the standard.

[0028] Global load balancing optimization, use the gradient descent method to adjust the task allocation strategy, minimize the load variance, and ensure the overall optimal energy efficiency of the data center.

[0029] Further: The step 4 includes:

[0030] Construct an intelligent group decision-making model, which consists of multiple intelligent agents. Each agent makes resource scheduling decisions based on local observations and global feedback, with the goal of minimizing the total system energy consumption and maximizing the overall computing efficiency.

[0031] Implement scheduling optimization based on dynamic trade-off, define the resource scheduling optimization objective function, introduce a global energy efficiency feedback adjustment strategy, and achieve resource competition and global optimal scheduling through the game mechanism among intelligent agents.

[0032] Conduct task assignment stability analysis, calculate the task migration stability index, and if it exceeds the threshold, use the gradient descent method to adjust the task scheduling to enhance stability.

[0033] Perform adaptive convergence adjustment, dynamically calculate the convergence speed of the global energy efficiency target, adjust the decision-making step size of the intelligent agent according to the convergence situation, ensure that the algorithm is highly adaptable under different load conditions, and achieve intelligent scheduling optimization of the computing resources in the data center.

[0034] Further: The step 5 includes:

[0035] Construct an energy efficiency autonomous learning framework, define the state space of the data center, including business load, energy consumption, resource scheduling, and overall energy efficiency target, and perform adaptive optimization using a policy generation network based on deep reinforcement learning;

[0036] Use imitation learning for initial policy training, and through supervised learning methods, adjust the policy network parameters in combination with historical optimal scheduling data to improve the policy accuracy in the initial stage;

[0037] Adopt reinforcement learning for autonomous optimization, dynamically adjust the policy network using the policy gradient method, and combine the experience replay mechanism to improve the policy convergence speed to ensure that the energy efficiency scheduling policy adapts to real-time environmental changes;

[0038] Implement adaptive tuning and dynamic energy efficiency adjustment, dynamically adjust the learning rate based on the target deviation, adapt to changes in business load in real time, and achieve long-term self-evolution optimization of the data center.

[0039] On the other hand, the present invention discloses an energy efficiency data detection and processing system for a data center, including:

[0040] Dynamic self-calibration energy efficiency data acquisition module: Deploy sensors and a distributed acquisition network, and through the calculation of the energy efficiency change rate and the adjustment of the adaptive sampling frequency, combined with edge computing preprocessing, achieve energy efficiency data acquisition;

[0041] Dynamic prediction module: Adopt adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling, and multi-level prediction fusion to achieve dynamic energy efficiency prediction;

[0042] Dynamic load balancing module: Introduce an energy efficiency-aware load evaluation index, and combine task migration decision-making, dynamic weight adjustment, and global load balancing optimization to achieve maximum energy efficiency;

[0043] Resource scheduling optimization module: Construct an intelligent group decision-making model, and through dynamic trade-off scheduling optimization, task allocation stability analysis, and adaptive convergence adjustment, achieve global allocation of computing resources;

[0044] Adaptive optimization module: Based on the energy efficiency autonomous learning framework, use imitation learning for initial policy training, reinforce learning to optimize the scheduling policy, perform adaptive tuning and dynamic adjustment, and achieve adaptive optimization.

[0045] Compared with the prior art, the technical progress achieved by the present invention lies in:

[0046] In the process of collecting energy efficiency data in traditional data centers, there are problems such as noise interference and inaccurate equipment energy consumption models, resulting in difficulties in achieving ideal effects in subsequent optimization. The present invention adopts a dynamic self-calibration mechanism, combines time series analysis, outlier detection, and adaptive calibration technology to achieve high-precision collection of energy efficiency data, enabling reliable energy efficiency data support for load balancing and resource scheduling, and fundamentally improving the accuracy of decision-making.

[0047] Existing load balancing and resource scheduling algorithms often rely on historical statistical data during adjustment and are unable to accurately predict future load and energy consumption trends, resulting in slow response speeds. The dynamic prediction based on energy efficiency feedback proposed by the present invention, through an adaptive attention mechanism and multi-modal fusion learning, combines historical business load, temperature, power consumption, etc. data to achieve high-precision prediction of energy efficiency trends, enabling the system to perceive load changes in advance, optimize the forward-looking nature of resource scheduling, and thus improve scheduling efficiency and response speed.

[0048] Existing load balancing algorithms are difficult to adapt to complex energy efficiency requirements, resulting in low utilization rate of computing resources and excessive server energy consumption. The dynamic load balancing algorithm of energy efficiency trade-off of the present invention, by constructing an energy efficiency optimization objective function, weighing performance requirements and power consumption overhead, and dynamically adjusting task allocation based on reinforcement learning, enables computing nodes to maximize energy efficiency benefits while ensuring quality of service (QoS).

[0049] Traditional resource scheduling strategies are difficult to adapt to sudden load changes in data centers. They often adopt static or heuristic methods and lack global optimization capabilities. The resource scheduling optimization algorithm of intelligent group decision-making of the present invention, by introducing multi-agent reinforcement learning and game theory-driven intelligent group decision-making, enables multiple computing nodes to collaboratively adjust resource allocation, dynamically optimize scheduling strategies in the case of sudden loads, improve the utilization rate of computing resources, and reduce power consumption waste.

[0050] Existing energy efficiency optimization methods usually rely on manually set thresholds and rules and are difficult to adapt to environmental changes during long-term operation. The present invention introduces an energy efficiency autonomous learning optimization mechanism, combines reinforcement learning and imitation learning, enabling the system to continuously learn historical best strategies during operation and adaptively adjust the energy efficiency optimization scheme according to current business requirements, realizing long-term self-optimization of the data center.

[0051] Traditional energy efficiency optimization methods often overly focus on reducing energy consumption while ignoring system performance, resulting in a decline in user experience. The present invention comprehensively considers the trade-off between energy efficiency and computing performance in each link, ensuring that while optimizing energy consumption, it does not affect service quality, and ultimately achieving a dynamic balance between optimal energy efficiency and optimal performance.

[0052] In summary, the present invention innovates from multiple aspects such as data acquisition, load prediction, resource scheduling, and autonomous optimization, breaking through the bottlenecks of traditional load balancing and resource scheduling, such as slow response speed and insufficient scheduling accuracy in dynamic adjustment. Through innovative technical means such as real-time high-precision energy efficiency data acquisition, intelligent prediction, reinforcement learning optimization, intelligent group decision-making, and autonomous learning, the intelligent, automated, and adaptive management of the energy efficiency of the data center is realized. Finally, the present invention can accurately, quickly, and dynamically adjust the allocation of computing resources in the case of rapid changes in the data center business, achieve the optimal energy efficiency and performance balance, greatly reduce the overall energy consumption of the data center, and improve the operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention.

[0054] In the drawings:

[0055] Figure 1 is a flowchart of the present invention;

[0056] Figure 2 is a system structure diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below with reference to the drawings.

[0058] Embodiment 1

[0059] As Figure 1 shown, the present invention discloses an energy efficiency data detection and processing method for a data center, including:

[0060] Specifically, step 1 includes:

[0061] In order to solve the contradiction between data lag and computational overhead in the traditional energy efficiency data acquisition method, this step adopts a dynamic self-calibration mechanism to adjust the sampling frequency in real time according to the energy efficiency fluctuation situation, so as to ensure the accuracy and timeliness of the data, and at the same time reduce unnecessary computational costs. The dynamic self-calibration energy efficiency data acquisition mainly includes four key sub-steps: sensor data acquisition, energy efficiency change rate calculation, adaptive sampling frequency adjustment, and edge preprocessing. The following is the detailed implementation process.

[0062] Sub-step 1.1: Sensor data acquisition

[0063] Deploy high-precision energy efficiency sensors at key positions of servers, storage devices, network switches, and cooling systems in the data center, and build a distributed data acquisition network to achieve the acquisition of energy efficiency data for different device types.

[0064] Set the device set S = {s1, s2,..., s n}, where s i represents the i-th device in the data center.

[0065] Deploy an energy efficiency sensor P i for each device s i . This sensor periodically measures the power consumption and operating load of the device and records the timestamp t.

[0066] Define the collected data vector as:

[0067]

[0068] where, is the power consumption data at the current moment, is the current CPU load data, is the timestamp information.

[0069] The data acquisition module is responsible for periodically reading sensor data and sending the data to the local edge node for real-time preprocessing.

[0070] Sub-step 1.2: Calculation of the energy efficiency change rate

[0071] To determine whether the energy efficiency state of the current data center has changed significantly, it is necessary to calculate the change rate of the energy efficiency data, so as to provide a basis for subsequent adaptive sampling frequency adjustment.

[0072] Set the time window size to Δt, and calculate the energy efficiency change rate within two adjacent time windows

[0073]

[0074] where, is the energy consumption value at the current moment t, is the energy consumption value of the previous time window.

[0075] Calculate the global energy efficiency change rate R t of the entire data center as the global adjustment parameter:

[0076]

[0077] where N is the total number of devices in the data center.

[0078] By setting the change rate threshold R th , when R t >R th , it indicates that the energy consumption status of the data center fluctuates violently, and the sampling frequency needs to be increased; when R t <R th , the sampling frequency can be reduced to reduce the data transmission and calculation burden.

[0079] Sub-step 1.3: Adaptive sampling frequency adjustment

[0080] Based on the calculated energy efficiency change rate, dynamically adjust the sampling frequency so that the sampling period can be automatically optimized according to the load change of the data center, improving the timeliness and accuracy of the data.

[0081] Set the initial sampling interval T0 (i.e., the default data sampling period in the initial state of the system).

[0082] Use a non-linear adjustment factor α to control the dynamic adjustment of the sampling period, and calculate the sampling interval T at the next moment t :

[0083]

[0084] where α is the adaptive adjustment coefficient, ensuring that the sampling interval is shortened during violent data fluctuations and increased during stable data.

[0085] Set the upper and lower limits of the sampling interval T min and T max , to ensure that the sampling period does not exceed a reasonable range:

[0086] T t = max(T min , min(T max , T t ))

[0087] Use historical data for exponentially weighted moving average smoothing of the sampling interval to avoid violent oscillations:

[0088] T t = λT t + (1 - λ)T t-1

[0089] where λ is the smoothing coefficient, used to control the influence degree of historical data on the current adjustment.

[0090] Sub-step 1.4: Edge computing preprocessing

[0091] Since the energy efficiency data collection in the data center involves a large amount of sensor data, directly transmitting it to the central server will cause a large amount of computing and communication overhead. Therefore, data preprocessing needs to be carried out on edge devices to reduce the computing burden on the main server.

[0092] Deploy a data anomaly detection module at the edge computing node, and use an algorithm based on Local Outlier Factor (LOF) to screen out abnormal data and eliminate extreme values.

[0093] Use the outlier detection formula to calculate the anomaly score of each data point:

[0094]

[0095] Among them, is the local reachability density, and k is the number of neighbor points.

[0096] Set the anomaly threshold LOF th , when , this data point is marked as abnormal and discarded.

[0097] Use moving average filtering based on a sliding window for data smoothing to reduce the noise impact in the data:

[0098]

[0099] Among them, W is the size of the sliding window, is the smoothed energy efficiency data.

[0100] The processed data is only uploaded to the central server after meeting a certain confidence level, thereby reducing the transmission of invalid data and improving the system response speed.

[0101] The dynamic self-calibration energy efficiency data collection realizes the efficient and accurate collection of the energy efficiency data in the data center through four sub-steps: sensor data collection, energy efficiency change rate calculation, adaptive sampling frequency adjustment, and edge computing preprocessing, ensuring the timeliness of the data and the optimization of the computing cost, and at the same time providing high-quality input data for subsequent energy efficiency prediction, load balancing, and resource scheduling.

[0102] Specifically, step 2 includes:

[0103] In order to make full use of the high-precision energy efficiency data collected in step 1 and solve the problems of low precision and lack of adaptability of the existing energy consumption prediction model due to fixed time windows and static parameter adjustments, this step designs a dynamic prediction model based on energy efficiency feedback. This model combines the energy efficiency change rate, short-term and long-term error feedback, and dynamic adjustment window strategy to achieve high-precision prediction of data center energy efficiency. Dynamic prediction based on energy efficiency feedback mainly includes four core sub-steps: adaptive time window adjustment, feedback-based error correction, energy efficiency trend modeling, and multi-level prediction fusion. The following is the detailed implementation process.

[0104] Sub-step 2.1: Adaptive time window adjustment

[0105] Since the energy efficiency fluctuations in data centers are not balanced, in order to avoid the fixed window prediction method losing key information when the fluctuations are drastic or causing over-computation in a stable state, it is necessary to use the energy efficiency change rate R t The prediction window size is dynamically adjusted to ensure the optimal time span selection.

[0106] The initial time window size W0 is usually set to the default number of time steps, such as 10 time steps).

[0107] Calculate the adaptive time window W t The rate of change of energy efficiency R t Dynamic Adjustment:

[0108] W t =W0+γR t

[0109] Among them, γ is the adjustment coefficient, which is used to control the window change amplitude so that the window size increases with the increase of energy efficiency fluctuation to adapt to the drastic changes.

[0110] Set the upper and lower limits of the window size W min and W max , to ensure that the window is not too small or too large:

[0111] W t =max(W min ,min(W max ,W t ))

[0112] Exponential smoothing is used to smooth the window size to avoid drastic fluctuations in the window size:

[0113] W t =λW t +(1-λ)W t-1

[0114] Among them, λ is the smoothing coefficient, which is used to suppress the influence of short-term data fluctuations on the window size.

[0115] Sub-step 2.2: Error correction based on feedback

[0116] Traditional prediction models use fixed parameters during training. However, the energy efficiency of data centers is affected by complex factors, and a single static model cannot adapt to dynamic changes. Therefore, an error correction mechanism based on energy efficiency feedback is introduced to improve the prediction accuracy through hierarchical adjustment of short-term and long-term errors.

[0117] Calculate the short-term error ΔS t , which is used to dynamically adjust the parameters of the prediction model:

[0118]

[0119] Among them, E t is the actual energy consumption value, is the predicted value at the previous moment.

[0120] Calculate the long-term error ΔL t , which is used to correct the trend prediction:

[0121]

[0122] Among them, N is the long-term error calculation window.

[0123] Dynamically adjust the set of prediction model parameters Θ, and correct it by combining short-term error and long-term error:

[0124] Θ t = Θ t-1 + η1ΔS t + η2ΔL t

[0125] Among them, η1, η2 are adjustment coefficients, which are used to control the influence of short-term and long-term errors on the model parameters.

[0126] Sub-step 2.3: Energy efficiency trend modeling

[0127] In order to improve the adaptability of the prediction model to the energy efficiency change trend, it is necessary to perform trend modeling on the energy efficiency change and construct a time series prediction model by combining historical data.

[0128] Construct a feature vector based on the energy efficiency change rate:

[0129]

[0130] Among them, W t is the adaptive time window, and R t is the current energy efficiency change rate.

[0131] Trend modeling is carried out using a non - linear autoregressive model, and its prediction formula is as follows:

[0132]

[0133] Among them, f(·) is a non - linear mapping function, and Θ t is the model parameter optimized based on error feedback.

[0134] A periodic factor is introduced to adjust the model output, considering the periodic characteristics of the energy consumption in the data center:

[0135]

[0136] Among them, ω is the periodic influence coefficient, and T is the period length.

[0137] Sub - step 2.4: Multi - level prediction fusion

[0138] Since the change of energy consumption in the data center is affected by various factors, a single prediction method is difficult to adapt to all situations. Therefore, a method of fusing short - term prediction and long - term prediction is adopted to improve the overall prediction accuracy.

[0139] Construct a short - term prediction model F s , which is used to quickly respond to energy efficiency fluctuations:

[0140]

[0141] Use LSTM (Long Short - Term Memory Network) for short - term energy consumption prediction, which is suitable for dealing with sudden fluctuations.

[0142] Construct a long - term prediction model F l , which is used to capture the overall energy efficiency trend:

[0143]

[0144] Use the autoregressive integrated moving average model for long - term energy consumption trend prediction, which is suitable for capturing periodic changes.

[0145] Fuse the short - term and long - term prediction results, and use a weighted method for the final prediction output:

[0146]

[0147] Among them, w s and w l are dynamically adjusted weights, which are optimized according to the short - term error and the long - term error:

[0148]

[0149]

[0150] Ensure that when the short-term error is large, the proportion of long-term prediction is higher, and when the long-term error is large, the proportion of short-term prediction is higher, so as to achieve the effect of adaptive optimization.

[0151] Based on the dynamic prediction of energy efficiency feedback, through four key sub-steps: adaptive time window adjustment, feedback-based error correction, energy efficiency trend modeling, and multi-level prediction fusion, high-precision dynamic prediction of energy efficiency data is achieved. The present invention can improve the prediction accuracy when the energy consumption of the data center changes rapidly, reduce the calculation cost at the same time, and provide high-quality prediction data input for subsequent load balancing and resource scheduling.

[0152] Specifically, step 3 includes:

[0153] In order to make full use of the high-precision energy efficiency data collected in step 1 and the future energy consumption trend predicted in step 2, and to solve the problems of slow response speed and insufficient energy efficiency optimization in the existing load balancing strategy during the dynamic adjustment process, this step designs a dynamic load balancing algorithm based on energy efficiency trade-off. By combining real-time energy consumption data, predicted energy consumption trend, and computing resource utilization rate, this algorithm optimizes the overall energy consumption while ensuring service performance, and realizes the efficient allocation of computing resources in the data center. The dynamic load balancing based on energy efficiency trade-off mainly includes four core sub-steps: energy efficiency-aware load assessment, task migration decision based on energy efficiency optimization, dynamic weight adaptive adjustment, and global load balancing optimization. The following is the detailed implementation process.

[0154] Sub-step 3.1: Energy efficiency-aware load assessment

[0155] Traditional load balancing methods usually rely only on CPU utilization or task queue length for load assessment, while ignoring the energy efficiency optimization goal. Therefore, an energy efficiency-aware load assessment metric is introduced to comprehensively consider the computing resource usage and energy consumption factors and achieve accurate load measurement.

[0156] Define the energy efficiency-aware load of computing node i at time t as:

[0157]

[0158] Where:

[0159] is the CPU / GPU utilization rate of computing node i, reflecting the computing resource load situation;

[0160] is the current power consumption of computing node i, provided by the energy efficiency data collected in step 1;

[0161] is the task waiting time of computing node i, measuring the congestion degree of the task queue;

[0162] α, β, and γ are adjustable weight coefficients to balance the influence of different factors.

[0163] Use the normalization method to standardize the indicators in different dimensions to eliminate the dimension difference:

[0164]

[0165] where X is any load indicator (such as ), X max and X min are the maximum and minimum values of the historical data respectively.

[0166] Sub-step 3.2: Task migration decision based on energy efficiency optimization

[0167] When the load of a certain computing node exceeds the set threshold L max , it is necessary to migrate some of its tasks to reduce the overall energy consumption and balance the computing load. For this purpose, a task migration decision model is introduced to achieve dynamic task scheduling.

[0168] Calculate the global energy efficiency optimization objective function of the current system:

[0169]

[0170] where N is the total number of computing nodes in the data center, and ω1, ω2 are the weights for adjusting energy efficiency and load balancing.

[0171] When selecting tasks to migrate, adopt a task selection strategy based on the principle of minimum incremental energy consumption:

[0172]

[0173] where j is the task to be migrated, is the power consumption of the current node, is the power consumption of the target migration node after task migration.

[0174] Set task migration constraint conditions to ensure that the migrated tasks will not cause the target node to be overloaded:

[0175]

[0176] where k is the set of all possible target nodes

[0177] Sub-step 3.3: Dynamic weight adaptive adjustment

[0178] Since the load condition of the data center changes dynamically over time, the load balancing algorithm needs to adaptively adjust the energy efficiency weight to flexibly optimize performance and energy consumption under different business requirements. Therefore, a weight adaptive adjustment strategy based on feedback regulation is designed.

[0179] Calculate the historical error feedback for dynamically adjusting the weight:

[0180]

[0181] Among them, and are the average values of the global load balancing target and the energy consumption target respectively, and η is the learning rate.

[0182] Update the weight parameters to ensure that the system dynamically adapts to the load changes:

[0183]

[0184] When the system load is unbalanced, ω1 increases to enhance the weight of the load balancing target;

[0185] When the system energy consumption exceeds the expectation, ω2 increases to strengthen the energy consumption optimization target.

[0186] Sub-step 3.4: Global load balancing optimization

[0187] After the task migration decision is executed, in order to ensure the optimal global load balancing state of the data center, global load optimization adjustment is required.

[0188] Define the global load balancing optimization objective function to minimize the load variance:

[0189]

[0190] Among them, is the average load of all computing nodes.

[0191] Adopt the method based on gradient descent to perform load balancing optimization iteration and adjust the task allocation strategy:

[0192]

[0193] Among them, η is the learning rate, is the gradient of the global optimization target with respect to the load.

[0194] Execute the final load adjustment to ensure the optimal overall energy efficiency of the data center:

[0195] After the task migration is completed, recalculate the global load balancing state;

[0196] If the global load variance is still higher than the threshold, iterate and adjust the task allocation until a stable state is reached.

[0197] Dynamic load balancing with energy efficiency trade - off realizes the dynamic load balancing of the data center through four key sub - steps: energy - efficiency - aware load evaluation, task migration decision based on energy - efficiency optimization, dynamic weight adaptive adjustment, and global load - balancing optimization. This algorithm maximizes the energy - efficiency optimization effect while meeting business requirements, providing reliable support for subsequent resource - scheduling strategies.

[0198] Specifically, step 4 includes:

[0199] Based on accurately collecting energy - efficiency data through a dynamic self - calibration mechanism in step 1, constructing a dynamic prediction model relying on an energy - efficiency feedback mechanism in step 2, and implementing dynamic load balancing using an energy - efficiency trade - off method in step 3, step 4 further designs a resource - scheduling optimization algorithm based on intelligent group decision - making to dynamically optimize the allocation of computing resources globally, enabling the data center to make optimal scheduling decisions under different business loads and energy - efficiency constraints. This algorithm comprehensively considers historical energy - consumption trends, real - time load changes, and future business requirements, and introduces an intelligent group - decision - making model, enabling multiple intelligent agents to make collaborative decisions to achieve optimal resource utilization and energy - efficiency ratio. Its core includes four key sub - steps: intelligent group - decision - making modeling, scheduling optimization based on dynamic trade - off, task - allocation stability analysis, and adaptive convergence adjustment. The following is the detailed implementation process.

[0200] Sub - step 4.1: Intelligent group - decision - making modeling

[0201] To construct an efficient resource - scheduling decision - making mechanism, an intelligent group - decision - making model is introduced. This model consists of multiple autonomous intelligent agents (IA). Each intelligent agent makes decisions on the resource - scheduling scheme based on its local information and global feedback. The core definition of the intelligent group - decision - making model is as follows:

[0202] Define the set of intelligent agents: Suppose the data center contains M computing nodes, and each node i is managed by an intelligent agent IA i whose decision variable is the resource - allocation vector:

[0203]

[0204] where, represents computing resources (CPU / GPU), represents memory resources, represents storage resources.

[0205] The goal of the intelligent agent is to optimize resource scheduling to minimize the total system energy consumption and maximize the overall computing efficiency

[0206]

[0207] Among them, λ is the energy efficiency trade-off parameter, which can be dynamically adjusted to adapt to different service requirements.

[0208] Each intelligent agent makes resource scheduling decisions based on local observations (local load, energy consumption, task execution status) and global feedback information (system overall load balancing status, future service prediction data).

[0209] Sub-step 4.2: Scheduling optimization based on dynamic trade-off

[0210] To ensure the reasonable allocation of computing resources, the intelligent agent needs to comprehensively consider load balancing, task execution efficiency, and energy consumption optimization goals during the decision-making process, and adopt a scheduling optimization strategy based on dynamic trade-off.

[0211] Define the resource scheduling optimization objective function to minimize the resource allocation imbalance degree and maximize the computing efficiency:

[0212]

[0213] Among them, represents the average resource allocation of all computing nodes.

[0214] Introduce a dynamic weight adjustment strategy based on global energy efficiency feedback to make resource scheduling more flexible:

[0215]

[0216] Among them, Φ is the overall system energy efficiency optimization goal, and η is the adjustment step size to make resource scheduling adapt to the real-time load change of the system.

[0217] The intelligent agents compete for resources through a game mechanism to ensure the globally optimal scheduling.

[0218] Sub-step 4.3: Task assignment stability analysis

[0219] Due to the non-linear characteristics of the intelligent group decision-making, task assignment may exhibit oscillations or instability, so stability analysis is required and a stability enhancement mechanism needs to be designed.

[0220] Calculate the task migration stability index

[0221]

[0222] If exceeds the set threshold, it indicates that there are large fluctuations in task assignment and adjustment is required.

[0223] Use the gradient descent method to perform convergence adjustment on task scheduling:

[0224]

[0225] Among them, η is the convergence factor, which ensures that the resource scheduling gradually tends to a stable state.

[0226] Sub-step 4.4: Adaptive convergence adjustment

[0227] To improve the adaptability of the algorithm and enable the resource scheduling to be automatically optimized under different load conditions, an adaptive convergence adjustment strategy is introduced.

[0228] Calculate the convergence speed δΦ of the global energy efficiency target t :

[0229]

[0230] If δΦ t is too large, it means that the scheduling adjustment is too drastic, and the adjustment step size needs to be reduced; if δΦ t is too small, then the adjustment intensity needs to be increased.

[0231] Dynamically adjust the decision-making step size α of the intelligent agent:

[0232] α t+1 = α t ×(1 - k·δΦ t )

[0233] Among them, k is the convergence adjustment coefficient, which ensures the adaptability of the algorithm under different load conditions.

[0234] The resource scheduling optimization of intelligent group decision-making realizes the intelligent scheduling optimization of computing resources in the data center through four key sub-steps: intelligent group decision-making modeling, scheduling optimization based on dynamic trade-off, task assignment stability analysis, and adaptive convergence adjustment. The present invention effectively combines historical energy consumption trends, real-time load data, and future service requirements, and uses the collaborative decision-making mechanism of intelligent agents to achieve the optimal energy efficiency balance in the data center while ensuring high computing performance.

[0235] Specifically, step 5 includes:

[0236] On the basis of accurately collecting energy efficiency data through the dynamic self-calibration mechanism in step 1, constructing a dynamic prediction model relying on the energy efficiency feedback mechanism in step 2, realizing dynamic load balancing using the energy efficiency trade-off method in step 3, and optimizing resource scheduling using the intelligent group decision-making strategy in step 4, step 5 further introduces an optimization mechanism based on energy efficiency autonomous learning to improve the energy efficiency adaptability, resource scheduling accuracy, and service load adaptability of the data center during long-term operation. This mechanism relies on an adaptive training framework that combines reinforcement learning and imitation learning, enabling the system to continuously optimize its own energy efficiency management strategy in a complex dynamic environment and ultimately achieve self-evolutionary optimization of the data center.

[0237] Sub-step 5.1: Construction of the energy efficiency autonomous learning framework

[0238] To enable the system to autonomously optimize the energy efficiency management strategy based on historical data and real-time feedback, an autonomous learning framework is introduced, which consists of three parts: the construction of the energy efficiency state space, the generation network of energy efficiency strategies, and the decision reward mechanism.

[0239] Define the energy efficiency state space S. Let the state of the data center at time t be represented as:

[0240] S t ={L t , P t , R t , Φ t}

[0241] Among them, L t is the business load state, P t is the energy consumption state, R t is the resource scheduling state, and Φ t is the overall energy efficiency target of the system.

[0242] Construct the energy efficiency strategy generation network. An adaptive optimization is carried out using a policy network based on deep reinforcement learning, and its decision function is:

[0243] π θ (A t |S t ) = P(A t |S t , θ)

[0244] Among them, A t represents the scheduling action, θ is the parameter of the policy network, and π θ represents the policy function.

[0245] During the training process, the EPN continuously adjusts θ through the reinforcement learning framework to optimize the policy decision and maximize the energy efficiency benefit of the system.

[0246] Sub-step 5.2: Initial policy training based on imitation learning

[0247] Since reinforcement learning often faces problems such as low sample efficiency and high exploration cost in the initial stage, an imitation learning method is introduced to use the historical best scheduling scheme for initial policy training to improve the efficiency of autonomous learning.

[0248] Use the historical optimal energy efficiency scheduling data D expert to train the initial policy network

[0249]

[0250] Among them, is the best scheduling strategy in history.

[0251] Adjust the parameters of the policy network through supervised learning methods so that it can approximate the optimal energy - efficient scheduling scheme in the initial stage, reducing the exploration cost.

[0252] Sub - step 5.3: Autonomous optimization based on reinforcement learning

[0253] After completing the initial policy training, the system continuously optimizes the energy - efficient scheduling policy in the real - time environment through reinforcement learning methods to adapt to different business loads and energy - efficiency constraints.

[0254] Optimize the energy - efficient policy using the policy gradient method:

[0255]

[0256] where α is the learning rate and J(θ) is the energy - efficiency optimization objective:

[0257]

[0258] where γ is the discount factor and R t is the reward function, which measures the energy - efficiency benefit of the current scheduling policy.

[0259] Improve the policy convergence speed and reduce the policy oscillation problem through the method based on Experience Replay.

[0260] Sub - step 5.4: Adaptive tuning and dynamic energy - efficiency adjustment

[0261] In order to enable the autonomous learning mechanism to adapt to different business requirements and environmental changes, design an adaptive tuning strategy to enable dynamic adjustment of the energy - efficiency optimization policy.

[0262] Adaptive - adjust the learning rate based on the target deviation:

[0263] α t+1 = α t ×(1 - k|J(θ t ) - J(θ t-1 )|)

[0264] where k is the adaptive adjustment coefficient. When the policy optimization benefit fluctuates greatly, reduce the learning rate to stabilize the optimization process.

[0265] Introduce a dynamic energy - efficiency adjustment mechanism so that the scheduling policy can adapt to different business scenarios in real - time:

[0266] If the business load suddenly increases, increase the resource scheduling response speed

[0267] If the energy consumption exceeds the set threshold, the scheduling strategy is automatically adjusted to put the computing nodes into the low-power mode.

[0268] The optimization mechanism of energy efficiency autonomous learning enables the data center to continuously optimize its energy efficiency management strategy, improve the accuracy of resource scheduling, and adapt to different business load requirements to achieve true self-evolution optimization during long-term operation by constructing an energy efficiency autonomous learning framework, using imitation learning for initial policy training, adopting reinforcement learning methods to optimize the energy efficiency scheduling strategy, and introducing an adaptive tuning and energy efficiency dynamic adjustment mechanism.

[0269] Embodiment 2

[0270] Dynamic self-calibrating energy efficiency data acquisition module: Deploy sensors and a distributed acquisition network, and through calculating the energy efficiency change rate and adjusting the adaptive sampling frequency, combined with edge computing preprocessing, realize the acquisition of energy efficiency data;

[0271] Dynamic prediction module: Adopt adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling, and multi-level prediction fusion to realize dynamic energy efficiency prediction;

[0272] Dynamic load balancing module: Introduce energy efficiency-aware load evaluation metrics, and through task migration decision-making, dynamic weight adjustment, and global load balancing optimization, achieve maximum energy efficiency;

[0273] Resource scheduling optimization module: Construct an intelligent group decision-making model, and through dynamically weighing scheduling optimization, analyzing the stability of task allocation, and adaptive convergence adjustment, realize the global allocation of computing resources;

[0274] Adaptive optimization module: Based on the energy efficiency autonomous learning framework, use imitation learning for initial policy training, and use reinforcement learning to optimize the scheduling strategy, with adaptive tuning and dynamic adjustment to achieve adaptive optimization.

[0275] The modules in Embodiment 2 are used to implement the functions in Embodiment 1. This embodiment can be implemented by a system, which includes a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the energy efficiency data detection and processing method of the data center according to Embodiment 1 of the present application is realized. The system also includes a communication bus, a communication interface, and other components well-known to those skilled in the art. Their settings and functions are known in the art, so they will not be elaborated here.

[0276] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting and processing energy efficiency data of a data center, characterized in that: include: Step 1: Deploy sensors and distributed data collection networks, calculate the energy efficiency change rate and adjust the adaptive sampling frequency, combined with edge computing preprocessing, to achieve energy efficiency data collection; Step 2: Adopt adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling and multi-level prediction fusion to achieve dynamic energy efficiency prediction; Step 3: Introduce energy-efficiency-aware load evaluation indicators, combine task migration decisions, dynamic weight adjustment, and global load balancing optimization to maximize energy efficiency; Step 4: Build an intelligent group decision-making model to achieve global allocation of computing resources through dynamic trade-off scheduling optimization, task allocation stability analysis, and adaptive convergence adjustment; Step 5: Based on the energy efficiency autonomous learning framework, use imitation learning to perform initial strategy training, reinforcement learning to optimize the scheduling strategy, adaptive tuning and dynamic adjustment to achieve adaptive optimization.

2. The method for detecting and processing energy efficiency data of a data center according to claim 1, characterized in that: The step 1 comprises: Deploy energy efficiency sensors on key equipment in the data center, build a distributed data collection network, periodically collect power, load and timestamp data of the equipment and send it to the edge nodes; Calculate the rate of change of energy efficiency and determine whether the energy consumption status of the data center fluctuates drastically by setting thresholds; The sampling frequency is dynamically adjusted based on the rate of change of energy efficiency, the sampling interval is controlled by a nonlinear adjustment factor, and the exponentially weighted moving average smoothing process is performed to ensure that the sampling period adapts to load changes; Data preprocessing is performed at the edge node, abnormal data is screened and eliminated using local anomaly factors, and data is smoothed using sliding window mean filtering. Data is uploaded to the central server only after meeting the confidence level, reducing communication and computing overhead.

3. The method for detecting and processing energy efficiency data of a data center according to claim 2, characterized in that: The step 2 comprises: Dynamically adjust the prediction time window size according to the rate of change of energy efficiency, and use the exponential smoothing method to suppress the impact of short-term fluctuations on the window, ensuring that the window size adapts to the fluctuation of energy efficiency; Introduce short-term and long-term error feedback mechanisms to dynamically adjust the forecast model parameters. Combine short-term errors with fine-tuning model parameters, and long-term errors with correcting trend forecasts. Construct a feature vector based on the energy efficiency change rate, use a nonlinear autoregressive model for trend modeling, and introduce a periodic factor to adjust the model output to adapt to the periodic characteristics of data center energy consumption; By integrating short-term and long-term prediction results, dynamically adjusting weights, and optimizing the final prediction output based on short-term and long-term errors, dynamic prediction of energy efficiency data can be achieved.

4. The method for detecting and processing energy efficiency data of a data center according to claim 3, characterized in that: The step 3 comprises: Define energy-efficiency-aware load evaluation indicators, comprehensively calculate the CPU / GPU utilization, current power consumption, and task waiting time of the computing node, and standardize each indicator through the normalization method to achieve load measurement; Based on the task migration decision of energy efficiency optimization, the global energy efficiency optimization objective function is calculated, the minimum incremental energy consumption principle is followed when selecting the migration task, and constraints are set to ensure that the target node is not overloaded; Dynamic weight adaptive adjustment, dynamically update weight parameters according to historical error feedback, increase load balancing weight when system load is unbalanced, and increase energy consumption optimization weight when energy consumption exceeds the standard; Global load balancing optimization uses the gradient descent method to adjust the task allocation strategy, minimize load variance, and ensure the optimal overall energy efficiency of the data center.

5. The method for detecting and processing energy efficiency data of a data center according to claim 4, characterized in that: The step 4 comprises: Build an intelligent group decision-making model consisting of multiple intelligent agents. Each agent makes resource scheduling decisions based on local observations and global feedback, with the goal of minimizing the total energy consumption of the system and maximizing the overall computing efficiency. Implement scheduling optimization based on dynamic trade-offs, define the resource scheduling optimization objective function, introduce a global energy efficiency feedback adjustment strategy, and achieve resource competition and global optimal scheduling through a game mechanism between intelligent agents; Perform task allocation stability analysis and calculate the task migration stability index. If it exceeds the threshold, use the gradient descent method to adjust task scheduling and enhance stability. Perform adaptive convergence adjustment, dynamically calculate the convergence speed of the global energy efficiency target, adjust the decision step size of the intelligent agent according to the convergence situation, ensure the algorithm is highly adaptable under different load conditions, and realize intelligent scheduling optimization of data center computing resources.

6. The method for detecting and processing energy efficiency data of a data center according to claim 5, characterized in that: The step 5 comprises: Build an energy efficiency autonomous learning framework to define the state space of the data center, including business load, energy consumption, resource scheduling, and overall energy efficiency goals, and use a strategy generation network based on deep reinforcement learning for adaptive optimization; Use imitation learning to conduct initial strategy training, and use supervised learning methods to adjust strategy network parameters in combination with historical optimal scheduling data to improve strategy accuracy in the initial stage; Adopt reinforcement learning for autonomous optimization, use the policy gradient method to dynamically adjust the policy network, and combine the experience replay mechanism to improve the policy convergence speed, ensuring that the energy efficiency scheduling strategy adapts to real-time environmental changes; Implement adaptive tuning and dynamic adjustment of energy efficiency, dynamically adjust the learning rate based on target deviation, adapt to changes in business load in real time, and achieve long-term self-evolution optimization of the data center.

7. A data center energy efficiency data detection and processing system, characterized in that: include: Dynamic self-calibration energy efficiency data collection module: deploy sensors and distributed collection networks, calculate energy efficiency change rate and adjust adaptive sampling frequency, combined with edge computing preprocessing, to achieve energy efficiency data collection; Dynamic prediction module: Adopts adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling and multi-level prediction fusion to achieve dynamic energy efficiency prediction; Dynamic load balancing module: Introduces energy-efficiency-aware load evaluation indicators, combines task migration decisions, dynamic weight adjustment, and global load balancing optimization to maximize energy efficiency; Resource scheduling optimization module: builds an intelligent group decision-making model to achieve global allocation of computing resources through dynamic trade-off scheduling optimization, task allocation stability analysis and adaptive convergence adjustment; Adaptive optimization module: Based on the energy efficiency autonomous learning framework, it uses imitation learning to perform initial strategy training, reinforcement learning to optimize scheduling strategies, adaptive tuning and dynamic adjustment to achieve adaptive optimization.

Images