Server energy efficiency dynamic optimization management method and system

Abstract

The invention provides a server energy efficiency dynamic optimization management method and system, and relates to the technical field of energy efficiency optimizing.The method comprises the steps that historical operation state data of a server cluster is collected, and after data cleaning, standardization and division, a time sequence prediction model is trained to generate a load prediction model; predicting a target period workload based on the model, and combining an external influence factor and a business calendar; according to the predicted load and the task priority constraint, adopting a non-dominated sorting genetic algorithm to carry out multi-objective optimization solution, and generating a resource scheduling strategy; scheduling operation is executed, and real-time operation state data are collected to update historical data; the system comprises a data acquisition and preprocessing module, a load prediction module, a resource scheduling strategy generation module, a resource scheduling execution module and a real-time data updating module. According to the method, the energy efficiency of the server is dynamically optimized, the energy consumption is reduced, the resource utilization rate is improved, and the service quality of high-priority tasks is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of energy efficiency optimization technology, and in particular to a method and system for dynamic optimization management of server energy efficiency. Background Technology

[0002] As the core carrier of data processing and business operations, the energy efficiency of server clusters directly affects the operating costs, energy consumption, and total carbon emissions of data centers. It is a key indicator for measuring the greenness and intelligence of data centers, involving multiple technical dimensions such as load perception, resource scheduling, model prediction, and task management. Specifically, real-time load fluctuations of server clusters (such as CPU utilization and memory usage), the accuracy of load prediction, the matching degree between resource scheduling strategies and task priorities, and the efficiency of collaborative updates between historical operating data and real-time status data all significantly impact energy efficiency optimization results. Simultaneously, the stability of server operation at the hardware level and the adaptability of scheduling algorithms at the software level also impose multiple constraints on energy efficiency management. In related technologies, some solutions perform static resource scheduling by monitoring server load in real time, or formulate energy-saving strategies only for a single task type, in an attempt to reduce server energy consumption. However, this approach has obvious limitations: on the one hand, it lacks a load prediction mechanism based on historical operating status data, making it impossible to predict load changes in the target period in advance, resulting in insufficient resource supply during high-load periods and idle and wasted resources during low-load periods, exacerbating energy efficiency losses; on the other hand, it does not combine task priority for multi-objective optimization during resource scheduling, which easily leads to an imbalance problem of "heavy task execution and light energy efficiency optimization" or "heavy energy efficiency control and light task protection", making it difficult to balance business stability and energy efficiency improvement; in addition, most solutions do not establish a dynamic update cycle of real-time operating status data to historical data, causing the load prediction model to be unable to adapt to changes in server operating status, and the long-term energy efficiency optimization effect to continuously decay.

[0003] Therefore, there is an urgent need for a brand-new method and system for dynamic optimization and management of server energy efficiency to meet market demands. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for dynamic optimization and management of server energy efficiency, in order to solve the problems of insufficient prediction accuracy, single optimization target and lack of optimization in the prior art. The specific technical solution is as follows: This invention relates to a method for dynamic optimization and management of server energy efficiency, comprising the following steps: Step 1: Train the preset time series prediction model based on the historical operating status data of the server cluster to generate a load prediction model; Step 2: Based on the load prediction model, predict the server cluster workload for the target period and generate predicted workload data. Step 3: Based on the predicted workload data and combined with task priority constraints, perform multi-objective optimization to generate a resource scheduling strategy; Step 4: Perform resource scheduling operations on each server in the server cluster according to the resource scheduling policy, generate real-time running status data, and update historical running status data based on the real-time running status data.

[0005] Furthermore, the historical operating status data includes load data, hardware status data, energy consumption data, and time data; the load data includes CPU utilization, memory usage, hard disk I / O throughput, network bandwidth utilization, and task concurrency; the hardware status data includes CPU temperature, memory temperature, hard disk temperature, and server hardware configuration data; the energy consumption data is the server's real-time power; and the time data is the collection timestamp.

[0006] Furthermore, in step 1, the historical operating status data is preprocessed, including data cleaning, data standardization, and data partitioning. In data cleaning, missing values ​​in continuous data are filled using linear interpolation, missing values ​​in discrete data are filled using the mode, and outliers are identified using the 3σ principle and replaced with the median. Data standardization uses the Z-Score standardization method. Data partitioning divides the data into training set, validation set, and test set in a 7:2:1 ratio.

[0007] Furthermore, the time series prediction models preset in step 1 include ARIMA model, LSTM model and Attention-LSTM model; the model training uses root mean square error as the loss function, adjusts the parameters through Adam optimizer, and adopts an early stopping strategy to prevent overfitting.

[0008] Furthermore, step 2 also includes predicting the server cluster workload for the target period based on the load prediction model, combined with historical operating status data windows and external influencing factors; external influencing factors include future weather forecast data, business calendars, and known task schedules; the prediction process includes feature construction, with features covering time features, statistical features, and trend features; the prediction output includes a workload prediction sequence and a 95% confidence interval; the prediction results are used after destandardization, pruning, and smoothing.

[0009] Furthermore, the task priority constraints in step 3 include priority level classification and constraint execution rules; the priority levels are divided into five levels from P0 to P4; the constraint execution rules include resource allocation priority rules, server load rate constraint rules, task migration constraint rules, and resource exclusivity constraint rules; the multi-objective optimization algorithm adopts the non-dominated sorting genetic algorithm NSGA-II; the optimization objectives include minimizing energy consumption, minimizing response time, and maximizing task completion rate; the algorithm flow includes encoding and initializing the population, calculating the fitness function, non-dominated sorting and crowding calculation, selection, crossover and mutation operations, population update and iteration termination.

[0010] Furthermore, the encoding adopts real number encoding, and the individual dimension is the number of tasks × the number of servers; the fitness function is a weighted sum of normalized objective functions; the selection operation combines elite retention and roulette wheel strategy; the crossover operation adopts simulated binary crossover, and the mutation operation adopts polynomial mutation.

[0011] Furthermore, the resource scheduling operations in step 4 include task scheduling, CPU resource adjustment, memory resource adjustment, and disk I / O scheduling. Task scheduling is implemented through a cluster management platform, and sharding technology is used for tasks that are not 100% allocated. CPU resource adjustment uses cgroups to bind cores and set frequency modes. Memory resource adjustment limits usage through cgroups and enables big page optimization. Disk I / O scheduling selects algorithms and sets priorities based on disk type. Real-time running status data includes task running status, server resource status, and energy consumption data. Data collection frequency is set differently, and historical data is updated after cleaning and format unification. Update timing includes real-time updates after the target period ends and weekly regular updates, with data annotation and lifecycle management.

[0012] The present invention also relates to a system for the aforementioned server energy efficiency dynamic optimization management method, comprising: The data acquisition and preprocessing module 310 is used to acquire and preprocess historical and real-time operating status data. The load forecasting module 320 is used to train the load forecasting model and perform workload forecasting. The resource scheduling strategy generation module 330 is used to generate resource scheduling strategies based on predicted workload data and task constraints. Resource scheduling execution module 340 is used to execute resource scheduling operations according to resource scheduling strategy; The real-time data update module 350 is used to collect real-time running status data during the scheduling process and update historical running status data.

[0013] Furthermore, the data acquisition and preprocessing module 310 includes a data acquisition unit, a data cleaning unit, a data standardization unit, and a data partitioning unit; the load prediction module 320 includes a model training unit, a workload prediction unit, a prediction result post-processing unit, and an accuracy monitoring unit; the resource scheduling strategy generation module 330 includes a constraint management unit, a multi-objective optimization unit, an encoding and initialization unit, a fitness calculation unit, a non-dominated sorting and congestion calculation unit, a genetic operation unit, and an optimal strategy selection unit; the resource scheduling execution module 340 includes a task scheduling execution unit, a CPU resource adjustment unit, a memory resource adjustment unit, and a hard disk I / O scheduling unit; and the real-time data update module 350 includes a real-time data acquisition unit, a real-time data preprocessing unit, and a historical data update unit.

[0014] Furthermore, the system is integrated with the Kubernetes cluster management platform; the data acquisition unit collects data through hardware sensors, platform interfaces, and system tools; the model training unit uses the RMSE loss function and the Adam optimizer; the workload prediction unit integrates external factor features; the multi-objective optimization unit configures the NSGA-II algorithm parameters; and the resource scheduling execution unit uses cgroup and scheduler technology to implement resource control.

[0015] The beneficial effects of this invention are as follows: Traditional time series models in the prior art have poor adaptability to sudden load changes, which can easily lead to a disconnect between resource allocation and actual demand. Moreover, most solutions take minimizing energy consumption as the sole objective, without taking into account constraints such as task priority and heat dissipation costs, resulting in a significant sacrifice of service quality. In addition, traditional static strategies cannot dynamically adjust scheduling parameters based on real-time operating data, making it difficult to continuously improve energy efficiency. This invention, by combining load prediction with multi-objective optimization scheduling, realizes dynamic closed-loop management of server cluster energy efficiency, which can significantly reduce energy consumption, improve resource utilization, and ensure the service quality of high-priority tasks. Furthermore, through real-time data updates, the system continuously adapts and optimizes, achieving a comprehensive improvement in energy efficiency and performance.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the server energy efficiency dynamic optimization management method in an embodiment of the present invention; Figure 2 This is a schematic diagram of the server energy efficiency dynamic optimization management system in an embodiment of the present invention. Detailed Implementation

[0018] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0019] This invention provides a method for dynamic optimization and management of server energy efficiency, such as... Figure 1 As shown, it includes: Step 1: Train the preset time series prediction model based on the historical operating status data of the server cluster to generate a load prediction model, including training the preset time series prediction model after preprocessing. The historical operational status data consists of multi-dimensional time-series data characterizing the operational features of the server cluster, serving as the input foundation for model training. This includes load data, hardware status data, energy consumption data, and time data. Load data includes CPU utilization (average CPU core utilization, %), memory usage (percentage of used memory to total memory, %), disk I / O throughput (data read / write per second, MB / s), network bandwidth utilization (percentage of used bandwidth to total bandwidth, %), and task concurrency (number of currently running tasks, units). Hardware status data includes CPU temperature, memory temperature, and disk temperature (all average temperatures, ℃) and server hardware configuration data (number of CPU cores, memory capacity, and disk capacity, in cores, GB, and TB, respectively). Energy consumption data represents the server's real-time power (total input power, W). Time data is the collection timestamp (format: YYYY-MM-DDHH:MM:SS). Preprocessing involves handling the original historical operational data to ensure model training accuracy. This includes data cleaning, data standardization, and data partitioning. During data cleaning, missing values ​​in continuous data (e.g., CPU utilization) are filled using linear interpolation, while missing values ​​in discrete data (e.g., task concurrency) are filled using the mode of the preceding and following data. If the percentage of missing data in a given time period exceeds 20%, all data for that time period is removed. Outliers are identified using the 3σ principle (data values ​​exceeding the mean ± 3 standard deviations are considered outliers) and replaced with the median of the corresponding data field. Data formats are standardized (e.g., timestamps are standardized to "YYYY-MM-DDHH:MM:SS", power data retains one decimal place, and utilization data retains two decimal places). Data standardization uses the Z-Score standardization method, with the formula: ;x normFor the standardized data, x represents the original data, μ represents the mean of the data field, and σ represents the standard deviation of the data field to eliminate differences in dimensions. The preprocessed data is divided into a training set (for model parameter training), a validation set (for model hyperparameter tuning, such as learning rate and number of iterations), and a test set (for model prediction accuracy evaluation) in a 7:2:1 ratio. The preset time series prediction models are a set of models capable of handling dependencies in time series data. They are used to learn the patterns of load changes through historical data and include ARIMA models, LSTM models, and LSTM models with an attention mechanism (Attention-LSTM model): The ARIMA model (Autoregressive Integral Moving Average) uses three parameters—AR (autoregressive term), I (difference order), and MA (moving average term)—to characterize the trend and periodicity of linear time series data; the LSTM model (Long Short-Term Memory Network) is an improved version of the Recurrent Neural Network (RNN), which solves the gradient vanishing problem of RNNs by using gating units composed of input gates, forget gates, and output gates to capture long-term dependencies in time series data; the Attention-LSTM model introduces an attention mechanism on the basis of the LSTM model, assigning differentiated weights to the input data at different time steps, focusing on historical data that has a significant impact on the prediction results; The load prediction model is a trained model that meets preset accuracy requirements (e.g., mean absolute percentage error (MAPE) ≤ 8% on the test set). It outputs predicted workload values ​​(e.g., CPU utilization, memory usage, and task concurrency) for the server cluster within a future target period, providing core model support for workload prediction in step 2. During model training, the root mean square error (RMSE) is used as the loss function, as shown in the formula: Where n is the number of samples in the training set, y i This is the actual load data. The model is programmed with predicted load data, and model parameters are tuned using the Adam optimizer. The RMSE is evaluated using a validation set every 10 training iterations; if the validation set RMSE does not decrease for three consecutive iterations, an early stopping strategy is employed to avoid overfitting. The RMSE and MAPE of each model are calculated using the test set, as shown in the formula. The model with the smallest error is selected as the load prediction model.

[0020] Step 2: Based on the load prediction model, predict the server cluster workload for the target period and generate predicted workload data. This includes predicting the workload within the target period based on the load prediction model, combined with the historical operating status data window of the server cluster before the prediction start point and relevant external influencing factors, and outputting the predicted workload value for the target period and the corresponding prediction confidence interval. The prediction start point is the time point at which the prediction calculation is started, such as the current time T0, which is used to determine the end point of the historical data window and the start point of the target period. The historical running status data window contains the server cluster running data for the most recent period required for prediction. Its time length is T (e.g., T=1 hour), and the sampling frequency is consistent with step 1 (e.g., 1 time / 10 seconds). The data fields include: CPU utilization, memory usage, hard disk I / O throughput, network bandwidth utilization, number of concurrent tasks, CPU / memory / hard disk temperature, real-time power, and timestamp. External influencing factors are non-real-time monitoring data affecting future workloads, including future weather forecasts (outdoor temperature, wind speed), business calendars (weekdays / weekends, holidays), and known task schedules (such as batch calculation tasks, data backup windows). Data preprocessing includes: filling missing values ​​in historical operational status data windows (linear interpolation for continuous data, mode filling for discrete data), outlier removal or replacement (identifying outliers using the 3σ principle and replacing them with the median), data standardization (Z-score standardization using the mean μ and standard deviation σ calculated in step 1), and feature construction (such as time features: hour of day, week of week, holiday status; statistical features: mean, variance, peak value of the past 15 / 30 / 60 minutes; trend features: first difference, slope). The target period is the continuous future time period to be predicted, such as the next hour. The prediction step size is consistent with the sampling frequency (e.g., 10 seconds / step, 360 prediction points), configured to different durations according to actual scheduling needs. Prediction execution includes processing the preprocessed historical data... The input consists of window and external influencing factors. This input is fed into the load prediction model (e.g., Attention-LSTM model) obtained in step 1. A multi-step-ahead or recursive prediction method is used to output the workload prediction sequence within the target period. The predicted output fields correspond one-to-one with the input load fields, including CPU utilization, memory usage, disk I / O throughput, network bandwidth utilization, and task concurrency. The prediction confidence interval is a statistical interval that quantifies prediction uncertainty, such as a 95% confidence interval, calculated using the model output distribution (e.g., QuantileRegression, Monte Carlo Dropout, or ensemble of multiple models), and is used for robustness considerations in subsequent energy efficiency optimization scheduling. Post-processing includes de-standardizing the prediction results (restoring them to their original physical units), pruning predicted values ​​outside a reasonable range (e.g., correcting values ​​with CPU utilization >100% or <0), and smoothing (e.g., using moving averages to eliminate high-frequency jitter). Among them, accuracy monitoring involves comparing the subsequent actual operating data with the prediction results after the prediction is completed, calculating the prediction error index, and triggering the model update or retraining process in step 1 if the error exceeds a preset threshold of 15% to ensure the long-term effectiveness of the prediction model. Through this step, a refined workload prediction curve and confidence interval within the target period are provided for the energy efficiency optimization scheduling in step 3, thereby supporting the formulation of dynamic resource allocation and energy efficiency optimization strategies.

[0021] Step 3: Based on the predicted workload data and combined with task priority constraints, perform multi-objective optimization to generate a resource scheduling strategy. This includes performing multi-objective optimization using the Non-Dominated Sorting Genetic Algorithm (NSGA-II) based on the predicted workload data and combined with predefined task priority constraints. Among them, the task priority constraint is a constraint condition to ensure the service quality of high-priority tasks and avoid resource competition conflicts. It is constructed based on three quantifiable dimensions: the service level agreement (SLA) of the task, the urgency, and the resource requirements. Specifically, it includes priority level classification and constraint execution rules. The priority level division specifically divides tasks in the server cluster into five levels, from P0 (highest priority) to P4 (lowest priority). The quantitative definitions of each level are shown in Table 1.

[0022] The constraint enforcement rules include resource allocation priority rules, server load rate constraint rules, task migration constraint rules, and resource exclusivity constraint rules. The resource allocation priority rule specifically allocates server resources in the order of "P0→P1→P2→P3→P4," meaning that the resource needs of high-priority tasks are prioritized (e.g., after the CPU and memory requirements of task P0 are fully met, the remaining resources are allocated to task P1). The resource allocation amount must satisfy the condition "required resources for the task ≤ remaining server resources," and the formula for calculating remaining resources is "remaining server resources = total server resources - resources already allocated to higher-priority tasks." The server load rate constraint rule specifically states that for servers carrying tasks P0-P1, both their CPU and memory load rates must not exceed 80%; for servers carrying tasks P2-P4, both their CPU and memory load rates must not exceed 90%. The calculation formula is "load rate = (allocated resources / total server resources) × 100%" (e.g., CPU load rate = allocated CPU cores / total server CPU cores × 100%). The specific task migration constraint rule is that if the load rate of a server exceeds the corresponding priority constraint threshold (e.g., the CPU load rate of the server where task P0 resides reaches 85%), low-priority tasks are migrated to healthy servers with a load rate < 70% in the order of "P4 → P3 → P2". During the migration process, the task interruption time is ≤ 500ms (P0-P1 tasks are prohibited from migration). The specific resource exclusivity constraint rule is that tasks P0 occupy dedicated server resources (e.g., dedicated CPU cores, independent memory areas). These dedicated resources are marked with a "P0 Exclusive" identifier using Linux cgroup technology, and the resource allocation logic for other priority tasks (P1-P4) automatically filters out these dedicated resources. Among them, multi-objective optimization algorithms can find Pareto optimal solutions among multiple objectives. This embodiment uses the Non-Dominated Sorting Genetic Algorithm (NSGA-II). The solution process includes encoding and initializing the population, calculating the fitness function, non-dominated sorting and crowding calculation, selection, crossover and mutation operations, population update and iteration termination, and selection of the optimal resource scheduling strategy. The specific solution process and parameter definitions are as follows: S301. Encoding and Population Initialization; Specific encoding method: Real number encoding is used, each individual corresponds to one resource scheduling scheme, and the individual dimension is the dimension of the task allocation matrix "number of tasks (N) × number of servers (M)", with individual element x... i,j This represents the resource allocation ratio of the i-th task to the j-th server (the sum of the resource allocation ratios for a single task is 1); Population initialization: Set the population size to 100, and randomly generate 100 individuals that meet the task priority constraints (e.g., task P0 is only allocated to servers with a load rate < 60%, and x...). i,j (Ensuring the resources required for the task match the remaining resources on the server). S302. Calculate the fitness function: The objective function is defined as follows: energy minimization objective, and the formula used is... Where E is the total energy consumption of the cluster (unit: Wh), M is the number of servers, and P is the total energy consumption of the cluster. j The operating power of the j-th server (in W) is obtained by fitting historical power data from step 1. j =a×L j +b, L j Let T be the load rate of the j-th server, a=2.5, b=150, and T be the fitting coefficients calculated using a linear regression tool. j The target period duration (unit: hours, e.g., 15 minutes, then T) j =0.25); The objective of minimizing response time is achieved using the following formula: Where R is the weighted average response time (in ms), N is the number of tasks, and w i The weight of the i-th task (positively correlated with priority, P0 task w) i =5、P1 task w i =4、P2 task w i =3、P3 task w i =2, P4 task w i =1, R i Let be the response time of the i-th task; The formula used to maximize task completion rate is: Where C is the task completion rate (unit: %). For indicator function (t) i <T) when II=1, otherwise II=0), t i The estimated completion time for task i (in hours) is calculated based on the task data volume and server processing capacity. For example, task P2 with 10GB of data on a 2-core CPU server... i =0.2), T is the target cycle duration (unit: h). Normalization: Due to the differences in the dimensions of the three objective functions, the function values ​​are normalized. The energy consumption normalization formula is as follows: E min and E max These represent the minimum and maximum energy consumption values, respectively. The fitness function is a weighted sum of three normalized objective functions (weights are set according to business requirements, such as energy consumption weight 0.4, response time weight 0.3, and task completion rate weight 0.3), and the formula is as follows: ; where C norm This represents the normalized fitness that maximizes the task completion rate.

[0023] S303. Non-dominated ranking and crowding calculation: Non-dominated ranking specifically involves dividing the population front into fronts based on "dominance relationships." The first front is the Pareto optimal solution, the second front is dominated only by individuals in the first front, and so on. "Dominance relationship" is defined as follows: if individual A's energy consumption ≤ individual B, response time ≤ individual B, task completion rate ≥ individual B, and at least one objective is strictly better than B's, then A dominates B. Crowding calculation is used to assess the distribution density of individuals in the fronts, avoiding optimal solution concentration. The calculation formula is as follows: Where k=1,2,3 correspond to the energy consumption, response time, and task completion rate targets, respectively, and x i+1 x i-1 Let f be the neighboring individuals of individual i in the frontier, and let f be the value of the target k; the crowding degree of individuals at both ends of the frontier is set to infinity, and the crowding degree of individuals in the middle is calculated according to the above formula; S304. Selection, Crossover, and Mutation Operations: The selection operation employs an "elite retention + roulette wheel selection" strategy. First, all individuals at the first frontier (maximum 50) are retained. Any remaining individuals are selected from the second frontier in descending order of crowding, resulting in 50 individuals for the crossover operation. The roulette wheel selection probability is positively correlated with the individual's fitness value, calculated as "selection probability = individual fitness value / total population fitness value". The crossover operation uses simulated binary crossover (SBX), with a crossover probability of 0.8 and a crossover distribution index of 20. The 50 selected individuals are paired to generate 50 new individuals, using the crossover formula: β is the crossover factor (calculated by the crossover distribution index; when the crossover distribution index = 20, β ranges from [0, 2.5]); the mutation operation specifically uses multinomial mutation, with a mutation probability set to 0.01 and a mutation distribution index set to 20; for the 50 new individuals generated by crossover, 1-2 mutation positions are randomly selected, and the mutation formula is... , where x max =1、x max =0 indicates the variable boundary. The mutation factor (value range [0, 0.1]) is used to check whether the individual meets the task priority constraint after mutation; if not, it is regenerated. S305. Population update and iteration termination; Population merging specifically involves merging the parent population (100 individuals) and the offspring population (50 new individuals) into 150 individuals; Population selection specifically involves re-performing non-dominated sorting and crowding calculation on the merged population, selecting the top 100 individuals (prioritizing individuals with high frontier rank and high crowding) to form a new parent population; Iteration termination specifically involves repeating steps 2-5 until the number of iterations reaches 200, or the difference in optimal fitness between two adjacent iterations is <0.001, at which point iteration stops; S306. Optimal resource scheduling strategy selection; The selection criteria are as follows: From the first frontier individuals, select the individual with the largest fitness function value; If there are multiple individuals with the same fitness, select the individual with the lowest energy consumption as the final resource scheduling strategy. Among them, the resource scheduling strategy is the final output of the multi-objective optimization solution. Its format is adapted to the resource scheduling interface of the cluster management platform (such as Kubernetes). The resource scheduling strategy includes a task-server allocation table and a server resource adjustment scheme: the task-server allocation table records the resource allocation object and core running parameters of each task, as shown in Table 2 (5 tasks and 3 servers within the target period).

[0024] Server resource adjustment plan: Based on the task allocation results, clarify the resource allocation details and operation commands for each server, as shown in Table 3 (total CPU cores: 32, total memory: 128GB).

[0025] Step 4: Perform resource scheduling operations on each server in the server cluster according to the resource scheduling policy, generate real-time running status data, and update historical running status data based on the real-time running status data, including performing resource scheduling operations according to the resource scheduling policy; at the same time, update the historical running status data of the server cluster by collecting real-time running status data during the scheduling process, after preprocessing. Resource scheduling is the specific execution action that implements the resource scheduling strategy. It is based on the cluster management platform (such as Kubernetes) and the underlying hardware control capabilities. The operation content is completely matched with the "task-server allocation table" and "server resource adjustment plan" output in step 3, including four dimensions: task scheduling, CPU resource adjustment, memory resource adjustment, and disk I / O scheduling. The execution method, tools, and parameter definitions for each dimension are as follows: Task scheduling execution: Execution subject: Task scheduling module of the cluster management platform (such as kube-scheduler in Kubernetes); Execution logic: According to the task allocation ratio in the "task-server allocation table", the task scheduling is executed through the platform API interface (such as the kubectlapply-ftask-deploy.yaml command in Kubernetes). Tasks are deployed to designated servers. For tasks with a non-100% allocation ratio (e.g., T2 is allocated to S0 at 50% and S1 at 50%), task sharding technology (e.g., Spark-based task sharding algorithm) is used to split the task into subtasks matching the allocation ratio (e.g., T2 is split into two subtasks, T2-1 and T2-2, deployed to S0 and S1 respectively). Fault rescheduling rules: After task deployment, the task running status (running / completed / failed) is monitored in real time through platform monitoring interfaces (e.g., Kubernetes' kubectllogs[task ID]). If a task fails, rescheduling is performed according to priority: P0-P1 tasks are immediately (delay ≤ 1s) reassigned to healthy servers with a load rate < 70%, and P2-P4 tasks are rescheduled after a 5-minute delay. During rescheduling, the original faulty server is skipped (fault status is reported by BMC). CPU resource adjustment execution: Execution entities: the server operating system's CPU scheduler (such as Linux's CFS scheduler) and hardware control module (BMC); Execution logic: Core allocation: using cgroup technology to bind the CPU cores specified in the "Server Resource Adjustment Scheme" to the corresponding task processes; Frequency mode adjustment: setting the CPU frequency according to task priority in governors mode (performance mode: running at the highest frequency to ensure response time; powersave mode: running at a low frequency to reduce energy consumption): servers carrying P0-P1 tasks are set to performance mode; servers carrying P3-P4 tasks are set to powersave mode; Memory resource adjustment execution: Execution subject: the server operating system's memory management module (such as Linux's cgroup memory subsystem); Execution logic: Memory limit: limit the maximum memory usage of tasks through cgroup technology; Huge page optimization: enable memory huge pages (HugePages) technology for tasks with high memory requirements (such as P0 task T0), configure huge page parameters by modifying the / etc / sysctl.conf file (e.g., vm.nr_hugepages=32768, corresponding to 2MB huge pages totaling 64GB), and bind huge pages to tasks by echo [huge page path]> / sys / fs / cgroup / memory / [task ID] / memory.hugetlb.page_size; Disk I / O scheduling execution: Execution subject: server operating system I / O scheduler; Execution logic: Scheduler algorithm selection: set I / O scheduler algorithm according to disk type: SSD disk uses mq-deadline scheduler; HDD disk uses noop scheduler; I / O priority setting: set I / O class and priority level according to task priority through ionice command: P0-P1 tasks are set to realtime class (1), priority 0 (highest); P2-P3 tasks are set to best-effort class (2), priority 3; P4 task is set to best-effort class (2), priority 7 (lowest); Among them, real-time operational status data refers to the server and task operation data collected in real time during resource scheduling, including task operation status data, server resource status data, and energy consumption data. Real-time operational status data is the core basis for evaluating scheduling effectiveness and updating historical data. Its collection subject, content, frequency, and format are fully compatible with the "historical operational status data" in step 1, and are specifically defined as follows: Data collected includes three main categories: task execution status data, server resource status data, and energy consumption data. Each data field corresponds one-to-one with the historical data fields from step 1, as detailed below: Task running status data: Task ID, Task priority, Task progress (unit: %), Runtime (unit: s), Task completion status (completed / incomplete / failed), collected from the cluster management platform monitoring module; Server resource status data: CPU utilization (%), memory usage (%), hard disk I / O throughput (MB / s), network bandwidth utilization (%), CPU / memory / hard disk temperature (°C), collected from software acquisition units and hardware acquisition units (such as temperature sensors). Energy consumption data: Real-time server power (W), total energy consumption within the target period (Wh), collected from hardware acquisition units (current / voltage sensors, BMC's IPMI interface). Data collection frequency: Set differentiated frequencies based on data type to ensure a balance between data real-time performance and resource consumption. Task running status data: 1 time / 10 seconds (such as task progress, running time); Server resource status data: once every 5 seconds (such as CPU utilization, disk I / O throughput). Energy consumption data: 1 time / 1 second (e.g., real-time power); Data format: Time-series data format is adopted. Each data entry contains "timestamp (format: YYYY-MM-DDHH:MM:SS), device ID (server ID / task ID), data type (such as CPU utilization), data value, and data quality identifier (0=normal, 1=abnormal)". Example data are "2025-01-01 10:00:05,S0,CPU utilization, 72.3%,0" and "2025-01-01 10:00:05,T0,task progress, 12%,0". Real-time data preprocessing is a necessary step to ensure that real-time operational status data can be used to update historical data. This includes data cleaning and format standardization. The processing logic is completely consistent with the "historical operational status data preprocessing" in step 1, to avoid data format or quality issues affecting subsequent model training. Data Cleaning: Missing Value Handling: Continuous data (such as CPU utilization, real-time power) is filled using linear interpolation, while discrete data (such as task completion status, data quality indicators) is filled using the mode of the previous and next 5 data points; if the proportion of missing data in a certain minute is greater than 20%, then all data in that minute is removed; Outlier Handling: Outliers (data values ​​exceeding the range of "mean ± 3 × standard deviation") are identified using the 3σ principle, and outliers are replaced with the median of the corresponding data field; for example, if the mean of the real-time power of server S0 is 380W and the standard deviation is 20W, then power data exceeding 380±60W (i.e., <320W or >440W) are judged as outliers and replaced with 380W (median); Standardized format: Timestamps should be formatted as “YYYY-MM-DDHH:MM:SS”, padded with zeros if necessary (e.g., “2025-01-01 9:5:3” should be corrected to “2025-01-01 09:05:03”); Numerical data should retain the same number of decimal places as in step 1: power data should retain 1 decimal place, utilization rate data should retain 2 decimal places, and temperature data should retain 1 decimal place. Among them, updating historical operational status data is the core link to achieve "closed-loop management," including update timing, update content, and data lifecycle management. Preprocessed real-time operational status data is appended to the historical data set and labeled and managed for lifecycle, ensuring that the updated data is directly used for iterative optimization of the load prediction model in step 1. Specific update rules are as follows: Update Timing: Real-time Update: When the target period ends (e.g., if the target period is 15 minutes, ending at 10:15), the preprocessed real-time data within that period is immediately appended to the historical data set (stored in the historical database corresponding to step 1, such as InfluxDB); Regular Update: Every Sunday at 4:00 AM, historical data is organized and optimized, including deleting duplicate data (based on "timestamp + device ID + data type"), correcting labeling errors (e.g., re-validating data mistakenly labeled as "abnormal"), and establishing timestamps. A composite index with the device ID (to improve subsequent query efficiency); Update content: Add data: Append the real-time running status data (task running status, server resource status, energy consumption data) within the target period to the corresponding data table according to the table structure of the historical data in step 1 (as shown in Table 1); Association annotation: Add association information annotations to the newly added historical data, including "the predicted workload data ID of the corresponding target period (e.g., P2025010101), resource scheduling policy ID (e.g., S2025010101), scheduling effect indicators (task completion rate, actual energy consumption)", with an example annotation format of "historical data time period: 2025-01-01 10:00-10:15, predicted CPU utilization: 78.5%, actual CPU utilization: 72.3%, scheduling policy ID: S2025010101, task completion rate: 100%, actual energy consumption: 95Wh"; Data lifecycle management: Short-term data (storage duration ≤ 1 month): stored in a high-performance time-series database (such as InfluxDB), supporting high-frequency queries (for incremental training of load prediction models within the next 1-2 weeks); Medium-term data (1 month < storage duration ≤ 6 months): migrated to a general relational database (such as MySQL) for periodic load analysis (such as weekly and monthly load trend statistics); Long-term data (6 months < storage duration ≤ 3 years): compressed and stored in low-cost object storage (such as AWS S3) for annual energy efficiency assessment and technology improvement analysis; Data stored for more than 3 years will be deleted or migrated to offline tape storage after being confirmed by the administrator as having no reuse value.

[0026] The working principle and beneficial effects of the above technical solution are as follows: This embodiment of the invention trains a time-series prediction model by collecting historical operating status data of the server cluster, generating a high-precision load prediction model to predict the workload of future target periods. Subsequently, based on the prediction results and combined with task priority constraints, a multi-objective optimization algorithm is used to solve the resource scheduling strategy, balancing multiple objectives such as energy consumption, response time, and task completion rate. Finally, scheduling operations are executed, and operating data is collected in real time to update the historical database, forming a closed-loop management system. Its beneficial effects lie in dynamically optimizing server energy efficiency, significantly reducing energy consumption while improving resource utilization, ensuring the service quality of high-priority tasks, and enabling the system to adapt to load changes through continuous data updates, achieving long-term stable energy efficiency improvement.

[0027] This invention provides a server energy efficiency dynamic optimization and management system, such as... Figure 2 As shown, it includes: a data acquisition and preprocessing module 310, a load prediction module 320, a resource scheduling strategy generation module 330, a resource scheduling execution module 340, and a real-time data update module 350. The data acquisition and preprocessing module 310 is used to collect historical and real-time operating status data of the server cluster and preprocess the collected data. This module includes a data acquisition unit 311, a data cleaning unit 312, a data standardization unit 313, and a data partitioning unit 314. The data acquisition unit 311 is responsible for collecting historical and real-time operating status data from data sources such as hardware sensors and the cluster management platform of the server cluster. The collected data includes load data, hardware status data, energy consumption data, and time data. The data cleaning unit 312 performs missing value imputation and outlier handling on the collected data. For continuous data, linear interpolation is used to imput missing values; for discrete data, the mode is used to imput missing values; outliers are identified using the 3σ principle and replaced with the median. The data standardization unit 313 uses the Z-Score standardization method to standardize the data and eliminate dimensional differences. The data partitioning unit 314 divides the historical data into a training set, a validation set, and a test set in a 7:2:1 ratio. The load prediction module 320 is used to train a load prediction model based on preprocessed historical operating status data and to predict the server cluster workload for a target period using the load prediction model. This module includes a model training unit 321, a workload prediction unit 322, a prediction result post-processing unit 323, and an accuracy monitoring unit 324. The model training unit 321 trains a preset time series prediction model using a training set. The preset time series prediction models include ARIMA, LSTM, and Attention-LSTM models. Hyperparameters are adjusted using a validation set, and an early stopping strategy is employed to avoid overfitting. The test set evaluates the model accuracy; the workload prediction unit 322, based on the trained workload prediction model, combines historical operating status data windows and external influencing factors, including weather forecasts, business calendars, etc., and outputs the target period's workload prediction value and confidence interval using a single multi-step or recursive multi-step prediction method; the prediction result post-processing unit 323 performs de-standardization, pruning, and smoothing on the prediction results. Pruning is used to correct values ​​that exceed the reasonable range, and smoothing uses methods such as moving averages; the accuracy monitoring unit 324 compares the actual operating data with the prediction results, calculates the prediction error, and triggers model retraining if the error exceeds a threshold. The resource scheduling strategy generation module 330 is used to generate resource scheduling strategies based on predicted workload data and task priority constraints using a multi-objective optimization algorithm. This module includes a constraint management unit 331, a multi-objective optimization unit 332, an encoding and initialization unit 333, a fitness calculation unit 334, a non-dominated sorting and congestion calculation unit 335, a genetic operation unit 336, and an optimal strategy selection unit 337. The constraint management unit 331 manages task priority constraints, including classifying priorities (from P0 to P4) and formulating constraint execution rules, which cover resource allocation priority, load factor constraints, task migration constraints, and resource exclusivity constraints. The multi-objective optimization unit 332 uses the NSGA-II algorithm to solve multi-objective optimization problems. The objectives include minimizing energy consumption, minimizing response time, and maximizing task completion rate. The encoding and initialization unit 333 uses real-number encoding to encode the resource scheduling scheme and randomly generates an initial population that meets task priority constraints. The fitness calculation unit 334 calculates the fitness function value for each individual, which is a weighted sum of energy consumption, response time, and task completion rate. The non-dominated sorting and crowding calculation unit 335 performs non-dominated sorting and crowding calculation on the population. The genetic operation unit 336 performs selection, crossover, and mutation operations. The selection operation uses an elite retention combined with roulette wheel selection, the crossover operation uses simulated binary crossover, and the mutation operation uses polynomial mutation. The optimal strategy selection unit 337 selects the individual with the highest fitness from the Pareto front as the final resource scheduling strategy. The resource scheduling and execution module 340 is used to execute resource scheduling operations according to the resource scheduling policy. This module includes a task scheduling and execution unit 341, a CPU resource adjustment unit 342, a memory resource adjustment unit 343, and a disk I / O scheduling unit 344. The task scheduling and execution unit 341 deploys tasks to designated servers through a cluster management platform (such as Kubernetes), and uses task sharding technology for tasks with a non-100% allocation ratio. The CPU resource adjustment unit 342 uses cgroup technology to bind CPU cores and sets the CPU frequency mode according to task priority. The CPU frequency mode includes performance mode and powersave mode. The memory resource adjustment unit 343 limits memory usage through cgroup and enables big page optimization for tasks with high memory requirements. The disk I / O scheduling unit 344 sets the I / O scheduler algorithm according to the disk type. The MQ-deadline algorithm is used for SSDs and the noop algorithm is used for HDDs. At the same time, I / O priority is set according to task priority. The real-time data update module 350 is used to collect real-time running status data during the scheduling process and update historical running status data after preprocessing. This module includes a real-time data acquisition unit 351, a real-time data preprocessing unit 352, and a historical data update unit 353. The real-time data acquisition unit 351 collects task running status, server resource status, and energy consumption data at different frequencies. The real-time data preprocessing unit 352 cleans and unifies the format of the real-time data. The cleaning operation includes filling in missing values ​​and handling outliers. The historical data update unit 353 appends the preprocessed real-time data to the historical database and annotates and manages the data's lifecycle, which covers short-term, medium-term, and long-term storage.

[0028] As an optional implementation of this disclosure, the data acquisition unit 311 is specifically used to acquire server hardware status data and energy consumption data through hardware sensors. The hardware status data includes CPU temperature, memory temperature, etc., and the energy consumption data includes real-time power. It also acquires task running status data, such as task progress and runtime, through the cluster management platform, and acquires resource usage data, such as CPU utilization and memory occupancy, through the operating system interface.

[0029] As an optional implementation of this disclosure, the model training unit 321 is specifically used to use the root mean square error (RMSE) as the loss function and adjust the model parameters through the Adam optimizer; evaluate the RMSE with the validation set every 10 training iterations, and adopt an early stopping strategy if the validation set RMSE does not decrease for 3 consecutive iterations; calculate the mean absolute percentage error (MAPE) through the test set, and select the model with MAPE ≤ 8% as the load prediction model.

[0030] As an optional implementation of this disclosure, the workload prediction unit 322 is specifically used to use future weather forecast data, business calendar, and known task schedules as external influencing factors; in the feature construction stage, time features, statistical features, and trend features are constructed. Time features include hours of day, week of week, etc., statistical features include the mean and variance of the past 15 minutes, 30 minutes, and 60 minutes, etc., and trend features include first-order differences; the prediction confidence interval is calculated using the Quantile Regression or Monte Carlo Dropout method.

[0031] The server energy efficiency dynamic optimization management system provided in this disclosure achieves comprehensive collection and standardized processing of server operating data through a data acquisition and preprocessing module, providing a high-quality data foundation for subsequent analysis. It enhances the accuracy and reliability of workload prediction by combining multiple time series models and external influencing factors through a load prediction module, while ensuring model applicability through precision monitoring. The resource scheduling strategy generation module generates optimal scheduling strategies that balance energy consumption, response time, and task completion rate based on multi-objective optimization algorithms and task priority constraints. The resource scheduling execution module precisely allocates resources such as CPU, memory, and hard disk I / O, fully utilizing hardware performance and reducing energy consumption. The real-time data update module enables dynamic data iteration, ensuring the timeliness and continuity of system decisions. The overall system forms a closed loop of "data acquisition - predictive analysis - strategy generation - execution scheduling - data update," effectively achieving dynamic optimization of server energy efficiency, improving task processing efficiency while reducing energy consumption.

[0032] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the server energy efficiency dynamic optimization management method.

[0033] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the server energy efficiency dynamic optimization management method when running.

[0034] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0035] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the server energy efficiency dynamic optimization management method.

[0036] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above embodiments of the server energy efficiency dynamic optimization management method.

[0037] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for dynamic optimization and management of server energy efficiency, characterized in that, Includes the following steps: Step 1: Train the preset time series prediction model based on the historical operating status data of the server cluster to generate a load prediction model; Step 2: Based on the load prediction model, predict the server cluster workload for the target period and generate predicted workload data. Step 3: Based on the predicted workload data and combined with task priority constraints, perform multi-objective optimization to generate a resource scheduling strategy; Step 4: Perform resource scheduling operations on each server in the server cluster according to the resource scheduling policy, generate real-time running status data, and update historical running status data based on the real-time running status data.

2. The server energy efficiency dynamic optimization management method as described in claim 1, characterized in that, The historical operating status data includes load data, hardware status data, energy consumption data, and time data; the load data includes CPU utilization, memory usage, hard disk I / O throughput, network bandwidth utilization, and task concurrency; the hardware status data includes CPU temperature, memory temperature, hard disk temperature, and server hardware configuration data; the energy consumption data is the server's real-time power; and the time data is the collection timestamp.

3. The server energy efficiency dynamic optimization management method as described in claim 1, characterized in that, Step 1 involves preprocessing historical operational data, including data cleaning, data standardization, and data partitioning. During data cleaning, missing values ​​in continuous data are filled using linear interpolation, missing values ​​in discrete data are filled using the mode, and outliers are identified using the 3σ principle and replaced with the median. Data standardization uses the Z-Score standardization method. Data partitioning divides the data into training, validation, and test sets in a 7:2:1 ratio.

4. The server energy efficiency dynamic optimization management method as described in claim 1, characterized in that, The time series prediction models preset in step 1 include ARIMA model, LSTM model and Attention-LSTM model; the model training uses root mean square error as the loss function, adjusts the parameters through Adam optimizer, and adopts an early stopping strategy to prevent overfitting.

5. The server energy efficiency dynamic optimization management method as described in claim 1, characterized in that, Step 2 further includes predicting the server cluster workload for the target period based on the load prediction model, combined with historical operating status data windows and external influencing factors; external influencing factors include future weather forecast data, business calendars, and known task schedules; the prediction process includes feature construction, and the features cover time features, statistical features, and trend features; The forecast output includes a workload forecast sequence and a 95% confidence interval; The prediction results are used after inverse standardization, cropping, and smoothing.

6. The server energy efficiency dynamic optimization management method as described in claim 1, characterized in that, The task priority constraints in step 3 include priority level classification and constraint execution rules; the priority levels are divided into five levels from P0 to P4; the constraint execution rules include resource allocation priority rules, server load rate constraint rules, task migration constraint rules, and resource exclusivity constraint rules; the multi-objective optimization algorithm adopts the non-dominated sorting genetic algorithm NSGA-II; the optimization objectives include minimizing energy consumption, minimizing response time, and maximizing task completion rate; the algorithm flow includes encoding and initializing the population, calculating the fitness function, non-dominated sorting and crowding calculation, selection, crossover and mutation operations, population update and iteration termination.

7. The server energy efficiency dynamic optimization management method as described in claim 6, characterized in that, The encoding uses real numbers, and the individual dimension is the number of tasks × the number of servers; the fitness function is a weighted sum of normalized objective functions; the selection operation combines elite retention and roulette wheel strategies; the crossover operation uses simulated binary crossover, and the mutation operation uses polynomial mutation.

8. The server energy efficiency dynamic optimization management method as described in claim 1, characterized in that, The resource scheduling operations in step 4 include task scheduling, CPU resource adjustment, memory resource adjustment, and disk I / O scheduling. Task scheduling is implemented through a cluster management platform, and sharding technology is used for tasks that are not 100% allocated. CPU resource adjustment uses cgroups to bind cores and set frequency modes. Memory resource adjustment limits usage through cgroups and enables big page optimization. Disk I / O scheduling selects algorithms and sets priorities based on disk type. Real-time running status data includes task running status, server resource status, and energy consumption data. Data collection frequency is set differently, and historical data is updated after cleaning and format unification. Update timing includes real-time updates after the target period ends and weekly regular updates, with data annotation and lifecycle management.

9. A system for the server energy efficiency dynamic optimization management method according to any one of claims 1-8, characterized in that the system include: The data acquisition and preprocessing module 310 is used to acquire and preprocess historical and real-time operating status data. The load forecasting module 320 is used to train the load forecasting model and perform workload forecasting. The resource scheduling strategy generation module 330 is used to generate resource scheduling strategies based on predicted workload data and task constraints. Resource scheduling execution module 340 is used to execute resource scheduling operations according to resource scheduling strategy; The real-time data update module 350 is used to collect real-time running status data during the scheduling process and update historical running status data.

10. The server energy efficiency dynamic optimization management system as described in claim 9, characterized in that, The data acquisition and preprocessing module 310 includes a data acquisition unit, a data cleaning unit, a data standardization unit, and a data partitioning unit; the load prediction module 320 includes a model training unit, a workload prediction unit, a prediction result post-processing unit, and an accuracy monitoring unit; the resource scheduling strategy generation module 330 includes a constraint management unit, a multi-objective optimization unit, an encoding and initialization unit, a fitness calculation unit, a non-dominated sorting and congestion calculation unit, a genetic operation unit, and an optimal strategy selection unit; the resource scheduling execution module 340 includes a task scheduling execution unit, a CPU resource adjustment unit, a memory resource adjustment unit, and a hard disk I / O scheduling unit; and the real-time data update module 350 includes a real-time data acquisition unit, a real-time data preprocessing unit, and a historical data update unit.

Images