Resource scheduling based high-density server cluster energy efficiency optimization method

Abstract

The present application relates to the technical field of high-density server energy efficiency optimization, in particular to a high-density server cluster energy efficiency optimization method based on resource scheduling, comprising: collecting multi-dimensional running state data of each computing node in the cluster in real time, and inputting the data into a pre-constructed load characteristic analysis model to generate node load characteristic values representing the load degree of the nodes; according to the node load characteristic values and the cluster energy efficiency optimization target, establishing a global resource scheduling model with the total power consumption of the cluster as the objective function and the resource capacity and service quality requirements as the constraints; solving the model to obtain an optimization resource scheduling strategy that explicitly specifies the tasks to be migrated, target nodes and migration timing; generating control instructions according to the strategy and issuing the instructions to the underlying resource manager to perform dynamic migration of tasks and adjustment of node states. The present application realizes fine and dynamic optimization management of the energy efficiency of the cluster.

Description

Technical Field

[0001] This invention relates to the field of energy efficiency optimization technology for high-density servers, and in particular to a method for optimizing the energy efficiency of high-density server clusters based on resource scheduling. Background Technology

[0002] The field of energy efficiency management for high-density server clusters. Current mainstream energy efficiency optimization methods primarily rely on single indicators such as CPU utilization to determine load, and then implement task scheduling or node power consumption control based on this. These methods treat load assessment and resource scheduling as two relatively independent processes, resulting in relatively simple assessment models and scheduling objectives mainly focused on resource utilization. Existing technologies have limitations; the simple thresholds or static weighting methods they employ fail to accurately reflect the true overall state of servers under mixed loads, leading to biased assessment results. Load balancing-oriented scheduling strategies generally lack closed-loop optimization of overall power consumption, typically only controlling the switching of idle nodes after task placement, which is a reactive measure and cannot proactively avoid energy efficiency bottlenecks in scheduling decisions.

[0003] The key problem this invention aims to solve lies in achieving precise load quantification and process-oriented control of scheduling. It requires establishing an analytical model that integrates multi-dimensional real-time data to output unified load characteristic values, overcoming the bias of single-indicator evaluation; and constructing a scheduling model with total system power consumption as the direct optimization target, whose output should guide the specific execution sequence of task migration, thereby extending energy management from the resource allocation level to the dynamic operation process level, achieving refined energy efficiency control. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a high-density server cluster energy efficiency optimization method based on resource scheduling.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a high-density server cluster energy efficiency optimization method based on resource scheduling, comprising:

[0006] Multi-dimensional operational status data of each computing node in a high-density server cluster is collected in real time. The multi-dimensional operational status data is input into a pre-built load characteristic analysis model. Based on the load characteristic analysis model, the real-time workload of each computing node is quantitatively evaluated, and node load characteristic values ​​that characterize the load level of the node are generated.

[0007] Based on the node load characteristics and the pre-set cluster energy efficiency optimization target, a global resource scheduling model is established. The global resource scheduling model takes minimizing the total power consumption of the cluster as the objective function and the resource capacity of the computing nodes and the service quality requirements of the tasks as constraints.

[0008] Solve the global resource scheduling model to obtain an optimized resource scheduling strategy. The optimized resource scheduling strategy specifically specifies the computing tasks to be migrated, the target computing nodes, and the execution sequence of task migration.

[0009] Based on the optimized resource scheduling strategy, specific resource scheduling control instructions are generated and sent to the underlying resource manager of the high-density server cluster. The underlying resource manager then performs dynamic migration of computing tasks and adjustment of the working status of computing nodes.

[0010] As a further aspect of the present invention, the real-time acquisition of multi-dimensional operational status data of each computing node in the high-density server cluster specifically includes:

[0011] The multi-dimensional operating status data includes at least the CPU utilization, memory usage, input / output throughput, and current operating temperature of the computing node;

[0012] By deploying a monitoring agent on each computing node, the hardware performance counter readings of the computing node are periodically obtained at a preset sampling period. The hardware performance counter readings include the number of active cycles of each core of the central processing unit, the number of cache misses, and the memory controller access latency.

[0013] At the same time, the system call interface of the computing node operating system is invoked to obtain process-level resource consumption statistics, which include the CPU time slice occupancy, physical memory resident set size, number of disk read / write operations, and number of network connections for each running process.

[0014] The physical environment monitoring data is read through the sensor interface of the compute node motherboard management controller. The physical environment monitoring data includes the central processing unit package temperature, dynamic random access memory temperature, motherboard temperature, and server air inlet and outlet temperatures.

[0015] The hardware performance counter readings, the process-level resource consumption statistics, and the physical environment monitoring data are aligned and packaged according to timestamps, and encapsulated into a multi-dimensional running status data packet in a unified format;

[0016] Through the cluster's internal communication network, the multi-dimensional operational status data packets generated by each computing node are transmitted in real time to a centralized data aggregation node for storage and subsequent processing.

[0017] As a further aspect of the present invention, the step of inputting the multi-dimensional operating status data into a pre-constructed load characteristic analysis model, and quantitatively evaluating the real-time workload of each computing node based on the load characteristic analysis model to generate node load characteristic values ​​characterizing the node load level, specifically involves:

[0018] The aggregated multi-dimensional operational status data packets are preprocessed, including data cleaning to remove outliers and data normalization to eliminate the influence of different units.

[0019] Extract a set of predefined key load characteristics from the preprocessed data. These key load characteristics include the moving average of CPU utilization, the difference between the peak and trough of memory usage, input / output operation frequency, and temperature rise rate.

[0020] The key load characteristic indicators are used as input vectors and fed into the pre-built load characteristic analysis model, which is a gradient boosting decision tree model trained based on historical data.

[0021] The gradient boosting decision tree model outputs a comprehensive score based on the input key load characteristic indicators. The comprehensive score reflects the composite load pressure level of the computing node at the current moment.

[0022] Based on the predefined score range of the comprehensive score and combined with the upper limit of the computing node's own hardware resource configuration, the comprehensive score is mapped to a standardized node load characteristic value. The node load characteristic value is a value between zero and one, with a larger value indicating a higher load level.

[0023] As a further aspect of the present invention, the step of establishing a global resource scheduling model based on the node load characteristic value and a pre-set cluster energy efficiency optimization target specifically includes:

[0024] Obtain the node load characteristic values ​​of all computing nodes in the cluster to form the cluster load state vector at the current moment;

[0025] Read the pre-set cluster energy efficiency optimization target, which is specifically stated as minimizing the integral of the total energy consumption of the cluster within a scheduling cycle, while meeting the deadlines and performance requirements of all computing tasks.

[0026] The decision variables for constructing the global resource scheduling model include binary variables and continuous variables. The binary variables represent the mapping relationship between computing tasks and computing nodes, and the continuous variables represent the operating frequency or voltage of computing nodes in each time slice within the scheduling period.

[0027] Establish a set of constraints for the global resource scheduling model. The set of constraints includes: a computing task allocation constraint that each computing task must be allocated and only allocated to one computing node for execution; a node capacity constraint that the total resource requirements of all computing tasks allocated to each computing node must not exceed the available resource capacity of the computing node; a hardware feasibility constraint that the operating frequency or voltage of each computing node must be within the dynamic adjustment range supported by its hardware; and a task deadline constraint that the time from the start of execution to the completion of each computing task must not exceed its specified deadline.

[0028] The objective function of the global resource scheduling model is constructed. The objective function is a mathematical expression for the total energy consumption of the cluster. The total energy consumption of the cluster is obtained by summing the static power consumption and dynamic power consumption of each computing node. The dynamic power consumption is positively correlated with the operating frequency, voltage and load of the computing node.

[0029] As a further aspect of the present invention, the step of solving the global resource scheduling model to obtain an optimized resource scheduling strategy specifically includes:

[0030] The global resource scheduling model is decomposed using the Lagrange relaxation algorithm, which decomposes the global resource scheduling model into a main problem and several sub-problems, and the sub-problems are associated with a single computing node or a single computing task.

[0031] The Lagrange multipliers in the main problem are initialized, and the termination conditions of the algorithm are set, including the maximum number of iterations or the duality gap being less than a set threshold.

[0032] In each iteration, the value of the Lagrange multiplier is fixed, and each subproblem is solved in parallel to obtain the feasible task allocation scheme and node working status under the current multiplier. Based on the solution results of the subproblems, the Lagrange multiplier in the main problem is updated, usually using the subgradient method.

[0033] Repeat the iterative process until the termination condition is met, at which point a set of Lagrange multiplier values ​​that make the duality gap sufficiently small and the corresponding task allocation scheme are obtained.

[0034] The final task allocation scheme is then subjected to feasibility repair to ensure that it meets all the original constraints. The repaired scheme is the optimized resource scheduling strategy.

[0035] As a further aspect of the present invention, the step of generating specific resource scheduling control instructions based on the optimized resource scheduling strategy specifically includes:

[0036] The optimized resource scheduling strategy is analyzed to extract the list of computing tasks that need to be migrated, the original computing node identifier of each task to be migrated, the target computing node identifier to which it is planned to be migrated, and the planned migration time window.

[0037] For each computing task to be migrated, query its running context information on the original computing node. The running context information includes memory page status, file descriptors, process status, and network connection status.

[0038] Based on the runtime context information, checkpoint data creation instructions and context serialization instructions for the computing task are generated to save and restore the task state during migration.

[0039] Based on the migration time window, a detailed migration operation sequence is generated for each task to be migrated. The operation sequence includes pausing the task on the original compute node, creating a checkpoint, transferring data to the target node, restoring the task state on the target node, and restarting the task.

[0040] The migration operation sequences of all tasks to be migrated are sorted and integrated according to the planned execution time order, and necessary synchronization waiting points are inserted to form the final executable scheduling plan.

[0041] The executable scheduling plan is translated into a sequence of scripts or application programming interfaces that the underlying resource manager can recognize and execute, namely the resource scheduling control instructions.

[0042] As a further aspect of the present invention, the step of issuing the resource scheduling control command to the underlying resource manager of the high-density server cluster, and having the underlying resource manager perform dynamic migration of computing tasks and adjustment of the working status of computing nodes, specifically involves:

[0043] The underlying resource manager receives the resource scheduling control command and performs security and legality verification on it;

[0044] After the verification is successful, the underlying resource manager first sends a task pause and checkpoint creation command to the original computing node agent where the task to be migrated is located, according to the instruction sequence;

[0045] The original compute node agent executes commands to pause the execution of the specified compute task, serializes and saves its memory state and processor state as a checkpoint file, and freezes related input / output operations.

[0046] The underlying resource manager coordinates the original compute node agent and the target compute node agent to transmit the checkpoint file and related context data to the target compute node via the network;

[0047] After receiving the data, the target computing node agent recreates the execution environment of the computing task on the target computing node according to the instructions, restores its state from the checkpoint file, and then resumes the execution of the computing task.

[0048] During or after task migration, the underlying resource manager sends instructions to the relevant computing nodes to adjust their operating frequency or voltage according to the node working status specified in the scheduling policy, in order to achieve the energy efficiency optimization goal.

[0049] As a further aspect of the present invention, the original computing node proxy executes commands to suspend the execution of a specified computing task, serializes and saves its memory state and processor state as a checkpoint file, and freezes related input / output operations, specifically including:

[0050] The original computing node agent sends a pause signal to the process of the target computing task, causing the process to enter an interruptible sleep state.

[0051] Traverse the virtual address space of the process and copy the contents of all its resident memory pages to a pre-allocated buffer area;

[0052] Read the contents of the processor register set of the process, including general-purpose registers, program counter, stack pointer and flag register, and pack them together with memory page data;

[0053] For the file descriptors and network sockets opened by the process, record their current file offset and socket status metadata information;

[0054] The memory page data, processor register data, and input / output metadata information are serialized and encoded, and written to a persistently stored checkpoint file.

[0055] During checkpoint creation, all new input / output requests to the process are blocked, and the process waits for existing input / output operations to complete to ensure data consistency.

[0056] As a further aspect of the present invention, the construction steps of the pre-built load characteristic analysis model include:

[0057] Collect historical multi-dimensional operational status data of multiple computing nodes in a high-density server cluster during historical periods, and label the historical multi-dimensional operational status data to obtain a training dataset with real load labels.

[0058] The training dataset is preprocessed, including missing value imputation, outlier handling, and data standardization.

[0059] Historical feature indicators corresponding to the key load feature indicators are extracted from the preprocessed training dataset to form a training feature vector set.

[0060] Initialize a gradient boosting decision tree model framework and configure the model hyperparameters, including the number of base learners, the learning rate, and the maximum depth of the tree;

[0061] Using the training feature vector set and the corresponding real load labels, the initialized gradient boosting decision tree model is iteratively trained with the goal of minimizing the prediction error.

[0062] The performance of the trained gradient boosting decision tree model is evaluated on the validation dataset, and hyperparameters are tuned based on the evaluation results until the model performance reaches the preset standard, thus obtaining the pre-built load feature analysis model.

[0063] As a further aspect of the present invention, before constructing the decision variables of the global resource scheduling model, a step of constructing a computing node power consumption estimation model is included:

[0064] Based on the hardware specifications and historical operating data of the computing nodes, a functional relationship is established between the power consumption of the computing nodes and the CPU utilization, memory usage, operating frequency and ambient temperature.

[0065] By training offline, multiple linear regression or support vector regression algorithms are used to fit the coefficients of the functional relationship to obtain the power consumption estimation model for each computing node.

[0066] The power consumption estimation model is embedded as a parameter into the objective function of the global resource scheduling model to calculate the dynamic power consumption of computing nodes under different loads and configurations.

[0067] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0068] The load characteristic analysis model fuses real-time collected multi-dimensional operational status data, including processor utilization, memory usage, storage I / O, network traffic, power consumption, and temperature, to generate standardized node load characteristic values. This model integrates various indicators through algorithms to comprehensively evaluate the real-time workload of nodes, replacing conventional load determination methods that rely on single or a few static indicators. This quantitative approach based on multi-dimensional data fusion can more accurately reflect the overall stress state and energy efficiency level of nodes running heterogeneous tasks, outputting consistent and comparable load characteristic values. This result provides high-precision input for subsequent scheduling decisions, reducing scheduling errors caused by biased or incomplete load assessments.

[0069] The global resource scheduling model takes minimizing the total power consumption of the cluster as its direct objective function, optimizing the solution while satisfying constraints on computing node resource capacity and task service quality. The resource scheduling strategy obtained after model solution explicitly specifies the tasks to be migrated, target nodes, and the execution sequence of task migration. This strategy, containing specific timing information, guides the underlying resource manager to execute migration operations in an orderly manner, controlling the dynamic process of cluster state transitions. This timing planning capability enables the system to coordinate the order of multiple migration actions, avoiding resource contention and instantaneous power consumption fluctuations caused by migration concurrency. Through fine-grained timing control of energy-saving operations, the system achieves smooth adjustment of load distribution and continuous optimization of power consumption, extending energy efficiency management from static resource allocation to the dynamic execution level. Attached Figure Description

[0070] Figure 1 This is a flowchart of the energy efficiency optimization method for high-density server clusters based on resource scheduling described in this invention;

[0071] Figure 2 A flowchart for the method of generating node load characteristic values. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0073] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0074] See Figure 1The system collects multi-dimensional operational status data of each computing node in a high-density server cluster in real time. This data is then input into a pre-built load characteristic analysis model. Based on this model, the real-time workload of each computing node is quantitatively evaluated, generating node load characteristic values ​​that characterize the node's load level. A global resource scheduling model is established based on these node load characteristic values ​​and a pre-set cluster energy efficiency optimization target. This model uses minimizing the total power consumption of the cluster as its objective function, constrained by the resource capacity of the computing nodes and the service quality requirements of the tasks. Solving the global resource scheduling model yields an optimized resource scheduling strategy, which specifically specifies the computing tasks to be migrated, the target computing nodes, and the execution sequence of the task migration. Based on the optimized resource scheduling strategy, specific resource scheduling control instructions are generated and sent to the underlying resource manager of the high-density server cluster. The underlying resource manager then performs the dynamic migration of computing tasks and the adjustment of the computing node's working status.

[0075] In one embodiment of the present invention, a detailed implementation of real-time multi-dimensional operational status data collection is described, using a high-density server cluster example scenario consisting of 32 identical computing nodes, each equipped with a dual-processor CPU, 256 gigabytes of memory, and solid-state drive storage. In the specific implementation, a monitoring agent initiates a data collection task on each computing node with a preset sampling period of 5 seconds. At the sampling time, the monitoring agent first reads the hardware performance counters to obtain the non-idle cycle count and total cycle count of each CPU core over the past 5 seconds. The instantaneous utilization rate of each core is obtained by calculating the ratio, and the arithmetic mean of the utilization rates of all cores is calculated as the CPU utilization data item for that node. The hardware performance counter readings also include the cumulative value of the last-level cache misses and the memory controller access latency. By calculating the difference between the cumulative values ​​in two adjacent sampling periods, the cache miss count and average memory access latency data for that period are obtained. The monitoring agent then calls the proc filesystem interface exposed by the operating system kernel to traverse the process list, calculate the total CPU time slices consumed by all processes in the most recent sampling period, and divide it by the product of the sampling period duration and the number of CPU cores for cross-validation of CPU utilization. By reading process status information, the monitoring agent summarizes the physical memory residency set size of all processes and divides it by the total physical memory capacity of the node to obtain the memory utilization data item. The monitoring agent further uses the iostat command to obtain the number of read / write operations and the amount of data on the disk device, calculating the input / output throughput data item. In specific implementations, the monitoring agent uses the ip-slink command to read the packet count and byte count of the network interface, calculating the network input / output throughput as a supplement. Finally, the monitoring agent calls the sensor interface of the motherboard management controller via the IPMI protocol to synchronously read the CPU package temperature, DRAM temperature, motherboard temperature, and temperature sensor readings deployed at the chassis inlet and outlet. These readings together constitute the current operating temperature data item.

[0076] In some embodiments, the monitoring agent organizes the collected multi-dimensional data into a set of key-value pairs with precise timestamps. The monitoring agent aligns hardware performance counter readings, process-level resource consumption statistics, and physical environment monitoring data acquired at the same sampling time according to their timestamps. For minor time deviations caused by system call delays, the monitoring agent considers data with deviations within 10 milliseconds as data from the same time. The monitoring agent packages all aligned data items into a JSON-formatted multi-dimensional runtime status data packet with a unified structure, predefining the names, types, and units of all data fields. After encapsulation, the monitoring agent uses the TCP protocol to send the multi-dimensional runtime status data packet in real time to a pre-configured centralized data aggregation node via the cluster's gigabit Ethernet network. The data aggregation node runs a data receiving service that listens on a specific port, receives data streams from all 32 compute nodes, and stores the received multi-dimensional runtime status data packets in chronological order and by node identifier in a time-series database for subsequent load characteristic analysis. This implementation ensures that the data collected from each compute node is dimensionally consistent, formatted uniformly, and transmitted reliably.

[0077] Optionally, for collecting hardware performance counter readings, the monitoring agent utilizes the Linux operating system's `perf_event_open` system call interface to create a performance monitoring counter for each CPU core, configuring it to monitor CPU reference cycle events and last-level cache miss events. The monitoring agent reads the counter value at the end of each sampling period and calculates the core's active cycle ratio using the following formula:

[0078]

[0079] Where: characters Indicates the utilization rate of a specific core, character This represents the number of periods during which the core is in a non-idle state within the sampling period. This represents the total number of periods within the sampling period. The monitoring agent monitors all cores... Calculate the average value to obtain the node-level CPU utilization data.

[0080] In some embodiments, for reading physical environment monitoring data, the monitoring agent does not directly read from the motherboard management controller sensor interface in every sampling cycle. Instead, a caching mechanism is used to reduce the pressure of frequent calls to the management controller interface. The monitoring agent maintains a temperature data cache, and reads the CPU package temperature, DRAM temperature, motherboard temperature, and inlet / outlet vent temperature completely from the motherboard management controller sensor interface every 30 seconds. During the 5-second sampling cycle in between, the monitoring agent uses the temperature value in the cache as the current operating temperature data item. Simultaneously, the monitoring agent monitors the timeliness of the cached data; if it has not been updated for more than 35 seconds, it forces a re-read of the sensor interface in the next cycle. This caching mechanism reduces the frequency of access to the motherboard management controller interface while ensuring data timeliness meets analysis requirements, avoiding potential interface blockage or delays caused by high-frequency access. The monitoring agent correlates the cached temperature value with chip heat dissipation data estimated by performance counters. If it finds that the cached temperature value is significantly inconsistent with the temperature rise trend estimated based on CPU utilization and operating frequency, it immediately triggers a real-time sensor read to refresh the cache, ensuring the reliability of the physical environment monitoring data. The monitoring agent will eventually package and send the temperature data with time stamps along with other real-time collected data items.

[0081] In one embodiment of the present invention, see [reference] Figure 2 This paper describes a scenario involving a high-density server cluster. The centralized data aggregation node continuously receives and stores multi-dimensional operational status data packets from all computing nodes in the cluster, as per the implementation example. The specific implementation method for generating node load characteristic values ​​is detailed below. In this implementation, the load characteristic analysis module retrieves all multi-dimensional operational status data packets within the most recent analysis window from the time-series database. The load characteristic analysis module performs data cleaning on the raw data, setting reasonable numerical ranges for each data item. For example, CPU utilization should be between 0 and 1, and temperature readings should be between 10 and 100 degrees Celsius. Data points outside these ranges are considered outliers and replaced with the value from the previous valid sampling period. For individual data packet losses due to network latency, the load characteristic analysis module uses linear interpolation to fill in the missing values ​​using data from adjacent timestamps. After outlier removal and missing value imputation, the load feature analysis module performs data normalization. For proportional data such as CPU utilization and memory usage, a min-max normalization method is used to scale them to the [0,1] interval. For data with different physical dimensions, such as input / output throughput and temperature values, a Z-score normalization method is used to eliminate the influence of dimensions, resulting in a mean of 0 and a standard deviation of 1 for the processed data. These preprocessing operations ensure the stability and comparability of subsequent feature extraction.

[0082] In some embodiments, the load characteristic analysis module extracts a set of predefined key load characteristic indicators from the preprocessed time series data. For each computing node, the load characteristic analysis module calculates the moving average of the CPU utilization over the past 12 sampling points as a key load characteristic indicator reflecting short-term load trends. The load characteristic analysis module scans memory occupancy data within the same time window, identifies its maximum and minimum values, and calculates the difference between the peak and trough values ​​as a key load characteristic indicator of memory load fluctuation. The load characteristic analysis module counts the total number of disk read / write operations within the window and divides it by the time window length to obtain the key load characteristic indicator of input / output operation frequency. The load characteristic analysis module also calculates the linear regression slope of the node CPU package temperature within the window as a key load characteristic indicator of temperature rise rate. An example of the calculation formula for the above key load characteristic indicators is shown below, used for moving average calculation:

[0083]

[0084] Where: characters The moving average of CPU utilization is represented by the character. Indicates the sliding window size, character Indicates the current time index, character Indicates at time The CPU utilization data has been normalized. All extracted key load characteristics are combined into a single input vector.

[0085] In its implementation, the load characteristic analysis module receives input vectors containing multiple key load characteristic indicators and feeds them into a pre-built load characteristic analysis model. This pre-built model is a gradient boosting decision tree model, consisting of 150 regression trees, each with a maximum depth limit of 6. After receiving the input vectors, each tree makes a preliminary prediction of the load level based on its feature partitioning rules. The model then weights and sums the predictions from all trees, outputting a comprehensive score ranging from 0 to 100. This comprehensive score integrates information from multiple aspects, including CPU, memory, input / output, and thermal load, reflecting the current composite load pressure level of the computing node. Essentially, the gradient boosting decision tree model learns the complex nonlinear relationship between key load characteristic indicators and the actual load pressure through historical data.

[0086] Optionally, the load feature analysis module internally maintains a mapping configuration file related to the hardware configuration of the computing nodes. This mapping configuration file predefines different score ranges based on the number of CPU cores, memory capacity, and storage type of the computing node. For example, for a high-configuration computing node, a comprehensive score in the 0-40 range might be considered low load, 40-70 medium load, and 70-100 high load. The load feature analysis module reads the configuration identifier of the current computing node, finds the corresponding predefined score range according to the mapping configuration file, and maps the comprehensive score output by the gradient boosting decision tree model to a standardized node load feature value. The mapping rule linearly maps the relative position of the comprehensive score range to the [0,1] range. For example, if the comprehensive score of 65 falls within the medium load range [40,70], the node load feature value is calculated as (65-40) / (70-40) = 0.833. The node load feature value is a value between zero and one; a larger value indicates a higher load level. The load characteristic analysis module performs the above process for each computing node in the cluster, generating a set of node load characteristic values ​​for subsequent resource scheduling.

[0087] In some embodiments, the construction of the pre-built load feature analysis model is independent of the online evaluation process. The model construction process collects historical, multi-dimensional operational status data of a large number of computing nodes in a high-density server cluster over historical periods. Operators label each historical data point with a true load label based on system monitoring logs and resource manager records. The load label is an integer from 1 to 5, representing the load state of the node at that moment, thus obtaining a training dataset with true load labels. Preprocessing operations are performed on the training dataset, including imputing missing values ​​using the median, identifying and correcting outliers using the 3σ principle, and standardizing all features. From the preprocessed training dataset, historical feature indicators such as the moving average of CPU utilization, the difference between peak and trough memory usage, input / output operation frequency, and temperature rise rate, identical to those in the online phase, are extracted to form a training feature vector set. A gradient boosting decision tree model framework is initialized, and the model hyperparameters are configured, including setting the number of base learners to 150, the learning rate to 0.05, and the maximum tree depth to 6. Using a training feature vector set and corresponding real load labels, an initial gradient boosting decision tree model is iteratively trained with the goal of minimizing the mean squared error between the predicted score and the real label level. The performance of the trained gradient boosting decision tree model is evaluated on a reserved validation dataset, with the mean absolute error as the evaluation metric. Based on the evaluation results, hyperparameters are tuned using grid search until the mean absolute error is below 0.5 levels, thus obtaining a pre-built load feature analysis model with satisfactory performance.

[0088] It is understandable that the gradient boosting decision tree model parameters and feature extraction rules used in the online evaluation phase must be completely consistent with those in the model building phase to ensure the accuracy and consistency of node load feature value evaluation. When loading the pre-built load feature analysis model, the load feature analysis module will also load the feature normalization parameters used in the model building phase to correctly standardize the key load feature indicators extracted online.

[0089] In one embodiment of the present invention, considering a high-density server cluster scenario, the load characteristic analysis module has generated node load characteristic values ​​for all computing nodes in the cluster as described in the embodiment. The specific implementation of establishing the global resource scheduling model is described in detail. In the specific implementation, the global resource scheduling module obtains the node load characteristic values ​​of all computing nodes in the cluster at the current scheduling moment from the output interface of the load characteristic analysis module. These node load characteristic values ​​are values ​​between zero and one. The global resource scheduling module arranges these node load characteristic values ​​in order of the physical number of the computing nodes, forming a cluster load state vector of dimension N at the current moment, where N represents the total number of computing nodes in the cluster. For example, in a 32-node cluster, the cluster load state vector is an array containing 32 floating-point numbers. Simultaneously, the global resource scheduling module reads a pre-set cluster energy efficiency optimization target from the policy configuration database. The cluster energy efficiency optimization target is specifically stated as minimizing the integral of the total energy consumption of the cluster within a scheduling period T, while meeting the deadlines and performance requirements of all computing tasks. The scheduling period T is set to, for example, 300 seconds. The cluster energy efficiency optimization target explicitly requires that the service quality agreement of the tasks should not be violated due to energy saving.

[0090] In some embodiments, the global resource scheduling module begins to construct the decision variables for the global resource scheduling model. Decision variables include two types: binary variables and continuous variables. Binary variables are used to represent the mapping relationship between computational tasks and computational nodes; for example, defining a binary variable... When it is 1, it indicates that computation task i is assigned to compute node j for execution; when it is 0, it indicates that it is not assigned. Continuous variables are used to represent the operating frequency or voltage of a compute node in each time slice within the scheduling period. For example, defining a continuous variable... , represents the operating frequency of the central processing unit of computing node j in discrete time slice t, and its value must be within the frequency range supported by the hardware of computing node j. Within the cluster, the global resource scheduling module instantiates a set of decision variables based on the number of computing tasks M and computing nodes N to be scheduled in the current cluster, as well as the number of time slices K into which the scheduling period T is divided.

[0091] The global resource scheduling module then establishes a set of constraints for the global resource scheduling model. This set of constraints first includes constraints on the allocation of computational tasks, expressed through mathematical equations that require, for each computational task i, all its corresponding binary variables... The summation across all compute nodes j must equal 1, ensuring that each computational task is allocated and executed on exactly one compute node. The second constraint set includes node capacity constraints, expressed through mathematical inequalities. These constraints require that for each compute node j, the total resource requirements of all tasks allocated to it must not exceed the node's available resource capacity at the current moment. Available resource capacity is calculated by subtracting the estimated resource occupancy of the existing load, as represented by the node's load characteristic value, from the node's total physical resources. The third constraint set includes hardware feasibility constraints, requiring that the operating frequency or voltage of each compute node must be within the dynamic adjustment range supported by its hardware, i.e., a continuous variable. Must meet The final constraint set includes the task deadline constraint, which calculates the expected execution time of the task under a given resource allocation and workload, requiring that the time from start to finish must not exceed the deadline specified in the task specification.

[0092] The global resource scheduling module ultimately constructs the objective function of the global resource scheduling model. The objective function is a mathematical expression for the total cluster energy consumption, defined as the sum of the integrals of the power consumption of all computing nodes over time within a scheduling cycle. Due to the use of a discrete time-slice model, the integral is transformed into a summation. The total cluster energy consumption is obtained by summing the static and dynamic power consumption of each computing node. Static power consumption is the basic power consumption of the computing node under no-load conditions at its lowest operating frequency, and is considered a constant. Dynamic power consumption is calculated through an embedded power estimation model and is positively correlated with the operating frequency, voltage, and load of the computing node. The specific form of the objective function is to find the minimum sum of the predicted power consumption values ​​of all computing nodes over all time slices, with binary variables as the decision variables. and continuous variables The optimization process involves finding the combination of decision variable values ​​that minimizes the objective function value while satisfying all constraints.

[0093] Optionally, when constructing node capacity constraints, the global resource scheduling module needs to convert node load characteristic values ​​into specific resource occupancy estimates. The global resource scheduling module internally maintains a resource conversion lookup table. For a node load characteristic value L, the lookup table provides its corresponding estimated CPU resource occupancy. The estimated memory usage is ,in: and These represent the total number of CPU cores and total memory capacity of the compute node, respectively. This non-linear conversion reflects that the relationship between memory usage and load pressure is not always linear. Node capacity constraints require that the resource requirements of newly allocated tasks, plus this estimated usage, must not exceed the physical resource limit of the compute node.

[0094] In one embodiment of the present invention, considering a high-density server cluster scenario, the global resource scheduling module has established a global resource scheduling model with the objective of minimizing the total power consumption of the cluster, including task allocation and frequency adjustment decision variables and multiple constraints, as described in the embodiment. The specific implementation methods for solving this model and obtaining the optimized resource scheduling strategy are described in detail. In the specific implementation, the solution algorithm module uses the Lagrange relaxation algorithm to process the global resource scheduling model, decomposing the complex original problem into more manageable subproblems. The solution algorithm module identifies the constraints in the model that couple the problems, such as computational task allocation constraints, and relaxes these constraints into the objective function in the form of penalty terms, thereby constructing the Lagrange relaxation dual problem. The Lagrange relaxation dual problem can be decomposed into a main problem and several subproblems. The main problem is responsible for updating the Lagrange multipliers, while the subproblems are associated with a single computing node or a single computing task. Each subproblem only involves the local decision variables of that node or task, thus allowing for independent parallel solutions. The formal expression of the Lagrange relaxation dual problem is as follows:

[0095]

[0096] Where: symbol Represents the value of the Lagrange function given the multipliers, with the sign... The original objective function represents the total power consumption of the cluster. and These are Lagrange multipliers corresponding to task allocation constraints and node capacity constraints, with the symbol... It is a binary allocation variable, symbol It is a frequency variable, symbol It refers to the resource requirements of the task, symbol. Is the node at frequency The effective capacity below.

[0097] In some embodiments, the solution algorithm module initializes the Lagrange multipliers in the main problem, typically by initializing all Lagrange multipliers. and Set to zero or a small positive number. The solution algorithm module also sets the termination conditions for the Lagrange relaxation algorithm, including setting the maximum number of iterations to 1000, or the duality gap being less than a set threshold, such as 0.01. In each iteration, the solution algorithm module fixes the current Lagrange multiplier. and The value of completely decomposes the relaxed optimization problem. The subproblem associated with computation node j receives the current multiplier value of that node. And the multiplier values ​​of all tasks that may be assigned to this node. Within the node's resource constraints and frequency feasible region, the subproblems determine which tasks should be assigned to a given node and the appropriate operating frequency for that node, in order to minimize the local objective function for that node. The subproblems associated with computational task i evaluate the "cost" of assigning that task to different nodes and select the node that minimizes the local cost. These subproblems can be solved in parallel to obtain a set of task allocation schemes under the current multiplier. Node operating frequency settings .

[0098] The algorithm module calculates the degree of violation of the original constraints based on the task allocation scheme and node working status obtained from solving the subproblems. For example, it checks whether each task is assigned to exactly one node and whether the resource usage of each node exceeds its capacity. The algorithm module repeats the iterative process of solving the subproblems with fixed multipliers and updating the multipliers based on the results until a preset termination condition is met, such as reaching the maximum number of iterations or a sufficiently small duality gap. At this point, a set of Lagrange multiplier values ​​that minimize the duality gap and the corresponding task allocation and frequency configuration scheme are obtained. In practice, since the Lagrange relaxation algorithm solves the dual problem, its optimal solution may be infeasible for the original problem. For example, a few tasks may be assigned to multiple nodes or not assigned to any node. The algorithm module performs feasibility repair on the final task allocation scheme. First, the algorithm module checks the computing node allocation status of each task. For unassigned tasks, the algorithm module assigns them to the computing node with the most abundant remaining resources that meets their deadline constraints. For a task assigned to multiple nodes, the algorithm module selects the node that minimizes the increase in the objective function as the final assignment node and removes the task from the other nodes. After the fix, the algorithm module recalculates the working frequency variable of each computing node under the fixed task allocation scheme. This is a continuous convex optimization problem, which can be solved quickly using gradient descent. The solution obtained after feasibility fix and frequency re-optimization is the optimized resource scheduling strategy that satisfies all the original constraints.

[0099] Optionally, the optimized resource scheduling strategy is represented by a structured data object containing the complete scheduling plan. The resource scheduling control instruction generation module parses the optimized resource scheduling strategy and extracts a list of computing tasks that need to be migrated. The resource scheduling control instruction generation module identifies the original computing node identifier, the target computing node identifier to which each task to be migrated is located, and the planned migration time window for execution. The migration time window is a time interval based on the start of the scheduling cycle. For example, the migration window for task A is [10 seconds, 25 seconds], indicating that the migration operation should be started and completed within this time interval. Referring to Table 1, the resource scheduling control instruction generation module generates a task migration plan table.

[0100] Table 1: Computation Task Migration Plan

[0101] Task identifier Original computing node identifier Target computing node identifier Planned migration time window (seconds) Task_001 Node_05 Node_12 [15,30] Task_007 Node_18 Node_08 [45,65] Task_015 Node_22 Node_03 [80,100]

[0102] For each computing task to be migrated, the resource scheduling control instruction generation module retrieves its runtime context information on the original computing node by querying the cluster metadata service. The runtime context information includes the task process's memory page state, a list of open file descriptors, the process state in the process control block, and the established network connection state. Based on the runtime context information, the resource scheduling control instruction generation module generates checkpoint data creation instructions and context serialization instructions for the computing task. These instructions specify the temporary path for storing the checkpoint file, the range of memory regions to be serialized, and the specific register states to be saved.

[0103] In some embodiments, the resource scheduling control instruction generation module, in conjunction with the migration time window extracted from the policy, generates a detailed migration operation sequence for each task to be migrated. The migration operation sequence is an ordered list of commands, including: the first command instructing the original compute node agent to pause the execution of the specified compute task; the second command instructing the original compute node agent to create a checkpoint for the task and serialize and save the memory and processor states; the third command instructing the underlying resource manager to transfer the checkpoint file and related context data from the original node to the target node; the fourth command instructing the target compute node agent to restore the task state from the received data; and the fifth command instructing the target compute node agent to restart the compute task. The resource scheduling control instruction generation module sorts the migration operation sequences of all tasks to be migrated according to the planned execution time order. The resource scheduling control instruction generation module inserts necessary synchronization wait points into the sorted command sequence. For example, after the command to transfer the checkpoint file is completed, a synchronization instruction to wait for the data transfer to complete must be inserted before the subsequent state recovery command can be executed, thus forming the final executable scheduling plan.

[0104] In one embodiment of the present invention, considering a high-density server cluster scenario, the resource scheduling control instruction generation module has generated resource scheduling control instructions containing specific migration operation sequences and frequency adjustment commands as described in the embodiment. The specific implementation method of the underlying resource manager executing these instructions to complete dynamic migration and state adjustment is described in detail. In specific implementation, the underlying resource manager receives resource scheduling control instructions through a message queue. The underlying resource manager performs security and legality verification on the instructions. The verification process includes verifying the digital signature of the instructions to ensure their source is trustworthy, parsing the instruction format to check if it conforms to a predefined pattern, and verifying whether the computing task identifier and computing node identifier mentioned in the instructions actually exist in the current cluster resource pool. For instructions involving task migration, the underlying resource manager additionally checks the network connectivity between the original computing node and the target computing node, and whether the target computing node has sufficient free resources to accommodate the migration task. Failure of any of these verifications will result in the instruction being rejected and an error log being recorded.

[0105] In some embodiments, after the underlying resource manager passes instruction verification, it sends a task pause and checkpoint creation command to the original compute node agent where the task to be migrated resides, according to the planned execution time specified in the instruction sequence. Upon receiving the command from the underlying resource manager, the original compute node agent executes the command and sends a SIGSTOP signal to the process of the target compute task, causing the process to enter an interruptible sleep state, and the process is no longer scheduled for execution by the operating system. The original compute node agent traverses the virtual address space of the target compute task process, identifies all pages resident in physical memory by parsing the process's page table, and copies the contents of these memory pages page by page to a pre-allocated buffer area. The original compute node agent attaches to the target process and reads its processor register set contents using the ptrace system call provided by the operating system. The register set contents include the current values ​​of general-purpose registers, program counter, stack pointer, and flag registers. The original compute node agent packages these register values ​​together with the previously copied memory page data. For file descriptors and network sockets opened by the target compute task process, the original compute node agent records their current file offset, socket status, peer address, and other metadata information. The native compute node agent serializes and encodes memory page data, processor register data, and input / output metadata. The encoded data stream is written to a persistent checkpoint file, typically stored on the native compute node's local temporary file system or shared storage. During checkpoint creation, the native compute node agent intercepts system calls or uses kernel modules to block all new input / output requests from the target process and polls for completed or timed-out input / output operations to ensure the checkpoint file captures a consistent task state.

[0106] The underlying resource manager coordinates the original compute node agent and the target compute node agent to execute the transfer of the checkpoint file. The underlying resource manager instructs the original compute node agent to cut the checkpoint file into fixed-size data blocks and stream these blocks to the target compute node via a secure network channel. The target compute node agent listens on a designated receiving port, receives the transmitted data blocks, and reassembles them into a complete checkpoint file. A checksum mechanism ensures data integrity during transmission; failure to transmit a single data block triggers a retransmission. After receiving all data, the target compute node agent, according to the detailed description of the resource scheduling control instructions, recreates the execution environment for the computation task on the target compute node. This includes allocating a virtual address space of the same size, establishing the same file descriptor table mapping, and restoring the network connection context. The target compute node agent deserializes memory page data, processor register data, and input / output metadata from the checkpoint file, writes the memory contents back to the address space of the newly created process, sets register values ​​in the context of the new process, and restores the file pointer and socket states based on the metadata information. Once the state is restored, the target compute node agent sends a SIGCONT signal to the new process to resume the execution of the compute task. The task will continue running from the instruction address where it was suspended.

[0107] During or after task migration, the underlying resource manager sends instructions to the relevant compute nodes to adjust their operating frequency or voltage based on the node operating status specified in the scheduling policy. The underlying resource manager sends specific frequency setting commands through the compute node's baseboard management controller interface or the operating system-level frequency adjustment driver. For example, sending an instruction to a compute node to adjust its central processing unit (CPU) frequency from 2.5 GHz to 2.0 GHz reduces the node's dynamic power consumption. The underlying resource manager monitors the execution results of the frequency adjustment commands to ensure that the node's operating status has been successfully adjusted to the target value, thereby achieving energy efficiency optimization goals.

[0108] It is understandable that the entire execution process is carried out strictly according to the timing specified in the resource scheduling control instructions. For complex scheduling plans that include multiple tasks to be migrated and multiple node frequency adjustments, the underlying resource manager maintains an execution state machine to ensure that subsequent operations are triggered only after the preliminary operations are completed, and confirms the results at critical steps, thereby ensuring the reliability and consistency of cluster state transitions.

[0109] Optionally, the size of the checkpoint file can be estimated using the following formula:

[0110]

[0111] Where: symbol Indicates the estimated total size of the checkpoint file, symbol This represents the number of pages of memory resident for the target computation task process, with the symbol [symbol missing]. Indicates the size of a memory page, symbol Indicates the size of the processor register data, sign This represents the size of the input and output metadata information, and it is proportional to the number of files and network connections opened by the process. The underlying resource manager can use this formula to estimate the network transmission load and required time during the migration process before scheduling.

[0112] In some embodiments, when the target compute node agent resumes a compute task, if a state recovery failure occurs due to differences in the environment between the target node and the original node, the target compute node agent will report an error to the underlying resource manager. The underlying resource manager, based on a preset fault handling strategy, may choose to migrate the task back to the original compute node or select another compatible standby compute node to re-execute the migration process. The underlying resource manager records key events, state changes, and any anomalies throughout the entire execution process in an audit log for traceability and analysis by operations and maintenance personnel.

[0113] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for optimizing the energy efficiency of high-density server clusters based on resource scheduling, characterized in that, Includes the following steps: Multi-dimensional operational status data of each computing node in a high-density server cluster is collected in real time. The multi-dimensional operational status data is input into a pre-built load characteristic analysis model. Based on the load characteristic analysis model, the real-time workload of each computing node is quantitatively evaluated, and node load characteristic values ​​that characterize the load level of the node are generated. Based on the node load characteristics and the pre-set cluster energy efficiency optimization target, a global resource scheduling model is established. The global resource scheduling model takes minimizing the total power consumption of the cluster as the objective function and the resource capacity of the computing nodes and the service quality requirements of the tasks as constraints. Solve the global resource scheduling model to obtain an optimized resource scheduling strategy. The optimized resource scheduling strategy specifically specifies the computing tasks to be migrated, the target computing nodes, and the execution sequence of task migration. Based on the optimized resource scheduling strategy, specific resource scheduling control instructions are generated and sent to the underlying resource manager of the high-density server cluster. The underlying resource manager then performs dynamic migration of computing tasks and adjustment of the working status of computing nodes. The step of generating specific resource scheduling control instructions based on the optimized resource scheduling strategy is as follows: The optimized resource scheduling strategy is analyzed to extract the list of computing tasks that need to be migrated, the original computing node identifier of each task to be migrated, the target computing node identifier to which it is planned to be migrated, and the planned migration time window. For each computing task to be migrated, query its running context information on the original computing node. The running context information includes memory page status, file descriptors, process status, and network connection status. Based on the runtime context information, checkpoint data creation instructions and context serialization instructions for the computing task are generated to save and restore the task state during migration. Based on the migration time window, a detailed migration operation sequence is generated for each task to be migrated. The operation sequence includes pausing the task on the original compute node, creating a checkpoint, transferring data to the target node, restoring the task state on the target node, and restarting the task. The migration operation sequences of all tasks to be migrated are sorted and integrated according to the planned execution time order, and synchronization waiting points are inserted to form the final executable scheduling plan. The executable scheduling plan is translated into a sequence of scripts or application programming interfaces that the underlying resource manager can recognize and execute, namely the resource scheduling control instructions.

2. The energy efficiency optimization method for high-density server clusters based on resource scheduling according to claim 1, characterized in that, The real-time acquisition of multi-dimensional operational status data for each computing node in the high-density server cluster specifically includes: The multi-dimensional operating status data includes at least the CPU utilization, memory usage, input / output throughput, and current operating temperature of the computing node; By deploying a monitoring agent on each computing node, the hardware performance counter readings of the computing node are periodically obtained at a preset sampling period. The hardware performance counter readings include the number of active cycles of each core of the central processing unit, the number of cache misses, and the memory controller access latency. At the same time, the system call interface of the computing node operating system is invoked to obtain process-level resource consumption statistics, which include the CPU time slice occupancy, physical memory resident set size, number of disk read / write operations, and number of network connections for each running process. The physical environment monitoring data is read through the sensor interface of the compute node motherboard management controller. The physical environment monitoring data includes the central processing unit package temperature, dynamic random access memory temperature, motherboard temperature, and server air inlet and outlet temperatures. The hardware performance counter readings, the process-level resource consumption statistics, and the physical environment monitoring data are aligned and packaged according to timestamps, and encapsulated into a multi-dimensional running status data packet in a unified format; Through the cluster's internal communication network, the multi-dimensional operational status data packets generated by each computing node are transmitted in real time to a centralized data aggregation node for storage and subsequent processing.

3. The energy efficiency optimization method for high-density server clusters based on resource scheduling according to claim 2, characterized in that, The process involves inputting the multi-dimensional operational status data into a pre-built load characteristic analysis model, and then quantitatively evaluating the real-time workload of each computing node based on the model to generate node load characteristic values ​​that characterize the node's load level. Specifically: The aggregated multi-dimensional operational status data packets are preprocessed, including data cleaning to remove outliers and data normalization to eliminate the influence of different units. Extract a set of predefined key load characteristics from the preprocessed data. These key load characteristics include the moving average of CPU utilization, the difference between the peak and trough of memory usage, input / output operation frequency, and temperature rise rate. The key load characteristic indicators are used as input vectors and fed into the pre-built load characteristic analysis model, which is a gradient boosting decision tree model trained based on historical data. The gradient boosting decision tree model outputs a comprehensive score based on the input key load characteristic indicators. The comprehensive score reflects the composite load pressure level of the computing node at the current moment. Based on the predefined score range of the comprehensive score and combined with the upper limit of the computing node's own hardware resource configuration, the comprehensive score is mapped to a standardized node load characteristic value. The node load characteristic value is a value between zero and one, with a larger value indicating a higher load level.

4. The energy efficiency optimization method for high-density server clusters based on resource scheduling according to claim 3, characterized in that, The step of establishing a global resource scheduling model based on the node load characteristics and a pre-set cluster energy efficiency optimization target is as follows: Obtain the node load characteristic values ​​of all computing nodes in the cluster to form the cluster load state vector at the current moment; Read the pre-set cluster energy efficiency optimization target, which is specifically stated as minimizing the integral of the total energy consumption of the cluster within a scheduling cycle, while meeting the deadlines and performance requirements of all computing tasks. The decision variables for constructing the global resource scheduling model include binary variables and continuous variables. The binary variables represent the mapping relationship between computing tasks and computing nodes, and the continuous variables represent the operating frequency or voltage of computing nodes in each time slice within the scheduling period. Establish a set of constraints for the global resource scheduling model. The set of constraints includes: a computing task allocation constraint that each computing task must be allocated and only allocated to one computing node for execution; a node capacity constraint that the total resource requirements of all computing tasks allocated to each computing node must not exceed the available resource capacity of the computing node; a hardware feasibility constraint that the operating frequency or voltage of each computing node must be within the dynamic adjustment range supported by its hardware; and a task deadline constraint that the time from the start of execution to the completion of each computing task must not exceed its specified deadline. The objective function of the global resource scheduling model is constructed. The objective function is a mathematical expression for the total energy consumption of the cluster. The total energy consumption of the cluster is obtained by summing the static power consumption and dynamic power consumption of each computing node. The dynamic power consumption is positively correlated with the operating frequency, voltage and load of the computing node.

5. The energy efficiency optimization method for high-density server clusters based on resource scheduling according to claim 4, characterized in that, The process of solving the global resource scheduling model to obtain an optimized resource scheduling strategy specifically involves: The global resource scheduling model is decomposed using the Lagrange relaxation algorithm, which decomposes the global resource scheduling model into a main problem and several sub-problems, and the sub-problems are associated with a single computing node or a single computing task. The Lagrange multipliers in the main problem are initialized, and the termination conditions of the algorithm are set, including the maximum number of iterations or the duality gap being less than a set threshold. In each iteration, the value of the Lagrange multiplier is fixed, and each subproblem is solved in parallel to obtain the feasible task allocation scheme and node working status under the current multiplier. Based on the solution results of the subproblems, the Lagrange multiplier in the main problem is updated using the subgradient method. Repeat the iterative process until the termination condition is met. At this point, a set of Lagrange multiplier values ​​that make the duality gap less than a set threshold and the corresponding task allocation scheme are obtained. The final task allocation scheme is then subjected to feasibility repair to ensure that it meets all the original constraints. The repaired scheme is the optimized resource scheduling strategy.

6. The energy efficiency optimization method for high-density server clusters based on resource scheduling according to claim 5, characterized in that, The step of issuing the resource scheduling control command to the underlying resource manager of the high-density server cluster, and having the underlying resource manager perform dynamic migration of computing tasks and adjustment of the working status of computing nodes, specifically involves: The underlying resource manager receives the resource scheduling control command and performs security and legality verification on it; After the verification is successful, the underlying resource manager first sends a task pause and checkpoint creation command to the original computing node agent where the task to be migrated is located, according to the instruction sequence; The original compute node agent executes commands to pause the execution of the specified compute task, serializes and saves its memory state and processor state as a checkpoint file, and freezes related input / output operations. The underlying resource manager coordinates the original compute node agent and the target compute node agent to transmit the checkpoint file and related context data to the target compute node via the network; After receiving the data, the target computing node agent recreates the execution environment of the computing task on the target computing node according to the instructions, restores its state from the checkpoint file, and then resumes the execution of the computing task. During or after task migration, the underlying resource manager sends instructions to the relevant computing nodes to adjust their operating frequency or voltage according to the node working status specified in the scheduling policy, in order to achieve the energy efficiency optimization goal.

7. The energy efficiency optimization method for high-density server clusters based on resource scheduling according to claim 6, characterized in that, The original computing node proxy executes commands to pause the execution of the specified computing task, serializes and saves its memory state and processor state as a checkpoint file, and freezes related input / output operations, specifically including: The original computing node agent sends a pause signal to the process of the target computing task, causing the process to enter an interruptible sleep state. Traverse the virtual address space of the process and copy the contents of all its resident memory pages to a pre-allocated buffer area; Read the contents of the processor register set of the process, including general-purpose registers, program counter, stack pointer and flag register, and pack them together with memory page data; For the file descriptors and network sockets opened by the process, record their current file offset and socket status metadata information; The memory page data, processor register data, and input / output metadata information are serialized and encoded, and written to a persistently stored checkpoint file. During checkpoint creation, all new input / output requests to the process are blocked, and the process waits for existing input / output operations to complete to ensure data consistency.

8. The energy efficiency optimization method for high-density server clusters based on resource scheduling according to claim 3, characterized in that, The construction steps of the pre-built load characteristic analysis model include: Collect historical multi-dimensional operational status data of multiple computing nodes in a high-density server cluster during historical periods, and label the historical multi-dimensional operational status data to obtain a training dataset with real load labels. The training dataset is preprocessed, including missing value imputation, outlier handling, and data standardization. Historical feature indicators corresponding to the key load feature indicators are extracted from the preprocessed training dataset to form a training feature vector set. Initialize a gradient boosting decision tree model framework and configure the model hyperparameters, including the number of base learners, the learning rate, and the maximum depth of the tree; Using the training feature vector set and the corresponding real load labels, the initialized gradient boosting decision tree model is iteratively trained with the goal of minimizing the prediction error. The performance of the trained gradient boosting decision tree model is evaluated on the validation dataset, and hyperparameters are tuned based on the evaluation results until the model performance reaches the preset standard, thus obtaining the pre-built load feature analysis model.

9. The energy efficiency optimization method for high-density server clusters based on resource scheduling according to claim 4, characterized in that, Before constructing the decision variables for the global resource scheduling model, the method further includes the step of constructing a computing node power consumption estimation model: Based on the hardware specifications and historical operating data of the computing nodes, a functional relationship is established between the power consumption of the computing nodes and the CPU utilization, memory usage, operating frequency and ambient temperature. By training offline, multiple linear regression or support vector regression algorithms are used to fit the coefficients of the functional relationship to obtain the power consumption estimation model for each computing node. The power consumption estimation model is embedded as a parameter into the objective function of the global resource scheduling model to calculate the dynamic power consumption of computing nodes under different loads and configurations.

Images