A resource management method, system and device for a data center

Abstract

The application discloses a resource management method, system and device for a data center, and relates to the technical field of cloud computing; work load time series is collected and input into a prediction model to obtain resource remaining amounts of each cloud computing node in a future preset time window and corresponding prediction uncertainty intervals and to determine resource reservation amounts of each cloud computing node; a task list to be scheduled at present is acquired; a multi-objective resource scheduling optimization model is constructed based on resource demand amounts of each task and the resource reservation amounts of each node; the multi-objective resource scheduling optimization model is solved through an adaptive optimization algorithm to generate an allocation scheme of each task to a cloud computing node; deviation between actual load and predicted demand is monitored during task operation; when the deviation exceeds a preset tolerance condition, the migration cost of the task and the expected income after migration is completed are calculated, and when the income / cost ratio exceeds a set threshold, a migration operation is performed, thereby improving the efficiency of resource management of the cloud computing data center.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing technology, specifically relating to a resource management method, system, and device for data centers. Background Technology

[0002] Global data center operations are rapidly transforming towards AI-driven, automated, knowledge-based, and platform-based approaches, with intelligent agents, large-scale models, and data platforms becoming mainstream technologies. This is driven by the comprehensive advancement of the East-West Data Center project, the explosive growth in AI computing power demand, and the increasing demands for cost reduction and efficiency improvement from enterprises. As data center deployments expand and the types of services they support increase, workloads exhibit high dynamism, suddenness, and spatiotemporal coupling, placing more stringent demands on resource scheduling efficiency, energy consumption control, service stability, and resource utilization. Traditional data center resource management methods often rely on manual experience-based configuration, static threshold strategies, or simple time-series statistical predictions, making them ill-suited for large-scale, high-concurrency, and highly volatile complex business scenarios, and thus exhibiting numerous shortcomings in practical applications.

[0003] Publication No. CN119225969A discloses a data center resource management method and device. This scheme allocates resources through modeling and iterative solution. However, existing load forecasting methods are inaccurate. When forecasting deviations occur, resource reservation is prone to falling into the dilemma of excessive redundancy and insufficient reservation. Using the reservation result with deviation as a scheduling constraint will cause resource scheduling to be based on a mismatch with the actual resource status from the initial stage. Under this premise, the scheduling model lacks a multi-objective collaborative balancing mechanism and adaptive optimization capability. It not only cannot dynamically correct the mismatch constraint, but also further aggravates the contradiction between energy consumption, task completion time and resource utilization, ultimately leading to low efficiency of resource management in cloud computing data centers. Summary of the Invention

[0004] The purpose of this invention is to solve the problem of low resource management efficiency in cloud computing data centers, and to propose a resource management method, system and equipment for data centers.

[0005] In a first aspect of this invention, a resource management method for a data center is first proposed, the method comprising: The workload time series of each cloud computing node in the cloud computing data center is collected, and the workload time series is input into the trained prediction model to obtain the remaining resources of each cloud computing node within a future preset time window and the corresponding prediction uncertainty interval. The resource reservation amount for each cloud computing node is determined based on the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval. Get the list of tasks currently waiting to be scheduled; each task has a corresponding resource requirement. Based on the resource requirements of each task and the resource reservation of each node, a multi-objective resource scheduling optimization model is constructed with the goal of minimizing total energy consumption, task completion time and maximizing resource utilization. The multi-objective resource scheduling optimization model is solved by an adaptive optimization algorithm to generate an allocation scheme for each task to a cloud computing node. The allocation scheme is executed, and the deviation between the actual load and the predicted demand is monitored during task execution. When the deviation exceeds the preset tolerance condition, the migration cost of the task is calculated and the expected benefit after the migration is completed. When the benefit-cost ratio exceeds the set threshold, the migration operation is performed.

[0006] Optionally, the prediction model is a deep neural network model based on a spatiotemporal attention mechanism, including an embedding layer, a position encoding layer, at least one spatiotemporal attention block, and an output layer; the spatiotemporal attention block sequentially includes a temporal attention module, a spatial attention module, a feedforward network, layer normalization, and residual connections; The workload time series is substituted into the embedding layer and mapped to a high-dimensional feature space through linear transformation to obtain high-dimensional features; The high-dimensional features are substituted into the position coding layer to generate time-position codes, and then summed with the high-dimensional feature embeddings to obtain fused features. After substituting the fused features into at least one spatiotemporal attention block, the output layer is used to obtain the predicted values ​​and confidence values ​​of each cloud computing node within a future preset time window. For each cloud computing node, the remaining resources are determined based on the predicted value and the total computing power of that cloud computing node; the uncertainty interval of the prediction is determined based on the confidence value.

[0007] Based on a multi-head spatiotemporal attention deep neural network architecture, it can simultaneously capture the temporal evolution patterns of load data and the spatial correlation features between nodes. Position encoding supplements the temporal position information, and residual connections and layer normalization ensure the stability of deep network training and the integrity of feature transmission. Through feature embedding, it achieves high-dimensional representation of data and outputs predicted values ​​and confidence levels simultaneously. It can accurately estimate the remaining amount of node resources and predict the uncertainty range based on confidence levels, so that subsequent resource reservation and scheduling decisions have data support and risk reference, thereby improving the accuracy of load prediction.

[0008] Optionally, the joint loss function of the prediction model is: ; Where N is the number of training samples, and i is the sample index. For the actual workload of the i-th sample, The predicted value output by the model. As the first weighting coefficient, The physical load limit of a node, A positive penalty is applied when the predicted value exceeds the physical limit; otherwise, the penalty is zero. Here, q is the second weighting coefficient, Q is the set of quantiles, and q is a specific quantile in set Q. Let q be the predicted value of the qth quantile. This is the quantile loss function.

[0009] Optionally, the resource reservation amount for each cloud computing node is determined based on the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval, and is calculated by the formula A=B-θ*C. Where A is the resource reserve, B is the remaining resource, C is half the width of the prediction uncertainty interval, and θ is a constant coefficient.

[0010] Optionally, based on the resource requirements of each task and the resource reservation of each node, a multi-objective resource scheduling optimization model is constructed with the objectives of minimizing total energy consumption, task completion time, and maximizing resource utilization. This model includes: Define the total energy consumption function ; Where K represents the total number of cloud computing nodes. Let be the idle power consumption of node i. Let be the load power consumption coefficient of node i. Let i be the load rate of node i; Define the task completion time function ; in Let be a binary decision variable, representing whether task j is assigned to node i. Let be the expected execution time of task j on node i; Define resource utilization function ; Where M is the total number of tasks to be scheduled. Let represent the dominant resource requirement of task j for node i. The effective available capacity of node i; By using the weighted Chebyshev method, the total energy consumption function, task completion time function, and resource utilization function are transformed into a single-objective fitness function to obtain a multi-objective resource scheduling optimization model.

[0011] By quantifying and constructing three types of core objective functions, and using the weighted Chebyshev method to transform multi-objective problems into single-objective optimization problems, the difficulty of solving problems is simplified and the computational efficiency is improved. The weights can be flexibly adapted to the priority requirements of different business scenarios, making the scheduling model rigorous, practical and adjustable, and ensuring the overall optimal scheduling results.

[0012] Optionally, the multi-objective resource scheduling optimization model can be solved using an adaptive optimization algorithm to generate an allocation scheme for each task to a cloud computing node, including: Each candidate solution is encoded as an integer vector of length equal to the total number of tasks to be scheduled, with each dimension representing the node number of the cloud computing node, indicating the node to which the corresponding task is assigned. An initial population containing M candidate solutions is randomly generated, and the fitness value of each candidate solution under the multi-objective resource scheduling optimization model is calculated. The control score is obtained through a time control function. When the control score is less than a preset score threshold, the first update scheme is executed; otherwise, the second update scheme is executed. Using the maximum fitness value as a condition, the update scheme is repeatedly executed until the preset maximum number of iterations is reached. The solution with the maximum current fitness value is then output as the allocation scheme for each task to the cloud computing node.

[0013] By dynamically switching between different candidate solution update schemes using a time control function, an adaptive balance can be achieved between global exploration and local development, thereby efficiently approximating the Pareto optimal solution within a limited number of iterations. At the same time, with the goal of maximizing fitness, the energy consumption of multi-objective conflicts, task completion time, and resource utilization are transformed into single-objective optimization, which significantly improves the quality and stability of scheduling decisions. The final task node allocation scheme can simultaneously reduce the total energy consumption of the data center, shorten the task completion time, and improve the utilization of heterogeneous resources.

[0014] Optionally, the first update scheme is through a formula Determine the position of the current global optimal solution and update the candidate solution; where, `round` rounds the result to the nearest integer, and `rand` generates a random number. Let be the assignment node for the j-th task in the i-th candidate solution. This is the node assigned to the j-th task in the current global optimal solution, where r and rand are random numbers in [0,1]. The second update scheme includes a passive update scheme and an active update scheme; The passive update scheme is based on the formula... Update; among them, The motion coefficient, The upper bound of the node number, This is the lower bound of the node number. This is the i-th candidate solution; The active update scheme involves randomly selecting another candidate solution. Through formula Compare fitness values Update; among them, Let be the fitness value of the i-th candidate solution. The fitness value of the j-th candidate solution; through Update.

[0015] Through the aforementioned adaptive optimization algorithm, the first update scheme can explore a wide range of solutions, effectively avoiding the algorithm from getting trapped in local optima too early. The passive and active updates in the second update scheme perform a refined search near the discovered high-quality solutions. By dynamically switching between the two update schemes with the help of a time control function, the algorithm can automatically balance between global exploration and local development, ensuring sufficient coverage of the solution space in the early stage and accelerating the convergence speed in the later stage. Thus, under the premise of satisfying the node capacity constraint, it efficiently generates a task allocation scheme that is optimal for multiple objectives, significantly improving the robustness of scheduling decisions and the quality of the final solution.

[0016] Optionally, the migration cost of computational tasks includes: Through formula The cost of migrating computational tasks; in, Save the time for task status checkpoints, through Calculations show that Let J be the amount of runtime state data for task j. This represents the disk write bandwidth of the source node. For the transmission time of task status data over the network, via Calculations show that For source node i and target node Available network bandwidth between This reduces cache warm-up and memory allocation delays on the target node.

[0017] In a second aspect of this invention, a resource management system for a data center is provided, comprising: The preset module is used to collect the workload time series of each cloud computing node in the cloud computing data center, input the workload time series into the trained prediction model, and obtain the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval within the preset time window in the future. The resource reservation determination module is used to determine the resource reservation amount for each cloud computing node based on the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval. The task list determination module is used to obtain the current list of tasks to be scheduled; each task has a corresponding resource requirement. The optimization model determination module is used to construct a multi-objective resource scheduling optimization model based on the resource requirements of each task and the resource reservation of each node, with the objectives of minimizing total energy consumption, task completion time and maximizing resource utilization. The allocation scheme determination module is used to solve the multi-objective resource scheduling optimization model through an adaptive optimization algorithm to generate an allocation scheme for each task to the cloud computing node; A monitoring module is used to execute the allocation scheme and monitor the deviation between the actual load and the predicted demand during task execution; The migration module is used to calculate the migration cost of the task and the expected benefit after the migration is completed when the deviation exceeds the preset tolerance condition, and to perform the migration operation when the benefit-cost ratio exceeds the set threshold.

[0018] In a third aspect of the invention, an electronic device is provided, characterized in that it comprises: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements a resource management method for a data center.

[0019] Beneficial effects: By combining time-series load forecasting, uncertainty considerations, multi-objective optimization scheduling, and dynamic task migration mechanisms, it can accurately predict node resource status and reasonably reserve resources, taking into account the three core indicators of system energy consumption, task timeliness, and resource utilization. It can also achieve dynamic adaptive optimization by relying on deviation monitoring and benefit-cost assessment, effectively avoiding resource waste and task congestion caused by load forecast deviations, improving the overall operational stability, resource utilization efficiency, and energy efficiency of the data center, while ensuring efficient and reliable task operation and improving the resource management efficiency of cloud computing data centers. Attached Figure Description

[0020] The invention will now be further described with reference to the accompanying drawings.

[0021] Figure 1 A flowchart illustrating a resource management method for a data center, provided as an embodiment of the present invention; Figure 2 The following is an execution flowchart of a prediction model provided in an embodiment of the present invention; Figure 3 This invention provides a schematic diagram of the structure of an electronic device. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0023] It should be noted that all formula calculations in the scheme are purely numerical calculations.

[0024] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] This invention provides a resource management method for data centers. See also... Figure 1 The method includes the following steps: S101: Collect the workload time series of each cloud computing node in the cloud computing data center, input the workload time series into the trained prediction model, and obtain the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval within the future preset time window. S102, Determine the resource reservation amount for each cloud computing node based on the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval; S103, Get the list of tasks currently to be scheduled; each task has a corresponding resource requirement; S104. Based on the resource requirements of each task and the resource reservation of each node, a multi-objective resource scheduling optimization model is constructed with the goal of minimizing total energy consumption, task completion time and maximizing resource utilization. S105 uses an adaptive optimization algorithm to solve the multi-objective resource scheduling optimization model and generates an allocation scheme for each task to the cloud computing node. S106, execute the allocation plan and monitor the deviation between actual load and predicted demand during task execution; S107: When the deviation exceeds the preset tolerance condition, calculate the migration cost of the task and the expected benefit after the migration is completed. When the benefit-cost ratio exceeds the set threshold, execute the migration operation.

[0026] In one implementation, a data center monitoring system periodically collects multi-dimensional workload metrics for each cloud computing node, including: CPU utilization, memory usage, GPU utilization, GPU memory usage, network transmit and receive throughput, disk read / write speed, and current power consumption. The sampling interval is typically set to 1–5 minutes. For each node, the metric vectors of the sampling points within a preset time window (determined by technical personnel) are arranged in chronological order to form a target tensor, which is the workload time series.

[0027] In one implementation, the target tensor is substituted into the prediction model to obtain the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval. The prediction model adopts a deep neural network based on a spatiotemporal attention mechanism. This model includes an input embedding layer, sine and cosine position encoding, and multiple stacked spatiotemporal attention blocks. Each block includes a temporal attention module, a spatial attention module, a feedforward network, and layer normalization. The output layer simultaneously outputs the point prediction value and multiple quantiles. For each node i, the point prediction value and quantile for the next H time steps are obtained. Then, the resource reservation of the node is calculated using the formula A=B-θ*C. Where A is the resource reservation, B is the remaining resource, C is half the width of the prediction uncertainty interval, and θ is a constant coefficient with a value between 0.4 and 0.6, which is determined by the technical personnel.

[0028] In one implementation, a container orchestration platform initiates or migrates tasks to designated nodes. Real-time data is collected on the actual resource consumption of each node to calculate the deviation between the actual load and the predicted value. If the deviation exceeds a preset tolerance condition, the migration cost and expected return after migration are calculated. Migration is performed when the return-cost ratio exceeds a set threshold; otherwise, no migration is performed. The tolerance condition is half the prediction uncertainty interval. For node i, the absolute error between the actual load and the predicted value is calculated within a time window. The calculation of the task migration cost includes: using the formula... Calculate the migration cost of the task; where, Save the time for task status checkpoints, through Calculations show that Let J be the amount of runtime state data for task j. This represents the disk write bandwidth of the source node. For the transmission time of task status data over the network, via Calculations show that For source node i and target node Available network bandwidth between Preheating the cache and delaying memory allocation on the target node; Migration benefits The remaining completion time of the task on the source node is subtracted from the estimated completion time on the target node using the formula. Calculation Let represent the remaining computational cost for task j. For node n, there are pairs of task types. The performance coefficients were determined by technicians through short-run benchmark tests on the target node; The benefit-cost ratio is Once the benefit-cost ratio is calculated, it is compared with a set threshold (determined by technical personnel). If the threshold is exceeded, the migration operation is executed; otherwise, it is not executed.

[0029] In one implementation, by introducing a prediction model based on a spatiotemporal attention mechanism, it is possible not only to capture the temporal evolution of the load of a single node, but also to model the spatial correlation between different resource indicators or different task flows, which significantly improves the prediction accuracy. At the same time, it outputs quantile prediction values, which quantifies the uncertainty of the prediction and provides a quantitative basis for subsequent adaptive resource reservation, overcoming the deficiency that traditional point prediction cannot express confidence.

[0030] One implementation utilizes dynamic cloud computing node resource reservation within the uncertainty interval of prediction, allowing the reservation amount to adaptively change with the prediction confidence: reserving fewer resources when the prediction is clear to improve utilization, and increasing reservations when the prediction is ambiguous to ensure service quality. Compared to traditional fixed-ratio reservations, this reduces the risks of resource idleness and overload, achieving an intelligent balance between reliability and efficiency.

[0031] In one implementation, three conflicting objectives—total energy consumption, task completion time, and resource utilization—are incorporated into a unified optimization framework and transformed into a single objective function using the weighted Chebyshev method. This comprehensively reflects the multidimensional demands for cost, performance, and efficiency in data center operations. Compared to schedulers that only optimize a single objective, this approach can generate high-quality compromise solutions on the Pareto front, satisfying differentiated preferences in different scenarios.

[0032] In one embodiment, see Figure 2 , Figure 2 A flowchart of the execution of a prediction model is provided. The prediction model is a deep neural network model based on a spatiotemporal attention mechanism, including an embedding layer, a position encoding layer, at least one spatiotemporal attention block, and an output layer. The spatiotemporal attention block includes a temporal attention module, a spatial attention module, a feedforward network, layer normalization, and residual connections. S1011, Substitute the workload time series into the embedding layer and map it to a high-dimensional feature space through linear transformation to obtain high-dimensional features; S1012, substitute the high-dimensional features into the position coding layer to generate time position codes, and sum them with the high-dimensional feature embeddings to obtain fused features; S1013, after substituting the fused features into at least one spatiotemporal attention block, the output layer is used to obtain the predicted value and confidence value of each cloud computing node within the future preset time window; S1014, for each cloud computing node, determine the remaining resource amount based on the predicted value and the total computing power of the cloud computing node; determine the prediction uncertainty interval based on the confidence value.

[0033] In one implementation, multi-dimensional workload data of each cloud computing node over past time steps is obtained from the data center monitoring system. The feature vector of each time step includes CPU utilization, memory usage, GPU utilization, video memory usage, network throughput, power consumption, etc. All features are normalized to form an input tensor. The features are mapped to a high-dimensional latent space through a learnable linear transformation. Since the self-attention mechanism is permutation invariant, positional information must be explicitly added. Therefore, sine and cosine positional encoding is used to finally obtain the encoded features. Then, the encoded features and the high-dimensional features are averaged to obtain the fused features.

[0034] In one implementation, a temporal attention module is used to capture temporal dependencies within a single workload stream; based on input features, through... , , The Q value was calculated. K value and V value Then, the Q-value, K-value, and V-value are substituted into the softmax activation function to obtain the result. This module groups the input features according to the workload type dimension and processes the time series of each workload stream independently. It transforms the input into three matrices through learnable query, key, and value projection, and then substitutes them into the softmax activation function. It also uses causal masking to prevent information from future time steps from leaking to the current step. In this way, the model can capture the long-short-term dependencies within a single workload stream.

[0035] In one implementation, the output of the spatial attention module is subsequently reshaped with the time step as the axis and the workload type as the sequence length. The spatial attention module computes attention across these different types, revealing the interactions between different workloads. Unlike temporal attention, spatial attention does not use causal masks because different workloads at the same time can reasonably influence each other.

[0036] In one implementation, the output of each attention module is residually connected to the module's input, and then the training process is stabilized through layer normalization. This facilitates gradient flow and allows the model to stack multiple attention layers without degradation. Finally, the normalized features are fed into a simple fully connected feedforward network containing two linear layers and an activation function to further extract nonlinear features. The output of this network is also added to its input and normalized through layer normalization, completing the computation of a spatiotemporal attention block.

[0037] In one implementation, after passing through several stacked spatiotemporal attention blocks, the final hidden state is fed into a fully connected output layer. This output layer simultaneously produces two types of results: first, the point prediction value for each time step within a future preset time window; and second, multiple quantile prediction values. These quantile predictions are directly used to construct the uncertainty interval of the prediction: the range between the upper and lower quantiles is the model's confidence interval for the current prediction, and the wider the range, the higher the uncertainty.

[0038] In one implementation, the remaining resource amount is determined based on the predicted value and the total computing power of the cloud computing node by subtracting the predicted value from the total computing power of the cloud computing node. The confidence value is a set of quantile predicted values. The quantiles output by the prediction model refer to the probability that the actual workload will be lower than a certain value at a future point in time. For example, the 10th quantile indicates a 10% probability that the actual workload will be lower than this value, and a 90% probability that it will be higher than this value; the 50th quantile, also called the median, indicates a 50% probability that the workload will be lower than it and a 50% probability that it will be higher than it; the 90th quantile indicates... There's a 90% probability that the actual load is lower than this value, and only a 10% probability that it's higher. We obtain two quantiles, such as the 10th quantile and the 90th quantile. The range between these two values ​​constitutes an 80% prediction uncertainty interval. For example, assuming the model outputs the average utilization of a GPU node over the next 30 minutes as follows: 42% for the 10th quantile; 55% for the 50th quantile (point prediction); and 68% for the 90th quantile, then the 80% prediction uncertainty interval is [42%, 68%]. The half-width of this interval is (68% - 42%) / 2 = 13%. This half-width can be directly used for the security resource reservation of cloud computing nodes.

[0039] In one implementation, the joint loss function of the prediction model is: ; Where N is the number of training samples, and i is the sample index. For the actual workload of the i-th sample, The predicted value output by the model. As the first weighting coefficient, The physical load limit of a node. A positive penalty is applied when the predicted value exceeds the physical limit; otherwise, the penalty is zero. Here, q is the second weighting coefficient, Q is the set of quantiles, and q is a specific quantile in set Q. Let q be the predicted value of the qth quantile. This is the quantile loss function.

[0040] In one implementation, the first weighting coefficient controls the strength of the capacity penalty term and is set to 0.5; the second weighting coefficient controls the strength of the quantile loss term and is set to 0.3; the quantile loss function is... calculate, That is, the error between the actual value and the predicted quantile value. It is an indicator function, when The value is 1 when it is active, and zero otherwise.

[0041] In one embodiment, a multi-objective resource scheduling optimization model is constructed based on the resource requirements of each task and the resource reservation of each node, with the objectives of minimizing total energy consumption, task completion time, and maximizing resource utilization. Define the total energy consumption function ; Where K represents the total number of cloud computing nodes. Let be the idle power consumption of node i. Let be the load power consumption coefficient of node i. Let i be the load rate of node i; Define the task completion time function ; in Let be a binary decision variable, representing whether task j is assigned to node i. Let be the expected execution time of task j on node i; Define resource utilization function ; Where M is the total number of tasks to be scheduled. Let represent the dominant resource requirement of task j for node i. The effective available capacity of node i; By using the weighted Chebyshev method, the total energy consumption function, task completion time function, and resource utilization function are transformed into a single-objective fitness function to obtain a multi-objective resource scheduling optimization model.

[0042] In one implementation, optimizing only a single objective can easily lead to over-concentration of tasks, causing local hotspots and queue congestion; optimizing only speed may blindly open a large number of nodes, resulting in soaring energy consumption. This model incorporates three objectives simultaneously, forcing the solution process to seek Pareto optimal trade-offs among them. The final allocation scheme will not exhibit extreme degradation in any dimension, better meeting the comprehensive needs of actual operation and maintenance. In one implementation, the weighted Chebyshev method is used to aggregate the total energy consumption function, task completion time function, and resource utilization function into a single-objective fitness function. The aggregation form is obtained through the formula... ,in Let be the total energy consumption function. For task completion time function, Let the resource utilization rate function be... , , These are the minimum values ​​that can be achieved when each objective is optimized individually. , , The weights are preset and sum to 1. The specific values ​​are determined by technical personnel. The optimization objective is to minimize the single-objective fitness function, thus obtaining a multi-objective resource scheduling optimization model.

[0043] In one implementation, the total energy consumption function distinguishes between the idle power consumption and the load power consumption coefficient of a node, accurately depicting the energy consumption changes of cloud computing nodes under different load rates, avoiding the bias in energy-saving strategies caused by linear coarse estimation. Based on this function optimization, the algorithm prioritizes integrating tasks onto nodes with higher energy efficiency, effectively reducing overall electricity and heat dissipation costs. The task completion time function, through a combination of binary decision variables and the expected execution time of tasks on each node, directly reflects the impact of the scheduling scheme on the overall completion time of batch tasks. This objective prompts the system to avoid stacking too many tasks onto weaker nodes, thereby shortening the user's perceived waiting time and meeting the service level agreement. The resource utilization function is measured by the ratio of dominant resource demand to effective available capacity. The introduction of dominant resources allows for automatic balancing of the allocation of multiple types of resources during optimization, preventing fragmentation where the CPU has a large amount of idle space but the memory is full and unable to accept more tasks, truly achieving global efficient utilization of multi-dimensional resources.

[0044] In one embodiment, the multi-objective resource scheduling optimization model is solved using an adaptive optimization algorithm to generate an allocation scheme for each task to a cloud computing node, including: Each candidate solution is encoded as an integer vector of length equal to the total number of tasks to be scheduled, with each dimension representing the node number of the cloud computing node, indicating the node to which the corresponding task is assigned. An initial population containing M candidate solutions is randomly generated, and the fitness value of each candidate solution under the multi-objective resource scheduling optimization model is calculated. Through time control function The control score is obtained. If the control score is less than a preset score threshold, the first update scheme is executed; otherwise, the second update scheme is executed. t represents the number of iterations at that time. Preset maximum number of iterations; Using the maximum fitness value as a condition, the update scheme is repeatedly executed until the preset maximum number of iterations is reached. The solution with the maximum current fitness value is then output as the allocation scheme for each task to the cloud computing node.

[0045] In one implementation, the maximum number of iterations is predetermined by technical personnel; the fitness value is the value of a single-objective fitness function, which is converted from the total energy consumption function, task completion time function, and resource utilization function using the weighted Chebyshev method, with the objective being optimality; the first update scheme's update mechanism has a wide range of random perturbation characteristics, which can maintain population diversity and effectively escape local optima; the passive movement and directional active movement of the second update scheme can perform fine mining around the current high-quality solution, accelerating convergence to a high-precision solution, and the combination of the two can significantly improve the search capability in a high-dimensional, multi-peaked scheduling solution space with complex constraints.

[0046] In one implementation, a time control function is used to generate control scores and set a threshold, so that the algorithm executes the first update scheme with a high probability in the early stage of iteration, extensively scanning the solution space; in the later stage of iteration, it gradually switches to the second update scheme to finely optimize the current optimal region. This adaptive mechanism can automatically adjust the search focus without manual parameter tuning, which not only ensures the global optimality of the solution but also speeds up the convergence speed, perfectly meeting the timeliness requirements of online scheduling.

[0047] In one implementation, the first update scheme is through a formula. Determine the position of the current global optimal solution and update the candidate solution; where, `round` rounds the result to the nearest integer, and `rand` generates a random number. Let be the assignment node for the j-th task in the i-th candidate solution. This is the node assigned to the j-th task in the current global optimal solution, where r and rand are random numbers in [0,1]. The second update plan includes a passive update plan and an active update plan; The passive update scheme is to use the formula Update; among them, The motion coefficient, The upper bound of the node number, This is the lower bound of the node number. This is the i-th candidate solution; The proactive update scheme involves randomly selecting another candidate solution. Through formula Compare fitness values Update; among them, Let be the fitness value of the i-th candidate solution. The fitness value of the j-th candidate solution; through Update.

[0048] In one implementation, the motion coefficient takes the value (0,1), which is determined by the technical personnel.

[0049] This invention also provides an electronic device, such as... Figure 3 As shown, it includes a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304. Memory 303 is used to store computer programs; When processor 301 executes a program stored in memory 303, it performs the following steps: The workload time series of each cloud computing node in the data center is collected, and the workload time series is input into the trained prediction model to obtain the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval within a future preset time window. The resource reservation amount for each cloud computing node is determined based on the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval. Get the list of tasks currently waiting to be scheduled; each task has a corresponding resource requirement. Based on the resource requirements of each task and the resource reservation of each node, a multi-objective resource scheduling optimization model is constructed with the goal of minimizing total energy consumption, task completion time and maximizing resource utilization. The multi-objective resource scheduling optimization model is solved by an adaptive optimization algorithm to generate an allocation scheme for each task to a cloud computing node. The allocation scheme is executed, and the deviation between the actual load and the predicted demand is monitored during task execution. When the deviation exceeds the preset tolerance condition, the migration cost of the task is calculated and the expected benefit after the migration is completed. When the benefit-cost ratio exceeds the set threshold, the migration operation is performed.

[0050] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0051] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0052] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0053] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0054] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0055] The foregoing has described one embodiment of the present invention in detail, but this content is merely a preferred embodiment and should not be considered as limiting the scope of the present invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the scope of the claims of this invention.

Claims

1. A resource management method for a data center, characterized in that, The method includes: The workload time series of each cloud computing node in the cloud computing data center is collected, and the workload time series is input into the trained prediction model to obtain the remaining resources of each cloud computing node within a future preset time window and the corresponding prediction uncertainty interval. The resource reservation amount for each cloud computing node is determined based on the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval. Get the list of tasks currently waiting to be scheduled; each task has a corresponding resource requirement. Based on the resource requirements of each task and the resource reservation of each node, a multi-objective resource scheduling optimization model is constructed with the goal of minimizing total energy consumption, task completion time and maximizing resource utilization. The multi-objective resource scheduling optimization model is solved by an adaptive optimization algorithm to generate an allocation scheme for each task to a cloud computing node. The allocation scheme is executed, and the deviation between the actual load and the predicted demand is monitored during task execution. When the deviation exceeds the preset tolerance condition, the migration cost of the task is calculated and the expected benefit after the migration is completed. When the benefit-cost ratio exceeds the set threshold, the migration operation is performed.

2. The resource management method for a data center according to claim 1, characterized in that, The prediction model is a deep neural network model based on a spatiotemporal attention mechanism, including an embedding layer, a position encoding layer, at least one spatiotemporal attention block, and an output layer; the spatiotemporal attention block includes a temporal attention module, a spatial attention module, a feedforward network, layer normalization, and residual connections in sequence. The workload time series is substituted into the embedding layer and mapped to a high-dimensional feature space through linear transformation to obtain high-dimensional features; The high-dimensional features are substituted into the position coding layer to generate time-position codes, and then summed with the high-dimensional feature embeddings to obtain fused features. After substituting the fused features into at least one spatiotemporal attention block, the output layer is used to obtain the predicted values ​​and confidence values ​​of each cloud computing node within a future preset time window. For each cloud computing node, the remaining resources are determined based on the predicted value and the total computing load of that cloud computing node; The uncertainty interval for prediction is determined based on the confidence level value.

3. A resource management method for a data center according to claim 2, characterized in that, The joint loss function of the prediction model is ； Where N is the number of training samples, and i is the sample index. For the actual workload of the i-th sample, The predicted value output by the model. As the first weighting coefficient, The physical load limit of a node. A positive penalty is applied when the predicted value exceeds the physical limit; otherwise, the penalty is zero. Here, q is the second weighting coefficient, Q is the set of quantiles, and q is a specific quantile in set Q. Let q be the predicted value of the qth quantile. This is the quantile loss function.

4. A resource management method for a data center according to claim 1, characterized in that, The resource reservation amount for each cloud computing node is determined based on the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval, and is calculated using the formula A=B-θ*C. Where A is the resource reserve, B is the remaining resource, C is half the width of the prediction uncertainty interval, and θ is a constant coefficient.

5. A resource management method for a data center according to claim 1, characterized in that, Based on the resource requirements of each task and the resource reservation of each node, a multi-objective resource scheduling optimization model is constructed with the objectives of minimizing total energy consumption, task completion time, and maximizing resource utilization. This model includes: Define the total energy consumption function ; Where K represents the total number of cloud computing nodes. Let be the idle power consumption of node i. Let be the load power consumption coefficient of node i. Let i be the load rate of node i; Define the task completion time function ; in Let be a binary decision variable, representing whether task j is assigned to node i. Let be the expected execution time of task j on node i; Define resource utilization function ; Where M is the total number of tasks to be scheduled. Let represent the dominant resource requirement of task j for node i. The effective available capacity of node i; By using the weighted Chebyshev method, the total energy consumption function, task completion time function, and resource utilization function are transformed into a single-objective fitness function to obtain a multi-objective resource scheduling optimization model.

6. A resource management method for a data center according to claim 1, characterized in that, The multi-objective resource scheduling optimization model is solved using an adaptive optimization algorithm to generate allocation schemes for each task to cloud computing nodes, including: Each candidate solution is encoded as an integer vector of length equal to the total number of tasks to be scheduled, with each dimension representing the node number of the cloud computing node, indicating the node to which the corresponding task is assigned. An initial population containing M candidate solutions is randomly generated, and the fitness value of each candidate solution under the multi-objective resource scheduling optimization model is calculated. The control score is obtained through a time control function. When the control score is less than a preset score threshold, the first update scheme is executed; otherwise, the second update scheme is executed. Using the maximum fitness value as a condition, the update scheme is repeatedly executed until the preset maximum number of iterations is reached. The solution with the maximum current fitness value is then output as the allocation scheme for each task to the cloud computing node.

7. A resource management method for a data center according to claim 6, characterized in that, The first update scheme is through the formula Determine the position of the current global optimal solution and update the candidate solution; where, `round` rounds the result to the nearest integer, and `rand` generates a random number. Let be the assignment node for the j-th task in the i-th candidate solution. This is the node assigned to the j-th task in the current global optimal solution, where r and rand are random numbers in [0,1]. The second update scheme includes a passive update scheme and an active update scheme; The passive update scheme is based on the formula... Update; among them, The motion coefficient, The upper bound of the node number, This is the lower bound of the node number. This is the i-th candidate solution; The active update scheme involves randomly selecting another candidate solution. Through formula Compare fitness values Update; among them, Let be the fitness value of the i-th candidate solution. The fitness value of the j-th candidate solution; through Update.

8. A resource management method for a data center according to claim 1, characterized in that, The migration costs of computational tasks include: Through formula The cost of migrating computational tasks; in, Save the time for task status checkpoints, through Calculations show that Let J be the amount of runtime state data for task j. This represents the disk write bandwidth of the source node. For the transmission time of task status data over the network, via Calculations show that For source node i and target node Available network bandwidth between This reduces cache warm-up and memory allocation delays on the target node.

9. A resource management system for a data center, characterized in that, The system includes: The preset module is used to collect the workload time series of each cloud computing node in the cloud computing data center, input the workload time series into the trained prediction model, and obtain the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval within the preset time window in the future. The resource reservation determination module is used to determine the resource reservation amount for each cloud computing node based on the remaining resources of each cloud computing node and the corresponding prediction uncertainty interval. The task list determination module is used to obtain the current list of tasks to be scheduled; each task has a corresponding resource requirement. The optimization model determination module is used to construct a multi-objective resource scheduling optimization model based on the resource requirements of each task and the resource reservation of each node, with the objectives of minimizing total energy consumption, task completion time and maximizing resource utilization. The allocation scheme determination module is used to solve the multi-objective resource scheduling optimization model through an adaptive optimization algorithm to generate an allocation scheme for each task to the cloud computing node; A monitoring module is used to execute the allocation scheme and monitor the deviation between the actual load and the predicted demand during task execution; The migration module is used to calculate the migration cost of the task and the expected benefit after the migration is completed when the deviation exceeds the preset tolerance condition, and to perform the migration operation when the benefit-cost ratio exceeds the set threshold.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program to implement the steps of the resource management method for a data center as claimed in any one of claims 1 to 8.

Images