A data center task migration and scheduling method and system

Abstract

This invention discloses a data center task migration and scheduling method and system, comprising: acquiring and determining task load index data and its characteristics for each node in the data center; calculating the task load value for each node based on the load index characteristics, and determining the task load balance of the data center based on the load value; determining whether a task needs to be migrated based on the task load balance, and constructing a task migration model after determining that migration is necessary; generating a task migration plan based on the task migration model, and performing task migration based on the plan; evaluating the degree of improvement in the load balance of the data center after task migration, and dynamically adjusting the parameters during the task migration process based on the results. This invention can effectively eliminate load hotspots and idle nodes in the cluster, significantly reduce task response latency, improve system throughput, ensure load balance among nodes, and avoid local overload that reduces operational reliability and stability.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and in particular to a data center task migration and scheduling method and system. Background Technology

[0002] With the rapid development of cloud computing, big data, and artificial intelligence technologies, modern data centers have become the core infrastructure supporting the operation of the digital economy. Millions of physical servers work together to handle a variety of computing tasks, from online transactions and live video streaming to scientific computing and model training.

[0003] However, in such a large-scale, high-concurrency distributed environment, the task load on different nodes fluctuates drastically over time due to user behavior patterns, business promotion activities, or unplanned traffic surges. Traditional migration and scheduling methods lack effective load balancing mechanisms, and severe resource distribution imbalances can easily occur within the cluster: some nodes become overloaded due to task accumulation, leading to soaring service response latency, decreased throughput, and even cascading failures; while other nodes remain idle or underloaded for extended periods, resulting in the waste of valuable computing resources. This not only impairs user experience and service level agreement compliance but also directly reduces the overall resource efficiency and return on investment of the data center. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a data center task migration and scheduling method and system, comprising: Obtain task load metric data for each node in the data center and determine the load metric characteristics of the task load metric data. The task load of each node is calculated based on the load index characteristics to obtain the task load value, and the task load balance of the data center is determined based on the task load value of each node. Based on the task load balancing, determine whether a task needs to be migrated, and after determining that migration is necessary, construct a task migration model with the goal of achieving optimal load distribution balance in the data center. A task migration plan is generated based on the task migration model, and tasks on each node in the data center are migrated based on the task migration plan. After the task migration is completed, the degree of improvement in the load balancing of the data center is evaluated, and the parameters during the task migration process are dynamically adjusted based on the evaluation results.

[0005] Furthermore, the step of acquiring task load index data for each node in the data center and determining the load index characteristics of the task load index data includes: Obtain task load metrics data for each node in the data center, including CPU utilization data, memory utilization data, and network utilization data; Calculate the mean, peak value, and variance of the task load index data, and determine the mean, peak value, and variance as the load index characteristics of the task load index data.

[0006] Furthermore, the calculation of the task load of each node based on load metric characteristics to obtain the task load value includes: Determine the preset weights for each load indicator feature, and then calculate the subtask load value of each task load indicator data by weighted summation of each load indicator feature with the preset weights. Determine the task load metric data with the largest task load value among the subtask load values, and then determine the task load value of each node as the subtask load value with the largest task load metric data.

[0007] Furthermore, determining the task load balancing of the data center based on the task load values ​​of each node includes: The highest and lowest load nodes are determined based on the task load values ​​of each node, and the load ratio between the highest and lowest load nodes is calculated. The calculated ratio is determined as the task load balancing degree of the data center.

[0008] Furthermore, the step of determining whether a task needs to be migrated based on task load balancing includes: Determine the pre-set task load balancing threshold, and determine whether a task needs to be migrated based on the relationship between the data center's task load balancing and the task load balancing threshold; If the task load balancing degree of the data center is less than the task load balancing threshold, it is determined that the tasks on each node in the data center do not need to be migrated. If the task load balancing degree of the data center is greater than or equal to the task load balancing degree threshold, it is determined that the tasks of each node in the data center need to be migrated.

[0009] Furthermore, after determining that migration is necessary, the step of constructing a task migration model with the goal of optimal data center load distribution balance includes: After determining that migration is necessary, an objective function is constructed with the goal of minimizing the variance of the task load values ​​of all nodes in the data center after migration. A task migration model is constructed based on the objective function and constraints, with node resource capacity, task resource requirements, and migration cost as constraints.

[0010] Furthermore, the step of generating a task migration plan based on the task migration model and migrating tasks on each node in the data center based on the task migration plan includes: The task migration model is solved using a genetic algorithm to obtain the optimal solution for task migration, and a task migration plan is generated based on the optimal solution. The task migration plan sorts the nodes in the data center from high to low according to their task load values, resulting in a sequence of nodes to be migrated. Nodes with sufficient remaining resource capacity to accommodate the tasks to be migrated are identified as candidate nodes, and the candidate nodes are sorted from low to high according to their task load values ​​to obtain the target migration node sequence. Based on the sequence of nodes to be migrated and the sequence of target nodes to be migrated, the tasks are migrated in the order from front to back to migrate the tasks of each node in the data center.

[0011] Furthermore, after the task migration is completed, the degree of improvement in data center load balancing is evaluated, including: After the task migration is completed, calculate the task load balancing degree of the data center after migration, and calculate the difference between the task load balancing degree after migration and the task load balancing degree before migration to obtain the load balancing degree difference. Calculate the ratio of the load balance difference to the load balance before migration, and evaluate this ratio to obtain the degree of improvement in the load balance of the data center.

[0012] Furthermore, the dynamic adjustment of parameters during the task migration process based on the evaluation results includes: Determine a pre-set threshold for improvement level, and determine whether dynamic adjustment of parameters is needed during the task migration process based on the relationship between the data center load balancing improvement level value and the improvement level threshold; If the load balancing improvement value of the data center is greater than or equal to the improvement threshold, then there is no need to dynamically adjust the parameters during the task migration process. If the improvement value of the load balancing in the data center is less than the improvement threshold, the parameters during the task migration process need to be dynamically adjusted. If it is necessary to dynamically adjust the parameters during the task migration process, record the changes in the task load balancing of the data center before and after each migration to form a historical sample set, and use reinforcement learning algorithms to dynamically adjust the threshold parameters involved in the task migration process.

[0013] The present invention also provides a data center task migration and scheduling system, comprising: The acquisition module is used to acquire task load index data of each node in the data center and determine the load index characteristics of the task load index data. The calculation module is used to calculate the task load of each node based on the load index characteristics, obtain the task load value, and determine the task load balance of the data center based on the task load value of each node. The modeling module is used to determine whether a task needs to be migrated based on the task load balance, and after determining that migration is necessary, to build a task migration model with the goal of optimal load distribution balance in the data center. The migration module is used to generate a task migration plan based on the task migration model, and to migrate tasks on each node in the data center based on the task migration plan. The adjustment module is used to evaluate the improvement in load balancing of the data center after the task migration is completed, and to dynamically adjust the parameters during the task migration process based on the evaluation results.

[0014] Compared with existing technologies, the data center task migration and scheduling method and system of this invention have the following advantages: This invention, through real-time collection and feature analysis of node task load index data, can accurately perceive the distribution status of cluster resources, providing a high-fidelity data foundation for subsequent decision-making. This invention quantifies multidimensional load characteristics into calculable task load values ​​and constructs a load balance evaluation index based on these values, thereby achieving an objective and quantitative assessment of cluster health status. This invention constructs a task migration model with the goal of optimal load distribution balance. It can generate the optimal migration scheme under the premise of comprehensively considering node capacity, task requirements and migration costs, thereby achieving global optimization of cluster load while minimizing migration overhead. The present invention forms a complete feedback loop through the effect evaluation and dynamic parameter adjustment mechanism after migration execution. It can continuously optimize its own decision-making logic based on the actual operation effect, which not only ensures the effectiveness of each migration, but also gives the system the ability to self-evolve. This invention can effectively eliminate load hotspots and idle nodes in the cluster, significantly reduce task response latency, improve system throughput, ensure load balance among nodes, avoid local overload that reduces operational reliability and stability, and achieve an overall improvement in resource utilization while ensuring service quality. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the process structure of the data center task migration and scheduling method in an embodiment of the present invention; Figure 2 This is a schematic diagram of the composition of the data center task migration and scheduling system in an embodiment of the present invention. Detailed Implementation

[0016] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0017] like Figure 1 As shown in the embodiments of this application, a data center task migration and scheduling method is provided, including: S100: acquiring task load index data of each node in the data center and determining the load index characteristics of the task load index data; S200: calculating the task load of each node based on the load index characteristics to obtain the task load value, and determining the task load balance of the data center based on the task load value of each node; S300: determining whether a task needs to be migrated based on the task load balance, and after determining that migration is required, constructing a task migration model with the goal of optimal load distribution balance in the data center; S400: generating a task migration plan based on the task migration model, and migrating the tasks of each node in the data center based on the task migration plan; S500: after the task migration is completed, evaluating the degree of improvement in the load balance of the data center, and dynamically adjusting the parameters in the task migration process based on the evaluation results.

[0018] Furthermore, this invention, through real-time collection and feature analysis of node task load index data, can accurately perceive the distribution status of cluster resources, providing a high-fidelity data foundation for subsequent decision-making. This invention quantifies multi-dimensional load characteristics into calculable task load values ​​and constructs a load balancing evaluation index based on these values, achieving an objective and quantitative assessment of cluster health. This invention constructs a task migration model with the goal of optimal load distribution balance, generating the optimal migration plan while comprehensively considering node capacity, task requirements, and migration costs, thereby minimizing migration overhead while achieving global optimization of cluster load configuration. The effect evaluation and dynamic parameter adjustment mechanism after migration execution forms a complete feedback loop, continuously optimizing its decision-making logic based on actual operating results, ensuring the effectiveness of each migration and endowing the system with self-evolution capabilities. This invention effectively eliminates load hotspots and idle nodes in the cluster, significantly reduces task response latency, improves system throughput, ensures load balancing across nodes, avoids local overload that reduces operational reliability and stability, and achieves an overall improvement in resource utilization while ensuring service quality.

[0019] In the embodiments of this application, a data center task migration and scheduling method is provided. The step of obtaining task load index data of each node in the data center and determining the load index characteristics of the task load index data includes: obtaining task load index data of each node in the data center, wherein the load indexes include CPU utilization data, memory utilization data and network utilization data; calculating the mean, peak value and variance of the task load index data, and determining the mean, peak value and variance value as the load index characteristics of the task load index data.

[0020] Specifically, by collecting multi-dimensional resource load index data of each node in the data center in real time, including CPU utilization, memory utilization and network utilization, a raw data foundation that comprehensively reflects the node's operating status is constructed; statistical analysis is performed on the load data of each dimension to calculate its mean, peak value and variance, and these three statistics are used as the load index characteristics of the node. In this step, the mean reflects the average load level of the nodes, providing a benchmark for judging whether the nodes are in a state of high or low load for a long time. The peak value reveals the upper limit of the node's load fluctuation, which can effectively identify the risk of sudden traffic or instantaneous overload and avoid ignoring the performance jitter caused by peak impacts due to normal average load. The variance characterizes the stability of the load. Excessive variance means that the load fluctuates drastically, which can easily lead to resource contention and performance instability. Insufficient variance may indicate insufficient resource utilization. By refining the original time-series load data into three statistical features with clear physical meaning—mean, peak value, and variance—effective compression and feature extraction from massive monitoring data to key status information are achieved. This provides high-quality data support for subsequent accurate calculation of node load values, objective evaluation of cluster load balance, and precise triggering of migration decisions, thus laying the foundation for the reliable operation of the entire intelligent scheduling system.

[0021] In the embodiments of this application, a data center task migration and scheduling method is provided. The step of calculating the task load of each node based on load index characteristics to obtain the task load value includes: determining the preset weight of each load index characteristic, and weighting and adding each load index characteristic with the preset weight to obtain the sub-task load value of each task load index data; determining the task load index data with the largest sub-task load value, and determining the sub-task load value of the largest task load index data as the task load value of each node.

[0022] Specifically, for each type of resource indicator, its mean, peak value, and variance are multiplied by preset weighting coefficients and summed to obtain the subtask load value of that resource indicator. This weighted calculation process can be differentiated according to the contribution of each statistical feature to the node pressure under different business scenarios. The maximum value among the three subtask load values ​​of CPU, memory, and network is selected as the final task load value of the node. This step compresses multidimensional statistical features into a single quantitative indicator through weighted summation, which not only retains the rich information of the original data but also achieves horizontal comparability of the load. By taking the maximum value rather than the average value, it deeply aligns with the engineering concept of the "barrel principle," where the true load of a node is determined by its most strained resource dimension. This avoids the risk of overload of bottleneck resources being masked by the idleness of other resource dimensions, enabling more sensitive and accurate identification of potential hotspots. This provides high-quality data input for subsequent load balancing decisions, ensuring that migration operations truly affect the nodes that urgently need intervention, and effectively improving the overall stability and resource utilization efficiency of the cluster.

[0023] In the embodiments of this application, a data center task migration and scheduling method is provided. The step of determining the task load balance of the data center based on the task load value of each node includes: determining the highest load node and the lowest load node based on the task load value of each node, and calculating the load ratio between the highest load node and the lowest load node; and determining the calculated ratio as the task load balance of the data center.

[0024] Specifically, based on obtaining the task load values ​​of each node, the highest-loaded and lowest-loaded nodes in the cluster are identified, and the load ratio between them is calculated. This ratio is directly used as the core indicator for measuring the overall task load balance of the data center. This step achieves an intuitive quantification and concise expression of the load balance status. The ratio of the highest to lowest-loaded nodes, in a single numerical form, highly condenses the degree of extreme differences in the distribution of cluster resources. The closer the ratio is to 1, the smaller the gap between the highest and lowest load points in the cluster, and the more balanced the overall distribution. The larger the ratio, the more severe the unevenness of work and idle periods, with significant hot spots and idle points. The highest-loaded node represents the most risky bottleneck in the cluster, and its load level is directly related to the degree of service level agreement (SLA) satisfaction. The lowest-loaded node represents a low point of resource waste, and its existence indicates the potential for resource integration and energy-saving optimization. Directly comparing the two allows for the immediate detection of the most dangerous overload signals and the most urgent integration needs, providing a clear, direct, and highly operational basis for triggering migration decisions.

[0025] In an embodiment of this application, a data center task migration and scheduling method is provided. The step of determining whether a task needs to be migrated based on task load balancing includes: determining a pre-set task load balancing threshold, and determining whether a task needs to be migrated based on the relationship between the task load balancing of the data center and the task load balancing threshold; when the task load balancing of the data center is less than the task load balancing threshold, it is determined that the tasks of each node in the data center do not need to be migrated; when the task load balancing of the data center is greater than or equal to the task load balancing threshold, it is determined that the tasks of each node in the data center need to be migrated.

[0026] Specifically, a task load balancing threshold is set as a control valve to determine whether the cluster needs to perform a migration operation. The real-time calculated data center task load balancing is compared with this threshold. If the load balancing is less than the threshold, it indicates that the current load distribution of the cluster is still within an acceptable range, and the difference between the highest and lowest points has not triggered the warning line. Therefore, it is determined that no migration operation needs to be performed. If the load balancing is greater than or equal to the threshold, it means that the cluster has a significant uneven state of busy and idle, and the difference between the load hotspots and idle points exceeds the tolerance range. Then it is determined that the migration process needs to be started. This step transforms the abstract concept of load balancing into a clear and actionable judgment logic by introducing preset threshold parameters. It finds the optimal balance between being too sensitive, leading to frequent migrations, and being too sluggish, causing performance risks. Without a threshold as a buffer, any small load fluctuation could trigger a migration operation, causing tasks to frequently travel between nodes, consuming valuable network bandwidth and CPU resources, and potentially affecting service quality due to migration jitter. By setting a reasonable threshold, a certain range of load fluctuations can be tolerated, and intervention is only carried out when the imbalance reaches a significant level, thus ensuring the necessity and high value of the migration operation. When the balance exceeds the threshold, it means that there are hot nodes in the cluster that are close to overload. If not intervened in time, it may cause a performance avalanche. At this time, the migration process is immediately triggered to distribute the pressure from high-risk nodes to low-load nodes, realizing proactive intervention and early resolution of potential performance risks, effectively ensuring the satisfaction of service level agreements, and providing a reliable guarantee for the stable operation of the data center.

[0027] In the embodiments of this application, a data center task migration and scheduling method is provided. After determining that migration is needed, a task migration model is constructed with the goal of optimizing the load distribution balance of the data center. The method includes: after determining that migration is needed, constructing an objective function with the goal of minimizing the variance of the task load values ​​of all nodes in the data center after migration; and constructing a task migration model based on the objective function and constraints, using node resource capacity, task resource requirements, and migration cost as constraints.

[0028] Specifically, after determining that task migration is necessary, this method constructs an objective function with the core objective of minimizing the variance of task load values ​​of all nodes in the data center after migration. At the same time, node resource capacity, task resource requirements, and migration costs are used as key constraints. Based on the above objective function and constraints, a complete task migration model is constructed. This step mathematically and optimizablely expresses the load balancing objective. Variance, as a statistic measuring the dispersion of data distribution, can accurately characterize the deviation between the load value of each node in the cluster and the average load. The smaller the variance, the closer the load of all nodes is to the average value, and the more balanced the overall distribution of the cluster. Using minimizing the load variance after migration as the objective function is equivalent to transforming the abstract concept of load balancing into a quantitative indicator with a clear mathematical definition, computability, and optimizability, providing a clear optimization direction for subsequent solutions to the optimal migration scheme. Node resource capacity constraints ensure that the generated migration plan will not overload the target nodes, avoiding ineffective balancing. Task resource requirement constraints ensure that each task is assigned to a node with sufficient resources to support its operation, ensuring that the optimization process is always within the limits allowed by physical reality. Migration cost constraints incorporate the overhead of the migration operation itself into the decision-making consideration, avoiding paying too high a practical cost in pursuit of theoretical absolute balance. By setting an adjustable migration cost budget, the optimal balance point between load balancing benefits and migration execution costs can be sought, ensuring that every migration is a high-value operation with positive benefits.

[0029] In embodiments of this application, a data center task migration and scheduling method is provided. The step of generating a task migration plan based on a task migration model and migrating tasks of each node in the data center based on the task migration plan includes: solving the task migration model based on a genetic algorithm to obtain the optimal solution for task migration, and generating a task migration plan based on the optimal solution; sorting each node in the data center according to the task load value from high to low according to the task migration plan to obtain a sequence of nodes to be migrated; determining nodes with sufficient remaining resource capacity to accommodate the tasks to be migrated as candidate nodes, and sorting the candidate nodes according to the task load value from low to high to obtain a sequence of target migration nodes; and performing task migration based on the sequence of nodes to be migrated and the sequence of target migration nodes in the order from front to back to migrate tasks of each node in the data center.

[0030] Specifically, a genetic algorithm is used to solve the migration model with the goal of minimizing load variance. By simulating selection, crossover, and mutation operations in the natural evolution process, a globally approximate optimal solution is efficiently searched in the complex solution space, generating a migration plan that includes specific migration tasks and target nodes. Based on this, to achieve efficient execution of the migration plan, all nodes in the data center are sorted from high to low according to their current task load values ​​to form a sequence of nodes to be migrated (i.e., the nodes with the highest load are processed first). At the same time, nodes with sufficient remaining resource capacity to accommodate the tasks to be migrated are selected and sorted from low to high according to their load values ​​to form a sequence of target migration nodes (i.e., the nodes with the least available load are selected first). Tasks are paired and migrated sequentially according to the order of the two sequences from front to back, and the tasks on the most urgent overloaded nodes in the sequence of nodes to be migrated are migrated to the least available nodes in the sequence of target nodes. In this step, the genetic algorithm ensures the global optimization of the migration scheme, escaping the trap of local optima and finding the best combination that balances load balancing and migration cost. The load-sorting-based dual-sequence migration mechanism achieves fine-grained scheduling, prioritizing urgent tasks and filling empty areas before full ones, quickly reducing the hottest spots and filling the lowest-lying areas, maximizing the efficiency of each migration operation in improving cluster balance. The combination of these two approaches not only significantly reduces the cluster load variance after migration but also avoids migration conflicts and resource contention through an orderly and efficient execution process. This allows the entire cluster to converge to a load-balanced state with minimal cost and maximum speed, thereby effectively guaranteeing the service quality and resource utilization efficiency of the data center.

[0031] In an embodiment of this application, a data center task migration and scheduling method is provided. The step of evaluating the degree of improvement in the load balancing of the data center after the task migration is completed includes: calculating the task load balancing degree of the data center after the migration, and calculating the difference between the task load balancing degree after the migration and the task load balancing degree before the migration to obtain the load balancing degree difference; calculating the ratio of the load balancing degree difference to the task load balancing degree before the migration, and evaluating the ratio to obtain the value of the degree of improvement in the load balancing of the data center.

[0032] Specifically, after the task migration is completed, load data for each node is recollected, and the post-migration task load balance is calculated. Then, the difference between this value and the pre-migration task load balance is calculated, and this difference is compared with the pre-migration balance to obtain a load balance improvement value reflecting the extent of improvement. This step quantifies the abstract load balance improvement effect into a specific numerical indicator, enabling an objective and accurate measurement of the actual effectiveness of each migration operation. Positive values ​​indicate improved load balance, with larger values ​​indicating more significant improvement, while negative values ​​suggest a deterioration in load balance and require attention.

[0033] In embodiments of this application, a data center task migration and scheduling method is provided. The method involves dynamically adjusting parameters during the task migration process based on evaluation results. This includes: determining a pre-set improvement threshold, and determining whether dynamic adjustment of parameters during the task migration process is necessary based on the relationship between the data center's load balancing improvement value and the improvement threshold. If the data center's load balancing improvement value is greater than or equal to the improvement threshold, no dynamic adjustment of parameters during the task migration process is required. If the data center's load balancing improvement value is less than the improvement threshold, dynamic adjustment of parameters during the task migration process is necessary. If dynamic adjustment of parameters during the task migration process is required, the changes in the data center's task load balancing before and after each migration are recorded to form a historical sample set. A reinforcement learning algorithm is then used to dynamically adjust the threshold parameters involved in the task migration process.

[0034] Specifically, after the migration is completed, the migration effect is quantitatively evaluated by introducing a preset improvement threshold: the calculated load balancing improvement value is compared with the threshold. If the improvement value reaches or exceeds the threshold, it means that the migration operation has effectively achieved the expected balancing goal, and the parameters in the current migration process remain unchanged; if the improvement value is lower than the threshold, it means that the migration effect has not met expectations, and it is determined that key parameters in the migration process (such as the load balancing threshold that triggers the migration, the weight coefficient of load indicator features, etc.) need to be dynamically adjusted. When parameter tuning is required, the load balancing change data of the data center before and after each migration is automatically recorded to form a historical sample set containing rich experience information. Based on this sample set, a reinforcement learning algorithm is used to iteratively optimize the threshold parameters. The reinforcement learning agent uses the historical migration effect as a reward signal and gradually learns the best parameter configuration that can adapt to the current load mode through continuous trial and error and policy updates. This step upgrades the one-time static configuration to a continuously evolving dynamic optimization process, which can learn from each successful or failed migration and automatically correct decision boundaries. This not only effectively avoids repeated invalid migrations caused by improper parameter settings, but also gives the system the ability to evolve with the load pattern, thereby continuously approaching the optimal load balancing strategy in long-term operation and significantly improving the intelligent operation and maintenance level and long-term stability of the data center.

[0035] like Figure 2As shown in the embodiments of this application, a data center task migration and scheduling system is provided, comprising: an acquisition module, used to acquire task load index data of each node in the data center and determine the load index characteristics of the task load index data; a calculation module, used to calculate the task load of each node based on the load index characteristics, obtain the task load value, and determine the task load balance of the data center based on the task load value of each node; a modeling module, used to determine whether a task needs to be migrated based on the task load balance, and after determining that migration is required, construct a task migration model with the goal of optimal load distribution balance in the data center; a migration module, used to generate a task migration plan based on the task migration model, and migrate the tasks of each node in the data center based on the task migration plan; and an adjustment module, used to evaluate the degree of improvement in the load balance of the data center after the task migration is completed, and dynamically adjust the parameters during the task migration process based on the evaluation results.

[0036] In summary, this invention provides a data center task migration and scheduling method and system, comprising: acquiring task load index data of each node in the data center and determining its load index characteristics; calculating the task load value of each node based on the load index characteristics, and determining the task load balance of the data center based on the load value; determining whether a task needs to be migrated based on the task load balance, and constructing a task migration model with optimal load distribution balance after determining that migration is necessary; generating a task migration plan based on the task migration model, and performing task migration based on the plan; evaluating the degree of improvement in the load balance of the data center after task migration, and dynamically adjusting the parameters during the task migration process based on the results. This invention can effectively eliminate load hotspots and idle nodes in the cluster, significantly reduce task response latency, improve system throughput, ensure load balance of each node, and avoid local overload that reduces operational reliability and stability.

[0037] Finally, it should be noted that those skilled in the art can obviously make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A data center task migration and scheduling method, characterized in that, include: Obtain task load metric data for each node in the data center and determine the load metric characteristics of the task load metric data. The task load of each node is calculated based on the load index characteristics to obtain the task load value, and the task load balance of the data center is determined based on the task load value of each node. The task migration model is constructed based on the task load balancing degree to determine whether the task needs to be migrated. After determining that the task needs to be migrated, the task migration model is constructed with the goal of optimal load distribution balance in the data center. A task migration plan is generated based on the task migration model, and tasks on each node in the data center are migrated based on the task migration plan. After the task migration is completed, the degree of improvement in the load balancing of the data center is evaluated, and the parameters during the task migration process are dynamically adjusted based on the evaluation results.

2. The data center task migration and scheduling method according to claim 1, characterized in that, The process of acquiring task load metric data for each node in the data center and determining the load metric characteristics of the task load metric data includes: Obtain task load metrics data for each node in the data center, including CPU utilization data, memory utilization data, and network utilization data; Calculate the mean, peak value, and variance of the task load index data, and determine the mean, peak value, and variance as the load index characteristics of the task load index data.

3. The data center task migration and scheduling method according to claim 2, characterized in that, The calculation of task load for each node based on load metric characteristics to obtain task load values ​​includes: Determine the preset weights for each load indicator feature, and then calculate the subtask load value of each task load indicator data by weighted summation of each load indicator feature with the preset weights. Determine the task load metric data with the largest task load value among the subtask load values, and then determine the task load value of each node as the subtask load value with the largest task load metric data.

4. The data center task migration and scheduling method according to claim 1, characterized in that, The process of determining the task load balance of the data center based on the task load values ​​of each node includes: The highest and lowest load nodes are determined based on the task load values ​​of each node, and the load ratio between the highest and lowest load nodes is calculated. The calculated ratio is determined as the task load balancing degree of the data center.

5. The data center task migration and scheduling method according to claim 1, characterized in that, The method of determining whether a task needs to be migrated based on task load balancing includes: Determine a pre-set task load balancing threshold, and determine whether a task needs to be migrated based on the relationship between the data center's task load balancing and the task load balancing threshold; If the task load balancing degree of the data center is less than the task load balancing threshold, it is determined that the tasks on each node in the data center do not need to be migrated. If the task load balancing degree of the data center is greater than or equal to the task load balancing degree threshold, it is determined that the tasks of each node in the data center need to be migrated.

6. A data center task migration and scheduling method according to claim 5, characterized in that, After determining that migration is necessary, the process of constructing a task migration model with the goal of optimal data center load distribution balance includes: After determining that migration is necessary, an objective function is constructed with the goal of minimizing the variance of the task load values ​​of all nodes in the data center after migration. The task migration model is constructed based on the objective function and constraints, with node resource capacity, task resource requirements and migration cost as constraints.

7. A data center task migration and scheduling method according to claim 1, characterized in that, The process of generating a task migration plan based on the task migration model and migrating tasks on each node in the data center based on the task migration plan includes: The task migration model is solved using a genetic algorithm to obtain the optimal solution for task migration, and a task migration plan is generated based on the optimal solution. The task migration plan sorts the nodes in the data center from high to low according to their task load values, resulting in a sequence of nodes to be migrated. Nodes with sufficient remaining resource capacity to accommodate the tasks to be migrated are identified as candidate nodes, and the candidate nodes are sorted from low to high according to their task load values ​​to obtain the target migration node sequence. Based on the sequence of nodes to be migrated and the sequence of target nodes to be migrated, the tasks are migrated in the order from front to back to migrate the tasks of each node in the data center.

8. A data center task migration and scheduling method according to claim 1, characterized in that, After the task migration is completed, the degree of improvement in load balancing of the data center is evaluated, including: After the task migration is completed, calculate the task load balancing degree of the data center after migration, and calculate the difference between the task load balancing degree after migration and the task load balancing degree before migration to obtain the load balancing degree difference. Calculate the ratio of the load balance difference to the load balance before migration, and evaluate this ratio to obtain the degree of improvement in the load balance of the data center.

9. A data center task migration and scheduling method according to claim 8, characterized in that, The dynamic adjustment of parameters during the task migration process based on the evaluation results includes: Determine a pre-set threshold for improvement level, and determine whether dynamic adjustment of parameters is needed during the task migration process based on the relationship between the data center load balancing improvement level value and the improvement level threshold; If the load balancing improvement value of the data center is greater than or equal to the improvement threshold, then there is no need to dynamically adjust the parameters during the task migration process. If the improvement value of the load balancing in the data center is less than the improvement threshold, the parameters during the task migration process need to be dynamically adjusted. If it is necessary to dynamically adjust the parameters during the task migration process, record the changes in the task load balancing of the data center before and after each migration to form a historical sample set, and use reinforcement learning algorithms to dynamically adjust the threshold parameters involved in the task migration process.

10. A data center task migration and scheduling system, characterized in that, include: The acquisition module is used to acquire task load index data of each node in the data center and determine the load index characteristics of the task load index data. The calculation module is used to calculate the task load of each node based on the load index characteristics, obtain the task load value, and determine the task load balance of the data center based on the task load value of each node. The modeling module is used to determine whether a task needs to be migrated based on the task load balance, and after determining that migration is necessary, to build a task migration model with the goal of optimal load distribution balance in the data center. The migration module is used to generate a task migration plan based on the task migration model, and to migrate tasks on each node in the data center based on the task migration plan. The adjustment module is used to evaluate the improvement in load balancing of the data center after the task migration is completed, and to dynamically adjust the parameters during the task migration process based on the evaluation results.

Images