Related task scheduling method based on delay sensitivity in cross-platform cloud environment
By using the improved random forest model and Markov model in a cross-platform cloud environment combined with the XGBoost algorithm, the problems of poor performance and insufficient consideration of latency factors of traditional models under high-dimensional large sample sets are solved, achieving more efficient Task scheduling and resource allocation.
Patent Information
- Application Number
- CN202411321657.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2025-02-07
AI Technical Summary
In a cross-platform cloud environment, traditional random forest models perform poorly under high-dimensional large sample sets, resulting in weak overfitting and generalization capabilities, and the inability to effectively consider the communication delay factors between platforms, affecting the efficiency of task scheduling.
The improved random forest model based on incremental learning is used to classify tasks, and combined with the Markov model and XGBoost algorithm, the task queue is scheduled comprehensively considering the communication delay factors and task correlation between platforms.
It improves the performance and efficiency of task scheduling, can predict the scheduling time of tasks more accurately, reduce task delay, improve task execution efficiency, and allocate cloud resources more intelligently.
Smart Images
Figure CN119396546A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of cloud computing, delay sensitivity, associated task scheduling, virtualization technology, and in particular to a method for scheduling associated tasks based on delay sensitivity in a cross-platform cloud environment. Background Art
[0002] The mission performance of the avionics system is affected by many factors, such as the mission scale, remaining resources of the hardware platform, constraints on related tasks, platform network communication delay, intermediate node processing capabilities, etc. In addition, due to the dynamic characteristics of related tasks, it is difficult to unify the analysis of system performance. It is necessary to analyze different scenarios and different constraint requirements. Therefore, it is difficult to describe the mission characteristics and simulate the estimated execution time of the calculation task.
[0003] Random forest is a variant of the decision tree algorithm, which combines the idea of ensemble learning and refers to a classifier that uses multiple decision trees to train and predict samples. Ishwaran et al. have previously verified the consistency and convergence of random forests, and believe that random forests are significantly superior to other classification methods in terms of sample imbalance. Problems such as sample category imbalance in high-dimensional vital sign data can be effectively solved under the random forest model, so the avionics system task is suitable for solving the random forest model.
[0004] However, traditional random forest models often perform poorly in high-dimensional and large sample sets. This is due to:
[0005] 1. High-dimensional data is often mixed with irrelevant and redundant variables, which can easily lead to overfitting problems and weaken the generalization ability of the model.
[0006] 2. The traditional random forest model performs sampling with replacement in the entire sample set. This sampling method often cannot cover the entire sample set under the condition of large sample set data, resulting in poor accuracy.
[0007] 3. In the task scheduling problem in a cross-platform cloud environment, it is necessary to consider the impact of inter-platform communication delay factors and add stronger constraints to the scheduling of delay-sensitive tasks. Summary of the invention
[0008] The purpose of the invention is to provide a method for scheduling delay-sensitive and associated tasks in a cross-platform cloud environment. The method uses an improved random forest model based on incremental learning to classify tasks, and then puts tasks into different task queues according to their types, and then schedules the tasks in the queues by comprehensively considering the communication delay factors between platforms and the task association. The method takes into account the characteristics of delay-sensitive and associated tasks, guides task scheduling with machine learning algorithms, and exhibits better performance and more reasonable scheduling.
[0009] The object of the present invention is achieved through the following technical solutions:
[0010] A method for scheduling latency-sensitive and correlated tasks in a cross-platform cloud environment includes the following steps:
[0011] Step 1: Classify the tasks on each cloud platform;
[0012] Step 2: After placing the tasks of each cloud platform into different task queues according to their types, the associated tasks are constructed into a Markov model, the communication delay factors between platforms are comprehensively considered, and the tasks in the queue are scheduled according to time-sensitive constraints.
[0013] Preferably, in step 1, an improved random forest model based on incremental learning is used to classify the tasks, and the process is as follows:
[0014] Step 11) Continuously collect task data from each cloud platform;
[0015] Step 12) extracting task features from the collected tasks, including attributes, performance indicators, and environmental data;
[0016] Step 13) Mark according to task completion time and resource utilization and convert into sample data;
[0017] Step 14) Build a random forest model:
[0018] A) Initialize an empty random forest model;
[0019] B) converting the newly arrived task data into sample data according to the above steps 11)-13);
[0020] C) For each tree, select a random subset of new features for training;
[0021] D) Use the stochastic gradient descent algorithm to update the tree structure and parameters for new samples;
[0022] E) Use a portion of the original sample data as a test set to evaluate the performance of the model. If the current performance is lower than the performance in the previous step, return to step C) and reselect features for training;
[0023] F) Merge the decisions of these trees to arrive at the final task classification;
[0024] G) When new task data arrives, repeat the above steps to continuously iterate the model update and performance evaluation.
[0025] Step 15) Use the trained model to classify the newly arrived tasks.
[0026] Preferably, the process of step 2 is as follows:
[0027] Step 21) Pre-training the XBoost model using historical data;
[0028] Step 22) Read all the task sets C to be processed in the current task queue K, read all the tasks C' that have dependency relationships with the tasks in C, and calculate the correlation scores between the tasks in C' and the tasks in C according to the strength and number of the relationships, and save them in the vector v;
[0029] Step 23) Use the XBoost model to predict the scheduling time of the tasks in set C and save it in vector t;
[0030] Step 24) Select the next task to be processed that is not associated with C from each queue and set it as set E, read all tasks E' that have dependency relationships with tasks in E, and calculate the correlation score between tasks in E' and tasks in E according to the strength and number of relationships, and save it in vector v';
[0031] Step 25) Use the XBoost model to predict the scheduling time of the tasks in set E and save it in vector t';
[0032] Step 26) If the result of v'*t' is less than the result of v*t, update the current strategy to schedule the tasks in E, otherwise keep the current strategy to schedule the tasks in C and return to step 24) until N tasks in the system have been selected; where N is a value dynamically selected based on specific needs in actual production.
[0033] The beneficial effects of the present invention are:
[0034] 1. Task categories may be unevenly distributed in a cloud environment, and some task types may be less numerous. The present invention can adjust the learning weights of the model according to the correlation measure of the tasks to better handle the unbalanced task distribution.
[0035] 2. By considering the correlation between tasks, the present invention can more accurately predict the scheduling time of tasks, reduce task delays, and improve task execution efficiency.
[0036] 3. The present invention can more intelligently allocate resources in the cloud environment to meet the resource requirements of associated tasks, which helps to fully utilize cloud resources and reduce resource waste.
[0037] 4. The present invention can adapt to the changing requirements of tasks in real time, dynamically adjust task scheduling according to task relevance and resource conditions, and provide a faster response time. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 The figure is a flowchart of a delay-sensitive and associated task scheduling method in a cross-platform cloud environment. DETAILED DESCRIPTION
[0039] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.
[0040] A method for scheduling latency-sensitive and associated tasks in a cross-platform cloud environment includes the following steps:
[0041] Step 1: Classify the tasks on each cloud platform.
[0042] In this embodiment, an improved random forest model based on incremental learning is used to classify tasks. The training process of the random forest model is as follows:
[0043] Step 11) Continuously collect task data from each cloud platform. The task data is an XML file used in the cloud platform.
[0044] Step 12) Extracting features such as task attributes, performance indicators, and environmental data from the collected tasks.
[0045] Step 13) Mark the task according to its completion time and resource utilization rate and convert it into sample data.
[0046] Step 14) Build a random forest model.
[0047] A) Initialize an empty random forest model;
[0048] B) converting the newly arrived task data into sample data according to the above steps 11)-13);
[0049] C) For each tree, select a random subset of new features for training;
[0050] D) Use the stochastic gradient descent algorithm to update the tree structure and parameters for new samples;
[0051] E) Use a portion of the original sample data as a test set to evaluate the performance of the model. If the current performance is lower than the previous step, return to step C) and reselect features for training;
[0052] F) Merge the decisions of these trees to arrive at the final task classification;
[0053] G) When new data arrives, repeat the above steps to continuously iterate the model update and performance evaluation.
[0054] Step 15) Use the trained model to classify the newly arrived tasks.
[0055] Step 2: After placing the tasks of each cloud platform into different task queues according to their types, the associated tasks are constructed into a Markov model, the communication delay factors between platforms are comprehensively considered, and the tasks in the queue are scheduled according to time-sensitive constraints.
[0056] Since the tasks in the cross-platform cloud environment are highly sensitive to latency and highly correlated, there is a certain execution order between tasks and the execution time interval needs to be constrained within a certain range. Therefore, the task scheduling model in this scenario is abstracted into a continuous-time Markov model, which comprehensively considers the communication delay factors between platforms and uses the XGBoost algorithm combined with the policy iteration algorithm to predict the scheduling time of the task, thereby obtaining the optimal scheduling execution flow. The algorithm is described as follows:
[0057] Step 21) Pre-train the XBoost model using historical data.
[0058] Step 22) Read all the task sets C to be processed in the current task queue K, read all the tasks C' that have dependencies with the tasks in C, and calculate the correlation scores between the tasks in C' and the tasks in C based on the strength and number of the relationships, and save them in the vector v.
[0059] Step 23) Use the XBoost model to predict the scheduling time of the tasks in set C and save it in vector t.
[0060] Step 24) Select the next task to be processed that is not associated with C from each queue and set it as set E, read all tasks E' that have dependency relationships with tasks in E, and calculate the correlation score between tasks in E' and tasks in E based on the strength and number of relationships, and save it in vector v'.
[0061] Step 25) Use the XBoost model to predict the scheduling time of the tasks in set E and save it in vector t'.
[0062] Step 26) If the result of v'*t' is less than the result of v*t, update the current strategy to schedule the tasks in E, otherwise keep the current strategy to schedule the tasks in C. Return to step 24) until N tasks in the system have been selected. N is a value dynamically selected according to the specific needs in actual production.
[0063] It is understandable that those skilled in the art can make equivalent substitutions or changes based on the technical solution and inventive concept of the present invention, and all these changes or substitutions should fall within the protection scope of the claims attached to the present invention.
Claims
1. A delay-sensitive correlation task scheduling method in a cross-platform cloud environment, characterized in that The following steps are involved: Step 1: Classify the tasks on each cloud platform; Step 2: After placing the tasks of each cloud platform into different task queues according to their types, the associated tasks are constructed into a Markov model, the communication delay factors between platforms are comprehensively considered, and the tasks in the queue are scheduled according to time-sensitive constraints.
2. According to claim 1, a delay-sensitive correlation task scheduling method in a cross-platform cloud environment is characterized in that In step 1, the improved random forest model based on incremental learning is used to classify the tasks. The process is as follows: Step 11) Continuously collect task data from each cloud platform; Step 12) extracting task features from the collected tasks, including attributes, performance indicators, and environmental data; Step 13) Mark according to task completion time and resource utilization and convert into sample data; Step 14) Build a random forest model: A) Initialize an empty random forest model; B) converting the newly arrived task data into sample data according to the above steps 11)-13); C) For each tree, select a random subset of new features for training; D) Use the stochastic gradient descent algorithm to update the tree structure and parameters for new samples; E) Use a portion of the original sample data as a test set to evaluate the performance of the model. If the current performance is lower than the performance in the previous step, return to step C) and reselect features for training; F) Merge the decisions of these trees to arrive at the final task classification; G) When new task data arrives, repeat the above steps to continuously iterate the model update and performance evaluation. Step 15) Use the trained model to classify the newly arrived tasks.
3. According to claim 1, a delay-sensitive correlation task scheduling method in a cross-platform cloud environment is characterized in that The process of step 2 is as follows: Step 21) Pre-training the XBoost model using historical data; Step 22) Read all the task sets C to be processed in the current task queue K, read all the tasks C' that have dependency relationships with the tasks in C, and calculate the correlation scores between the tasks in C' and the tasks in C according to the strength and number of the relationships, and save them in the vector v; Step 23) Use the XBoost model to predict the scheduling time of the tasks in set C and save it in vector t; Step 24) Select the next task to be processed that is not associated with C from each queue and set it as set E, read all tasks E' that have dependency relationships with tasks in E, and calculate the correlation score between tasks in E' and tasks in E according to the strength and number of relationships, and save it in vector v'; Step 25) Use the XBoost model to predict the scheduling time of the tasks in set E and save it in vector t'; Step 26) If the result of v'*t' is less than the result of v*t, update the current strategy to schedule the tasks in E, otherwise keep the current strategy to schedule the tasks in C and return to step 24) until N tasks in the system have been selected; where N is a value dynamically selected based on specific needs in actual production.