Parallel task scheduling method based on node dynamic load and task correlation metric

Abstract

The application discloses a parallel task scheduling method based on node dynamic load and task correlation measurement, constructs a global scheduler of calculation resource virtualization and sample distribution based on reinforcement learning and a local scheduler based on hybrid driving model scheduling on a server; in a preprocessing stage: statistics calculation resource, carries out calculation resource virtualization, GPU virtualization multiplexing technology, GPU acceleration and the like; a global scheduling stage facing test task parallel optimization: carries out sample scheduling feasibility analysis containing fuzzy time, selects calculation node based on EXP3 algorithm, sample scheduling facing node load; a multi-level model parallelization stage based on domain knowledge and data hybrid driving: contains model cohesion coupling degree measurement mechanism, model coupling mode knowledge base, multi-level model parallelization method. The application realizes sample inter-parallelism and model inter-parallelism in samples by comprehensively utilizing the global scheduler and the local scheduler, and has wide application value and use prospect in various fields.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of cloud computing and parallel computing, and particularly relates to a parallel task scheduling method based on node dynamic load and task correlation measurement. BACKGROUND

[0002] To improve the real-time simulation efficiency of multiple scenarios and multiple samples of GPU, it is necessary to study the methods of multiple sample parallel execution and model parallel execution within samples.

[0003] In terms of sample parallelization, the essence is to establish the mapping between sample models and resources, so that more samples can be executed in parallel under resource constraints. Foreign researches more consider the resource allocation of samples with complex constraints. Considering the constraints such as local cold start, tasks are preferentially allocated to virtual machines that already have related service software, which can reduce the preparation time and virtual machine rental cost. Melbourne University in Australia proposes a heuristic algorithm for allocating workflow tasks to heterogeneous services, considering data transmission between tasks and selection of different virtual machine types, but does not consider virtual machine loading time and installation time of related professional software; Valencia Polytechnic University in Spain proposes a method of establishing task execution copies using idle time slots to increase reliability, but this method does not consider the software localization factor. Domestic researches more consider the resource matching problem of independent samples, and the main consideration targets are response speed, data processing efficiency, load prediction, and optimal configuration of virtual machines or servers. National University of Defense Technology proposes a computing power division method with front and back virtual machine layers, realizes parallel load merging based on priority rules, and improves the response time of tasks; Beijing University of Posts and Telecommunications proposes an adaptive particle swarm algorithm to solve the minimum service rental cost minimization problem of independent tasks with a deadline; related researches of Tsinghua University point out that fuzzy prediction and time series analysis are the main means of task load prediction, which can calculate the number and type of servers required by cloud system, and realize the optimal configuration of cloud service resources; when the load increases to the point that the existing resources cannot meet the demand, the number of virtual machines is increased, and the automatic application and release of virtual machines can be predicted through admission control and feedback control.

[0004] For model parallelization, mainstream business modeling platforms at home and abroad all provide their own business parallel modeling and execution methods. They use workflow process definition language to achieve clear and accurate description of serial and parallel models, including model parameter types, output data types, etc., and also describe the calling relationship between models, which helps to assist model disassembly during simulation. For example, the webSphere platform of IBM in the United States is based on SOA, supports the writing and parallel execution of business processes based on application business rules; other domestic and foreign business modeling platforms including BPEL Process Manager of Oracle in the United States, BPM platform of SAP in Germany, Apsis BPM of Kingdee in China, and bpmPaaS platform of Yanhuang in China all support parallel construction and execution of business models. In terms of theoretical research, Chonnam National University in South Korea and Imperial College in the United Kingdom respectively proposed a business model parallelization reconstruction method based on matching pattern mining; Duesseldorf-Essen University in Germany uses deep reinforcement learning method to optimize the business process of business support system and triggers active process adjustment in business process monitoring. In China, Shandong University mines association rules between historical logs to optimize business activity structure remodeling and business process; Southeast University designs a business process reorganization scheme based on deep reinforcement learning (DRL) for adaptive combination of large-scale business models.

[0005] In summary, current research does not consider the simulation efficiency optimization of multi-sample and intra-sample multi-model parallelization. Combining model parallelization and sample parallelization, the multi-level model parallelization method driven by mixed field knowledge and data and the hierarchical load balancing scheduling strategy based on cold start can effectively improve the simulation efficiency of multi-scenario multi-sample. SUMMARY

[0006] The application proposes a parallel task scheduling method based on node dynamic load and task correlation measurement, which comprehensively utilizes global scheduler and local scheduler to realize inter-sample parallelism and intra-sample model parallelism.

[0007] The technical solution for realizing the application is as follows: a parallel task scheduling method based on node dynamic load and task correlation measurement, including the following stages:

[0008] A. Preprocessing stage of hierarchical dynamic scheduling: according to the collected resource environment, the resources include computer servers, storage devices, network devices and other resources; then perform computing resource virtualization, abstract physical computing resources into virtual resources through virtualization technology, thereby improving the utilization rate and flexibility of resources, and dividing physical GPU resources into multiple virtual GPUs, thereby realizing sharing and reuse of GPU resources; finally, utilize the high parallel computing capability of GPU to accelerate scientific computing and data processing tasks, specifically as follows:

[0009] A1. Resource and Environment Data Collection and Recording: Record real-time data on server GPU utilization, memory utilization, and network bandwidth utilization to form a dataset; in addition, it is also necessary to record the utilization status of resources such as computer servers, storage devices, and network devices to ensure a comprehensive understanding of the current resource status.

[0010] A2. Data Analysis and Frequency Distribution Mapping: The dataset from step A1 is sampled and analyzed to obtain frequency distribution maps of GPU utilization, memory utilization, and network bandwidth utilization. Based on the frequency distribution maps and mathematical models, the membership functions of GPU utilization, memory utilization, and network bandwidth utilization for each server are determined, providing data support for further resource scheduling.

[0011] A3. Computing Resource Virtualization: Through virtualization technology, physical computing resources are abstracted into virtual resources, thereby improving resource utilization and flexibility. Specifically, this includes the virtualization of servers, storage devices, and network equipment, ensuring that various resources can operate effectively in a virtualized environment.

[0012] A4. GPU Virtualization Reuse: This involves dividing physical GPU resources into multiple virtual GPUs, thereby enabling the sharing and reuse of GPU resources. Time-slice scheduling or hardware partitioning techniques are used to ensure that each virtual machine can efficiently utilize GPU resources.

[0013] A5. GPU Accelerated Task Configuration: Leveraging the high parallel computing capabilities of GPUs to accelerate scientific computing and data processing tasks. Specifically, this involves decomposing tasks and distributing them across GPUs for execution, fully utilizing the GPU's computing performance, accelerating the processing, and improving overall system efficiency.

[0014] B. Global Scheduling Phase for Parallel Optimization of Test Tasks: Based on collected historical task execution data, a task prediction model based on random forest is trained, and the task resource requirements are determined based on the model's prediction results. Furthermore, an execution time window model based on probability distribution is used to estimate the task execution time. Node load rate is introduced to evaluate the performance of each computing node, and a performance threshold is used to determine whether a node participates in task distribution. Finally, using the multi-armed gambling machine theoretical algorithm framework, a feedback mechanism, reward model, and environmental information are designed to construct the EXP3 algorithm, which selects the optimal node matching the task, as detailed below:

[0015] B1. Integrate the historical task features and the computational resource usage obtained in stage A. Split the data into feature part and label part. The data in feature part includes the number of server instances, the number of worker instances, the size of dataset, the size of batch task. The data in label part includes the usage of gpu core, the usage of gpu memory, the usage of bandwidth. Use Bootstrap self-sampling method to generate the training set for each decision tree.

[0016] B2. Input the data obtained in stage B1 into a random forest model with 20 initialized decision trees. The algorithm of decision tree learning is usually a recursive process of selecting the optimal feature and splitting the training data set according to the feature, so as to have the best classification process for each sub-data set. This process corresponds to the division of feature space and the construction of decision tree. Thus, the resource demand of the task is obtained.

[0017] The random forest resource prediction method comprises the following steps:

[0018] B21. Randomly sample n samples from the sample set with replacement.

[0019] B22. Randomly select k features from all features, and use these features to establish a decision tree for the selected samples.

[0020] B23. Repeat the above two steps 20 times, that is, generate 20 decision trees to form a random forest.

[0021] B24. For new data, make a decision through each tree, and finally vote to confirm which class it belongs to.

[0022] B25. Integrate the results of all decision trees, and convert all results into a task resource demand vector.

[0023] B3. Construct a multi-armed bandit method with an adversarial reward model to analyze the computing node load balancing problem using Bandit theory. Design a feedback mechanism, a reward model and environment information. When considering the adversarial bandit problem with expert advice, use the EXP3 algorithm to solve it.

[0024] The multi-armed bandit method for analyzing the computing node load balancing problem using Bandit theory can be simply described as:

[0025] B31. A scheduling party has a possible action to choose from, and each selection item can be called an arm. Consider that the scheduling process will perform rounds, and in each round, the scheduling party selects an arm to pull, and the pulled arm will feedback a reward value. The ultimate goal is to make the reward value after T rounds as large as possible.

[0026] B32. The cost of any arm a e [K] is r t (a) ³ 0;

[0027] B33. The sample space of the selected arm a is t e [K];

[0028] B34. The cost of the selected arm a is r t (a t );

[0029] B35. The cost of all arms a e [K] is r t (a), a e [K];

[0030] B36. Define the optimal arm as argmin a∈[K] cost(a), then the regret is defined as: The average loss cost expectation of all selected items.

[0031] B4. The EXP3 algorithm first initializes the weight function of each computing node, and randomly selects an arm in the multi-arm gambling machine according to the probability formula, randomly selects a computing node according to the probability distribution, and obtains the cost value of all current computing nodes according to the setting of the full feedback mechanism.

[0032] C. Multi-level model parallelization stage based on mixed driving of domain knowledge and data: This stage is mainly for model parallel local schedulers. The inter-model dependencies within the sample have priority constraints and can be represented by a DAG graph. The constraint relationships between different inter-models within the sample are heterogeneous. First, analyze the priority structure between the models within the sample, that is, the DAG graph, call the model scheduling sequence decision maker to analyze the scheduling order of each model according to the inter-model DAG graph, and sequentially call the reinforcement learning reward function for scoring, while referring to the instance utilization rate of each GPU. Finally, according to the score, it is sequentially distributed to the corresponding GPU for execution, as follows:

[0033] C1. Analyze the priority structure of the models within the sample that have dependency and priority constraints, and access the model scheduling sequence decision maker.

[0034] C2. In the scheduler, the models that need to be prioritized are calculated in the reinforcement learning reward function through the application of the combined and dynamically adjusted resource placement algorithm, and the order of model execution is obtained according to the score size, as follows:

[0035] C21. Initialize the parameters of the application of the combined and dynamically adjusted resource placement algorithm, including the limit number of instances that can run on the server, the difference between the maximum resource amount and the minimum resource amount, and the number of placed models;

[0036] C22. Calculate a complementary model set of the model, obtain all non-idle GPUs on the server, and select the GPU with the largest number of remaining idle instances and capable of meeting the instance demand from the GPUs;

[0037] C23. Perform a loop operation, calculate the target placement point using the vector distance formula corresponding to the placeable GPU core and GPU memory point;

[0038] C24. After confirming the priority execution model, calculate the placement point of the subsequent model according to the score;

[0039] C25. In order to reduce the time consumption caused by model data transmission, the GPU where all the predecessor models are located should be considered first; in this way, the delay of data transmission can be reduced, and the overall system efficiency can be improved; if the GPU resource of the predecessor model is insufficient, the GPU where other models in the same sample are located can be considered to ensure the continuity and efficiency of model execution.

[0040] C3. According to the obtained model execution order, execute the models on the server GPU in sequence.

[0041] Compared with the prior art, the present application has the following advantages: the present application can determine the execution relationship of the models by analyzing the serial and parallel constraint relationship between the simulation models. The parallel optimization problem is modeled by using theoretical methods, the parallel degree of the model is calculated, and the hardware parallel capability is improved. Considering the local resource utilization rate, load balancing, resource serving, hierarchical scheduling, GPU acceleration and other factors, intelligent methods are used for global allocation and scheduling of services and resources, task allocation, learning scheduling knowledge, real-time scheduling samples and the like; considering the model reconstruction, model optimization, model placement and other factors, the application combination and dynamic adjustment of the resource placement algorithm are used for model acceleration. Therefore, the present application comprehensively considers the sample deadline, resource utilization rate and sample and model acceleration, optimizes the total running time of the task in the test running process, thereby reducing the energy consumption of the cluster. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a service and resource scheduling model diagram of the present application.

[0043] Figure 2 is a model parallel diagram in the test task of the present application.

[0044] Figure 3 is a model parallelization implementation process diagram of the present application.

[0045] Figure 4 is a two-layer scheduling system framework diagram of the present application.

[0046] Figure 5 is a local resource scheduling flowchart of the present application.

[0047] Figure 6 Figure 1 is a schematic diagram of a task topology relationship model according to the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort fall within the protection scope of the present application.

[0049] The technical solutions in the embodiments of the present application can be combined with each other, but the combination of the technical solutions should be based on the fact that a person of ordinary skill in the art can realize the combination. When the combination of the technical solutions appears contradictory or unachievable, it should be considered that the combination of the technical solutions does not exist and is not within the protection scope of the present application.

[0050] The specific embodiments, technical difficulties and points of the present application will be further introduced below with reference to the design examples.

[0051] In combination with Figures 1-6 , the parallel task scheduling method based on node dynamic load and task correlation measurement provided by the present application comprehensively utilizes a global scheduler and a local scheduler to realize inter-sample parallelism and inter-model parallelism in a sample, fully utilizes the computing resources and bandwidth resources provided by a high-performance server, and improves the test efficiency. The method includes the following stages:

[0052] A. A preprocessing stage of hierarchical dynamic scheduling: according to the collected resource environment, the resources include computer servers, storage devices, network devices and other resources; then, computing resource virtualization is performed, the physical computing resources are abstracted into virtual resources through virtualization technology, so as to improve the utilization rate and flexibility of the resources, and the physical GPU resources are divided into multiple virtual GPUs, so as to realize the sharing and reuse of the GPU resources; finally, the high parallel computing capability of the GPU is utilized to accelerate scientific computing and data processing tasks.

[0053] The specific steps of the preprocessing stage are as follows:

[0054] A1. Resource environment data collection and recording: real-time data of GPU utilization rate, memory utilization rate and network bandwidth utilization rate of the server are recorded to form a data set; in addition, the utilization of each computer server, storage device, network device and other resources needs to be recorded to ensure that the current resource state is fully mastered.

[0055] A2. Data analysis and frequency distribution chart drawing: sample analysis is performed on the data set in step A1 to obtain the frequency distribution chart of GPU utilization, memory utilization and network bandwidth utilization. According to the frequency distribution chart and mathematical model, the membership function of each server GPU utilization, memory utilization and network bandwidth utilization is determined to provide data support for further resource scheduling, as shown in the following table

[0056]

[0057]

[0058] A3. Virtualization of computing resources: through virtualization technology, physical computing resources are abstracted into virtual resources, thereby improving the utilization and flexibility of resources. Specifically, it includes virtualization operations on servers, storage devices and network devices to ensure that various resources can operate effectively in the virtualization environment.

[0059] A4. GPU virtualization and multiplexing: physical GPU resources are divided into multiple virtual GPUs to realize the sharing and multiplexing of GPU resources. Through time slice scheduling or hardware division technology, it ensures that each virtual machine can efficiently use GPU resources.

[0060] A5. GPU accelerated task configuration: use the high parallel computing capability of GPU to accelerate scientific computing and data processing tasks. Specifically, it includes decomposing and assigning tasks to GPU for execution to fully utilize the computing performance of GPU, speed up the processing process and improve the overall system efficiency.

[0061] B. Global scheduling phase (i.e. solution phase) for test task parallel optimization: according to the random forest prediction model, the required resource amount of the incoming task is obtained. A multi-armed bandit method with an adversarial reward model is constructed to analyze the node load balancing problem using the Bandit theory. Finally, the EXP3 algorithm is used to calculate the node cost value and schedule the task, including the following steps:

[0062] B1. Integrate the historical task features and computing resource usage obtained in stage A. Divide the data into feature part and label part. The data in the feature part includes the number of server instances, the number of worker instances, the size of the data set, and the size of the batch processing task. The data in the label part includes the amount of gpu core, the amount of gpu memory, and the amount of bandwidth. Bootstrap self-sampling method is used to generate the training set of each decision tree.

[0063] B2. The data obtained in B1 is input into a random forest model with 20 decision trees. The algorithm of decision tree learning is usually a recursive process that selects the optimal feature and splits the training data set according to the feature, so that each sub-data set has the best classification process, which corresponds to the division of the feature space and the construction of the decision tree. Thus, the task estimated resource requirement is obtained.

[0064] B21. Randomly sample n samples from the sample set with replacement;

[0065] B22. Randomly select k features from all features, and use these features to build a decision tree for the selected samples;

[0066] B23. Repeat the above two steps 20 times, i.e. generate 20 decision trees to form a random forest;

[0067] B24. For new data, make a decision through each tree, and finally vote to confirm which class it belongs to;

[0068] B25. Integrate the results of all decision trees, and convert all results into a task resource demand vector to obtain the final strategy.

[0069] B3. Construct a multi-armed bandit machine with an adversarial reward model to analyze and calculate node load balancing problems using Bandit theory. Design a feedback mechanism, reward model and environment information. When considering the adversarial bandit problem with expert advice, use the EXP3 algorithm to solve it.

[0070] The multi-armed bandit problem can be simply described as:

[0071] B31. A scheduler has a possible action to choose from, each choice can be called an arm; consider that the scheduling process will perform rounds, and in each round the scheduler selects an arm to pull, and the pulled arm will feedback a reward value, the ultimate goal is to make the reward value after T rounds as large as possible;

[0072] B32. The cost of any arm a∈[K] is r t (a)≥0;

[0073] B33. The sample space of the selected arm is a t ∈[K];

[0074] B34. The cost of the selected arm is r t (a t );

[0075] B35. The cost of all arms is r t (a),a∈[K];

[0076] B36. Define the optimal arm as argmin a∈[K] cost(a), then the regret is defined as: cost(ALG) represents the expected value of the average loss cost of all selected items.

[0077] B4. The EXP3 algorithm first initializes the weight function of each computing node, and randomly selects an arm in the multi-arm gambling machine according to the probability formula, randomly selects a computing node according to the probability distribution, and obtains the cost value of all current computing nodes according to the setting of the full feedback mechanism.

[0078] C. Multi-level model parallelization stage based on hybrid driving of domain knowledge and data: This stage is mainly for model parallel local scheduler. The inter-model dependency within the sample and the priority constraint relationship can be represented by a DAG graph, and the constraint relationship between different inter-models within the sample is heterogeneous. First, analyze the priority structure between the models within the sample, that is, the DAG graph, call the model scheduling sequence decision maker to analyze the scheduling order of each model according to the inter-model DAG graph, and score in sequence according to the reinforcement learning reward function, while referring to the instance utilization rate of each GPU, and finally distribute to the corresponding GPU for execution according to the score.

[0079] The multi-level model parallelization stage based on hybrid driving of domain knowledge and data includes the following steps (as shown in Figure 5 ):

[0080] C1. Analyze the priority structure of the models within the sample with dependency and priority constraint relationship, access the model scheduling sequence decision maker, such as Figure 6 ;

[0081] C2. In the scheduler, the models that need to be prioritized are calculated in the reinforcement learning reward function by applying the combined and dynamically adjusted resource placement algorithm, and the score of the model is calculated. According to the score size, the execution order of the model is obtained.

[0082] C21. Initialize the parameters of the combined and dynamically adjusted resource placement algorithm, including the number of limited instances that can be run on the server, the difference between the maximum resource amount and the minimum resource amount, and the number of placed models;

[0083] C22. Calculate the complementary model set of the model, obtain all non-idle GPUs on the server, and select the GPU with the most remaining idle instances and capable of meeting the instance demand;

[0084] C23. Perform a loop operation, calculate the target placement point using the vector distance formula corresponding to the GPU core and GPU memory point that can be placed;

[0085] C24. After confirming the priority execution model, the placement point of the subsequent model is calculated, and the score is obtained;

[0086] C25. In order to reduce the time consumption caused by model data transmission, all the GPUs where the predecessor models are located should be considered first; in this way, the delay of data transmission can be reduced, and the overall system efficiency can be improved; if the GPU resource of the predecessor model is insufficient, the GPU where other models in the same sample are located can be considered to ensure the continuity and efficiency of model execution.

[0087] The model symbols have the following meanings:

[0088]

[0089] C3. According to the obtained model execution order, the models are executed on the server GPU in sequence.

[0090] The present application can determine the execution relationship of the models by analyzing the serial and parallel constraint relationship between the simulation models. (1) The parallel optimization problem is modeled by using theoretical methods, the parallel degree of the model is calculated, and the hardware parallel capability is improved. (2) Considering the local resource utilization rate, load balancing, resource serving, hierarchical scheduling, GPU acceleration and other factors, intelligent methods are used for global allocation and scheduling of services and resources, task allocation, learning scheduling knowledge, real-time scheduling samples and the like; (3) Considering the model reconstruction, model optimization, model placement and other factors, the application combination and dynamic adjustment of the resource placement algorithm are used for model acceleration. Therefore, the present application comprehensively considers the sample deadline, resource utilization and sample and model acceleration, optimizes the total running time of the task in the test running process, so as to reduce the energy consumption of the cluster.

[0091] The local scheduler should call the model scheduling sequence decision maker to effectively analyze the scheduling sequence according to the DAG graph, and then sequentially schedule each model: after determining all the exit models of the sample, the scheduling sequence of all the models can be obtained according to the inverse order of the priority relationship between the models represented by the DAG graph. The inverse order of the scheduling sequence is the correct scheduling sequence of all the models in the sample.

[0092] The local scheduler should call the model, including the following steps (as shown in Figure 5 ):

[0093] S1. Analyze the sample S i and obtain the scheduling sequence M of each model in the sample S i .

[0094] S2. Obtain the allocation set and the GPU set in the sample, including the following steps:

[0095] S21. Collect the sample Si allocation set of each model

[0096] S22. Collecting sample S i current server ψ i GPU set of models already allocated

[0097] S3. Traversing each GPU Φ on the current server of sample xy , y e {1, 2,..., β}, specifically comprising the following steps:

[0098] S31. If the current GPU Φ is traversed xy , update the GPU set G of models already allocated as the union set of G and Φ xy .

[0099] S4. According to the size of the first scheduled model M i1 , the processing capacity of the GPU instance, estimate the required instance number g1 of M i1 .

[0100] S5. Traversing each GPU in G, select the GPU with the most remaining idle instance number and the idle instance number greater than or equal to g1 as Φ i1 .

[0101] S6. If Φ is empty i1 finally, randomly select a GPU in the GPU set G.

[0102] S7. Assign M i1 to Φ i1 for execution, and store the allocation scheme to the allocation set ≡ i ← ≡ i ∪(M i1 , Φ i1 ).

[0103] S8. If there are still models in S i to be scheduled, traverse all models M ij to execute the following steps:

[0104] S81. According to the size of M ij and the processing capacity of the GPU instance, estimate the required GPU instance number g j ;

[0105] S82. Search the DAG graph to obtain the set of all predecessor models M ij of M pre ;

[0106] S83. Query Allocation Scheme ≡ i Select M pre Number of remaining free instances on the GPU where each model is located Most and Greater than or equal to g j GPU and mark it with Φ ij The tags are for later use;

[0107] S84. If Φ ij If empty, perform the following steps:

[0108] S841. Search the DAG graph to obtain M. ij The set of all non-precursor models M npre ;

[0109] S842. Query Allocation Scheme ≡ i Select M npre Number of remaining free instances on the GPU where each model is located Most and Greater than or equal to g j GPU and mark it with Φ ij The tags are for later use;

[0110] S843. If Φ ij If it is still empty, proceed to step D3 and select the remaining number of free instances in G. Most and GPUs of g1 or higher and marked with Φ ij Tags;

[0111] S9. Calculate sample S i Server ψ i Total GPU instance utilization μ i And return the utilization rate.

Claims

1. A parallel task scheduling method based on node dynamic load and task affinity metric, characterized in that, The specific steps are as follows: A. Preprocessing stage of hierarchical dynamic scheduling: According to the collected resource environment, the resources include computer servers, storage devices, network devices; Then perform computing resource virtualization, abstract physical computing resources into virtual resources through virtualization technology, thereby improving resource utilization and flexibility, and dividing physical GPU resources into multiple virtual GPUs to realize sharing and reuse of GPU resources; Finally, use the high parallel computing capability of GPU to accelerate scientific computing and data processing tasks; B. Global scheduling stage for test task parallel optimization: According to the collected task historical running data, a task prediction model based on random forest is trained, and the task resource demand is obtained based on the prediction results of the above task prediction model; In addition, a probability distribution-based execution time window model is used to estimate the execution time of the task; The node load rate is introduced to evaluate the performance of each computing node, and the performance threshold is used to determine whether the node participates in task distribution; Finally, using the multi-armed bandit theory algorithm framework, a feedback mechanism, a reward model and environment information are designed to build an EXP3 algorithm to select the optimal node corresponding to the task; C. Multi-level model parallelization stage based on domain knowledge and data hybrid driving: This stage is mainly for model parallel local scheduler. The inter-model dependency within the sample is represented by a DAG graph, and the constraint relationship between different inter-models within the sample is heterogeneous. First, analyze the inter-model priority structure within the sample, that is, the DAG graph, and call the model scheduling sequence decision maker to determine the scheduling order of each model according to the DAG graph analysis, and score in the reinforcement learning reward function in sequence, while referring to the GPU instance utilization rate, and finally distribute to the corresponding GPU for execution according to the score.

2. The parallel task scheduling method based on node dynamic load and task correlation metric according to claim 1, characterized in that: The preprocessing stage of hierarchical dynamic scheduling has the following specific steps: A1. Resource environment data collection and recording: Record the real-time data of GPU utilization rate, memory utilization rate and network bandwidth utilization rate of the server to form a data set; In addition, the resource utilization of each computer server, storage device and network device needs to be recorded to ensure a comprehensive understanding of the current resource status; A2. Data analysis and frequency distribution chart drawing: Sample analysis is performed on the data set in A1 to obtain the frequency distribution chart of GPU utilization rate, memory utilization rate and network bandwidth utilization rate; According to the frequency distribution chart and mathematical model, the membership function of the GPU utilization rate, memory utilization rate and network bandwidth utilization rate of each server is determined to provide data support for further resource scheduling; A3. Computing resource virtualization: Through virtualization technology, physical computing resources are abstracted into virtual resources to improve resource utilization and flexibility; Specific operations include virtualization of servers, storage devices and network devices to ensure that various resources can operate effectively in a virtualized environment; A4. GPU virtualization multiplexing: dividing physical GPU resources into multiple virtual GPUs to achieve sharing and multiplexing of GPU resources; through time slice scheduling or hardware division technology, ensuring that each virtual machine can efficiently use GPU resources; A5. GPU acceleration task configuration: using the high parallel computing capability of GPU to accelerate scientific computing and data processing tasks; specifically including decomposing and allocating tasks to GPU for execution to fully utilize the computing performance of GPU, accelerate the processing process, and improve the overall system efficiency.

3. The parallel task scheduling method based on node dynamic load and task affinity metric according to claim 2, characterized in that: The global scheduling phase for test task parallel optimization includes the following steps: B1. Integrate the historical task characteristics and computing resource usage obtained in the preprocessing phase of the hierarchical dynamic scheduling, and divide the data into feature part and label part; the data in the feature part includes server instance number, worker instance number, data set size, and batch task size; the data in the label part includes GPU core usage, GPU memory usage, and bandwidth usage; Bootstrap self-sampling method is used to generate the training set of each decision tree; B2. Input the data obtained in B1 into a random forest model initialized with 20 decision trees; the algorithm for decision tree learning is to recursively select the optimal feature and divide the training data set according to the feature, so that each sub-data set has the best classification process; this process corresponds to the division of the feature space and the construction of the decision tree, thus obtaining the estimated resource demand of the task; B3. Construct a multi-armed bandit method with an adversarial reward model to analyze the computing node load balancing problem using Bandit theory: design a feedback mechanism, reward model, and environment information; when considering the adversarial bandit problem with expert advice, use the EXP3 algorithm to solve it; B4. The EXP3 algorithm first initializes the weight function of each computing node, randomly selects an arm in the multi-armed bandit according to the probability formula, randomly selects a computing node according to the probability distribution, and obtains the cost value of all current computing nodes according to the full feedback mechanism, and finally selects the optimal node corresponding to the task.

4. The parallel task scheduling method based on node dynamic load and task affinity metric according to claim 3, characterized in that, In B2, resource prediction is performed using the random forest model, that is, the random forest model is applied to integrate multiple decision trees to obtain the final strategy, with the following steps: B21. Randomly sample n samples from the sample set with replacement; B22. Randomly select k features from all features, and use these features to establish a decision tree for the selected samples; B23. Repeat the above two steps 20 times to generate 20 decision trees and form a random forest; B24. For new data, make a final decision after each tree decision, and finally vote to determine which class it belongs to; B25. Integrate the results of all decision numbers, convert all results into a task resource demand vector, and obtain the final strategy.

5. The parallel task scheduling method based on node dynamic load and task affinity metric according to claim 3, characterized in that, In B3, the multi-armed bandit method is used to analyze the computing node load balancing problem using Bandit theory, as follows: The multi-armed bandit problem can be simply described as: B31. One scheduler has a possible action to choose, each selection item can be called an arm; consider the scheduling process will perform rounds, each round the scheduler selects an arm to pull, and the pulled arm will feedback a reward value, the final goal is to make the reward value after T rounds as large as possible; B32. For any cost r, for each arm a e [K] t (a) ≥ 0; B33. The sample space for the selected arm is a t ∈ [K]; B34. The cost of the selected arm is r t (a t ); B35. The cost of all arms is r t (a), a e [K]; B36. Define the optimal arm as argmin a∈[K] cost(a), then the regret is defined as: cost(ALG) denotes the expected value of the average loss cost over all choices.

6. The parallel task scheduling method based on node dynamic load and task affinity metric according to claim 1, characterized in that: The multi-level model parallelization stage based on the hybrid driving of domain knowledge and data includes the following steps: C1. Analyzing the priority structure of the models with dependency and priority constraint relationship in the sample, accessing the model scheduling sequence decision maker; C2. In the scheduler, the priority models are calculated by applying the combined and dynamically adjusted resource placement algorithm in the reinforcement learning reward function, and the model execution order is obtained according to the score; C3. According to the obtained model execution order, the models are executed on the server GPU in turn.

7. The parallel task scheduling method based on node dynamic load and task affinity metric according to claim 6, characterized in that: The combined and dynamically adjusted resource placement algorithm applied in C2 includes the following steps: C21. Initialize the parameters of the combined and dynamically adjusted resource placement algorithm, including the number of limited instances that can run on the server, the difference between the maximum resource amount and the minimum resource amount, and the number of models to be placed; C22. Calculate the complementary model set of the model, obtain all non-idle GPUs on the server, and select the GPU with the largest number of remaining idle instances and the instance demand that can be met; C23. Perform a loop operation, calculate the target placement point using the vector distance formula corresponding to the GPU core and GPU memory point that can be placed; C24. After confirming the priority execution model, calculate the placement point of the subsequent model according to it, and obtain the score; C25. To reduce the time consumption caused by model data transmission, the GPU where all the predecessor models are located should be considered first; this can reduce the delay of data transmission and improve the overall system efficiency; if the GPU resource of the predecessor model is insufficient, the GPU where other models in the same sample are located can be considered to ensure the continuity and efficiency of model execution.

Images