Depth learning task node allocation method and system for executing time-aware computing power network heterogeneous GPU (Graphics Processing Unit) cluster

Abstract

The invention discloses an execution time aware computing power network heterogeneous GPU cluster deep learning task node allocation method and system. The method comprises the following steps: firstly, based on a deep learning task, extracting and preprocessing task features and available node features; secondly, a sampler equally divides new tasks without historical data to available nodes, and each node performs mixed sampling on the tasks until all the tasks estimate execution time data; taking execution time data as a training set, taking the task features and the node features as a test set, and using a regression decision tree model to predict the execution time of the task on each node; performing task allocation on each node by using a cost search algorithm and a short job total JCT priority strategy; and finally, periodically monitoring node resources released in the cluster to obtain an optimal node allocation result. According to the method, task delay and total task JCT are remarkably reduced, cluster node resource changes are monitored in real time, and the resource utilization rate is increased.

Description

Technical Field

[0001] This invention relates to the field of computer equipment and task scheduling, and in particular to a method and system for allocating deep learning task nodes in a heterogeneous GPU cluster of a computing power network with execution time awareness. Background Technology

[0002] In recent years, with the rapid development of deep learning technology, especially the introduction of models such as Transformer and MOE, the scale of deep learning models has continued to expand, and the computational demands of deep learning tasks have been increasing. These large-scale models have achieved remarkable results in fields such as natural language processing and computer vision, but they have also brought enormous computational pressure to training and inference. The traditional single-machine, single-GPU training computational capabilities are limited and can no longer meet the needs of large-scale models. Therefore, distributed parallel computing technologies, including single-machine multi-GPU and multi-machine multi-GPU approaches, are needed to accelerate model training and expand its scale.

[0003] However, most clusters consist of GPUs (Graphics Processing Units) of different models and computing capabilities. Failure to allocate these nodes with varying computing power to tasks effectively will result in significant waste of computing resources and prolonged task waiting times, posing a major challenge to parallel computing and resource allocation. The performance of various kernel resources in heterogeneous clusters differs, making it difficult to predict the execution time of individual tasks. Furthermore, the cluster receives a large number of training tasks with varying execution times and deadlines. Therefore, how to rationally and dynamically schedule computing resources for these tasks to meet their deadlines while reducing the overall task completion time is a crucial issue.

[0004] Traditional deep learning task submission involves users specifying one or more GPU nodes for task execution. While this method has no impact when the required GPU nodes are idle, it significantly hinders task submissions when those nodes are busy. Users cannot monitor the real-time status of all GPU nodes, making it difficult to immediately formulate appropriate task allocation strategies. Other tasks may be waiting on the current node, and submitting tasks there will greatly increase the Task Completion Time (JCT). Furthermore, users cannot know the completion times of tasks on each GPU node, potentially causing them to miss suitable idle GPU nodes in the coming period. These scenarios are common but significantly impact the user's computing resource usage experience, leading to task delays and reduced cluster resource utilization.

[0005] In the mainstream trend of deep learning services, too many tasks in the cluster compete for a few GPU resources, causing the JCT (Job Time Completion) of some tasks to often exceed user expectations. In heterogeneous GPU clusters for deep learning, common task scheduling strategies such as First-Come, First-Served, Shortest Job First, and priority strategies are ineffective because they cannot perceive the task's running status and remaining execution time in real time, making it difficult to provide more available resources for tasks nearing their deadlines, thus failing to guarantee that most tasks will complete before their deadlines. Furthermore, in a heterogeneous GPU cluster environment, the JCT of the same task differs significantly across different GPU nodes. Different node allocation schemes result in large differences in the total JCT of tasks, making the node allocation problem complex and diverse. Therefore, providing a node allocation method that accurately perceives task execution time and ensures on-time completion for deadline-sensitive deep learning tasks on heterogeneous GPU clusters is a new challenge facing modern task schedulers.

[0006] To ensure that the completion time of deep learning tasks does not deviate from the deadline, accurate prediction of task execution time is a crucial prerequisite for this work. Alibaba's cluster tracking over two months, involving more than 6,500 GPUs, revealed that approximately 65% ​​of deep learning tasks are repetitive. Traditional task execution prediction methods rely on historical task statistics to build a task pattern model to predict execution time. Specifically, this involves collecting records of previously run tasks in the cluster, including GPU node parameters, task characteristics (such as model structure and dataset size), and actual execution times. Analyzing these records reveals statistical relationships between task characteristics and execution time, such as a linear relationship between data size and execution time. Finally, based on the characteristics of new tasks, the established task pattern model is used to predict the execution time of those tasks. The advantage of this method is its simplicity; it requires no additional computing resources for prediction. However, in the dynamic and complex environment of heterogeneous GPU clusters, traditional methods struggle to learn online and adapt to continuous changes. Over time, the gap between historical patterns and current actual execution increases, leading to decreased prediction accuracy. Furthermore, they cannot accurately predict the execution time of tasks on GPU nodes different from historical ones, and they struggle to distinguish the impact of single-machine multi-GPU parallelism versus cross-node distributed execution. Therefore, this statistical pattern-based method has very limited ability to predict the JCT of deep learning tasks in heterogeneous GPU cluster environments.

[0007] Currently, most research on distributed cluster deep learning task schedulers focuses on optimizing task completion time or improving resource utilization in homogeneous cluster environments. However, there is still a significant gap in research on scheduling in heterogeneous GPU clusters. Deep learning tasks that are sensitive to task deadlines in heterogeneous GPU clusters lack unified management. Further research is needed on accurately predicting task execution time and searching for optimal node allocation schemes in heterogeneous GPU clusters, while simultaneously reducing task delay time and total JCT. Summary of the Invention

[0008] To address the aforementioned issues, this invention proposes a method and system for allocating deep learning task nodes in a heterogeneous GPU cluster with execution time-aware computing power.

[0009] In a first aspect, the present invention provides a method for allocating deep learning task nodes in a heterogeneous GPU cluster of a computing power network with time-aware execution, comprising the following steps:

[0010] Based on the deep learning tasks submitted by users, extract and preprocess their task features and available node features;

[0011] The sampler distributes new tasks without historical data evenly across the available nodes, and each node performs mixed sampling of the tasks until all tasks have estimated execution time data.

[0012] Using the execution time data as a reference, existing historical data as a training set, and the task features and node features as a test set, a regression decision tree model is used to predict the execution time of the task at each node.

[0013] When predicting the execution time, a cost search algorithm is used to allocate tasks to each node with a shortest job total JCT priority strategy;

[0014] The system periodically monitors the released node resources in the cluster. If a new available node is found, it is added to the queue, and unexecuted tasks are marked as pending allocation, triggering the node task allocation process to obtain the optimal node allocation result.

[0015] Secondly, the present invention provides a time-aware computing power network heterogeneous GPU cluster deep learning task node allocation system, comprising:

[0016] The deep learning task preprocessing module is used to extract and preprocess the task features and available node features based on the deep learning tasks submitted by users.

[0017] The iterative execution time sampling module is used by the sampler to evenly distribute new tasks without historical data to the available nodes. Each node performs mixed sampling of tasks until the execution time data of all tasks is estimated.

[0018] The decision tree model prediction module is used to predict the execution time of the task at each node using the execution time data as a reference, existing historical data as a training set, and the task features and node features as a test set.

[0019] The cost search node allocation module is used to allocate tasks to each node using a heuristic cost search algorithm with a shortest job total JCT priority strategy when predicting the execution time.

[0020] The real-time resource monitoring module is used to periodically monitor the node resources released in the cluster. If a new available node is found, it is added to the queue, and unexecuted tasks are marked as pending allocation, triggering the node task allocation process to obtain the optimal node allocation result.

[0021] The beneficial effects of this invention are:

[0022] This invention proposes a hybrid sampling method for the iterative execution time of deep learning tasks. Leveraging the high in-task predictability of deep learning tasks, the total execution time of a task on a corresponding node can be accurately predicted. Secondly, a regression decision tree model is used to predict the execution time of all tasks on each node. The prediction results are then used to perform a minimum cost search to find the optimal node allocation scheme with the minimum total execution time. Compared to sampling on all nodes, the decision tree model significantly reduces prediction overhead and accurately predicts the execution time of a task on each node. Finally, a cost-search node allocation algorithm is used to find the minimum cost allocation scheme, significantly reducing task delays and total task completion time (JCT), improving cluster resource utilization, and enabling real-time monitoring of cluster node resource changes. This significantly reduces task delays and total task completion time, thereby improving the resource utilization of heterogeneous GPU clusters. Attached Figure Description

[0023] Figure 1 A diagram illustrating the node allocation architecture for time-aware deep learning tasks on heterogeneous GPU clusters.

[0024] Figure 2 Flowchart for assigning nodes to the heuristic cost search. Detailed Implementation

[0025] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and do not constitute a limitation thereof.

[0026] This invention provides a method for allocating deep learning task nodes in a heterogeneous GPU cluster with execution time awareness. In the heterogeneous GPU cluster, the execution time of each deep learning task on a specific GPU node is obtained through mixed sampling, thus estimating the execution time of that task on that node. This data is then used as a reference for subsequent decision tree prediction. Next, the execution data of historical deep learning tasks on various nodes in the cluster is used as the training set for the regression decision tree. New task execution data is added to the training set periodically to update the decision tree prediction model. Then, using the task features of a batch of deep learning tasks submitted by the user and the node features of available nodes as input, the decision tree model is used to predict the execution time of each task on each node. Finally, based on the prediction results, a heuristic cost search node allocation algorithm is used, prioritizing shortest job total JCT (Job Time Completion Time). Each task is traversed sequentially, and the allocation scheme with the minimum total JCT is selected until all tasks are allocated. If a node's task sequence has a task delay, it is either promoted to the front of the sequence or swapped with a node with a more lenient task deadline, thus obtaining the optimal node allocation scheme with the minimum task delay and total JCT.

[0027] Specifically, the present invention includes three parts: hybrid sampling of deep learning task iteration execution time, task execution time prediction based on regression decision tree, and real-time resource-aware heuristic cost search node allocation.

[0028] (1) Hybrid sampling of deep learning task iteration execution time

[0029] Furthermore, deep learning is characterized by periodic iterations, with each iteration typically consisting of three steps: forward computation, backpropagation, and gradient update. The time required for each iteration on the same type of GPU is relatively stable, thus the resource usage of a deep learning task exhibits certain regular fluctuations. Due to the repetitive nature of deep learning tasks, some tasks submitted by users may have been previously executed. Therefore, the predictor can cache and reuse historical prediction results and actual execution data from these tasks to predict the execution time of the next round of tasks. The predictor records the execution results of each task, saving data such as the task's file name, submission date, model structure, dataset type and size, task execution node, node GPU type, startup time, iteration time, waiting time, and task JCT. After a batch of deep learning tasks are submitted, the predictor first queries the available nodes in the cluster to see if there is historical prediction data for the corresponding task. If it exists, the predictor will reuse the historical prediction results for the task for subsequent cost search. Finally, during the task execution, the predictor will collect execution data for the task again to update the historical prediction data. If it does not exist, the predictor will perform a mixed sampling task execution time prediction for the task on the node.

[0030] Furthermore, to obtain the execution time of a deep learning task, one of the most direct methods is to run the task until it completes. While this accurately yields the execution time on the GPU-type node, it clearly contradicts the invention's objective of reducing overall latency and node allocation. Therefore, this invention designs a hybrid sampling method for task execution time estimation, which performs hybrid sampling of the execution time of new tasks without historical data on candidate allocable nodes. The tasks exhibit high intra-task predictability, performing repetitive iterative processes with regular behavior and constant execution time. When a batch of tasks { The task was submitted and requested by the user. The training rounds are When the total number of iterations for the task is [number], then [number] iterations are [number]. The total number of tasks can be used as a reference. and the index length of the dataset Through calculation The conclusion is as follows.

[0031] When a GPU executes a deep learning task, it needs to initialize the CUDA library and transfer task data from CPU memory to GPU memory, resulting in a startup overhead known as the "cold start" effect. In terms of iteration execution time, this manifests as a short period after task startup before the data is fully loaded onto the GPU for training, and the execution time of the first few iterations is significantly longer than that of subsequent, more stable iterations. Therefore, if only head or intermediate sampling is performed during the process, the data may not be accurate enough. This is because this sampling method ignores the task's initialization overhead. This invention proposes a hybrid sampling strategy: the first part of the predictor collects the initialization execution time of the task from startup to the start of formal iterative training. The second part of the data collection iteration was not stable enough. Secondary header iteration execution time Once the iteration execution time of the task becomes stable, the predictor begins sampling the iterations of that task. A threshold is set. ,when When the iteration of the task is considered to have reached a stable state, the predictor proceeds to backward sampling. The iteration time is denoted as _n_. , This indicates the number of sampling training iterations. This task is located at node... Total execution time The estimation method is as follows:

[0032]

[0033] Furthermore, the hybrid sampling algorithm for task execution time is as follows:

[0034]

[0035]

[0036]

[0037]

[0038]

[0039]

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] The parameters are explained below:

[0055] Hybrid sampling of deep learning task execution time For the first One task, To specify the first Training rounds for a deep learning task The batch size for this task. For the dataset index length, Specifies the number of sampling iterations;

[0056] Get the current time;

[0057] Task initialization start time;

[0058] Task initialization;

[0059] Task initialization completion time;

[0060] The task's data loader, where These represent the current iteration step and the corresponding training data, respectively.

[0061] : This refers to the training process in one iteration.

[0062] Furthermore, the algorithm steps include:

[0063] Preparation Phase: Users submit a batch of deep learning tasks with deadlines. Tasks without historical prediction data are selected from this batch, and each task is matched with a node for mix sampling. The purpose is to obtain relevant data such as task parameters, model structure, dataset size, and the running status on the corresponding node.

[0064] Sampling Phase: A three-step hybrid sampling process is performed for task initialization, unstable iteration, and stable iteration. The sampled data is used to calculate and predict the task execution time on a given node. After sampling, several one-to-one predicted execution times for deep learning tasks on a single GPU type node are obtained.

[0065] (2) Task execution time prediction based on regression decision tree

[0066] Specifically, the deep learning task execution time prediction data obtained by mixed sampling is used as an important feature. Existing historical data is used as the training set for the regression decision tree prediction model. The features of a new batch of deep learning tasks submitted by users and the features of available GPU nodes are used as the test set. The trained regression decision tree model is then applied to the task execution time prediction.

[0067] Furthermore, task characteristics Including the Task name, task type, model structure, dataset type, dataset size, training epochs, Task features are used to identify specific deep learning tasks, describing task types such as classification, regression, or object detection. The complexity of different model structures directly affects the computational cost of training. The size of the dataset determines the scale of data reading and computation during training, while the number of training epochs and the number of training samples directly impact task execution time. GPU node features. Including the Each GPU node's name, GPU model, GPU architecture, number of CUDA cores, memory bandwidth, and operating frequency are characteristics used to identify and measure the computing power of each GPU node. The execution time of the same task varies significantly across different GPU nodes. Appropriately allocating tasks to suitable nodes will reduce task delays, total JCT, and improve the resource utilization of heterogeneous GPU clusters.

[0068] Furthermore, constructing a regression decision tree model requires the execution time of historical deep learning tasks on each GPU node, as well as task features and GPU node features, as the training dataset, and setting a dynamically updated training interval. When there is a task to perform, every The system automatically adds new task datasets to the training dataset to update the prediction model. Using Mean Absolute Error (MAE) as the criterion for node splitting, for each possible split point of each feature, the MAE of the two child nodes after the split is calculated. The feature and split point that minimizes the MAE of the node samples are selected as the optimal split, with minimum error as the stopping condition. Node splitting is performed recursively to generate a complete regression decision tree.

[0069] Furthermore, using the constructed regression decision tree prediction model, based on the task feature data submitted by the user... and GPU node features This method predicts the execution time of a task on each GPU node. First, task features and node features are fed as input to the regression decision tree model. Starting from the root node, the task features are compared with the root node according to its partitioning rules. Based on the partitioning rules of each node, the model traverses each node of the decision tree sequentially until a leaf node is reached, returning the average target value of the training samples at that node as the predicted value. For each user-submitted task feature, the above steps are repeated to predict the execution time of each task. In various GPUs Execution time Compared to tasks that perform mixed sampling across all nodes, decision tree models significantly reduce prediction overhead.

[0070] (3) Real-time resource-aware heuristic cost search node allocation

[0071] Furthermore, this invention proposes a real-time resource-aware heuristic cost search node allocation algorithm, which aims to dynamically search for the minimum cost node allocation scheme under the premise of minimizing task delay in real time in complex and ever-changing heterogeneous GPU clusters, thereby reducing task delay and improving the total task JCT and cluster resource utilization.

[0072] The minimum cost can be calculated by iterating through all deep learning tasks and all GPU nodes. However, as the number of tasks, available nodes, and their types increase, the cost of minimum cost search will increase exponentially, which is unsuitable for complex and variable heterogeneous GPU clusters. This invention proposes a heuristic cost search algorithm. First, long jobs are allocated to the task sequence of GPU nodes sorted by computing power according to their execution time, ensuring that nodes do not suffer from severe load imbalance due to long jobs. Second, tasks are deployed on GPU nodes with lower computing performance first, based on the total JCT of short jobs, and then allocated to all nodes in turn, while maintaining a minimum total task completion time. During the cost search process, select to make The smallest branch generates the task sequence for each node. This continues until all tasks have been assigned. Simultaneously, it checks for delays in subsequent tasks within the sequence. The task deadline is... If there are task delays in the task sequence of a node, the order of the task sequence is adjusted, moving the delayed tasks to the front of the sequence. If delays still exist, the tasks are swapped with those on nodes with more lenient deadlines. In heterogeneous GPU clusters, while a new batch of deep learning tasks is being executed sequentially, other occupied nodes are usually released. At this time, the real-time resource awareness module needs to capture the new available computing resources and immediately activate the cost search module to add the new nodes to the available resource queue. A new round of cost search is then performed on the unexecuted tasks to obtain the latest minimum cost node allocation scheme.

[0073] Based on the above records, such as Figure 1 As shown, the present invention proposes a time-aware method for allocating deep learning task nodes in a heterogeneous GPU cluster of a computing power network, which is implemented through the following steps:

[0074] Step S1: The user submits a batch of deep learning tasks, and the task features and available GPU node features are extracted;

[0075] Furthermore, when a user submits a batch of deep learning tasks, the first step is to extract and preprocess task features so that these features can be used as part of the input to a regression decision tree model for task execution time prediction. Feature extraction includes extracting task name, task type, model structure, dataset type, dataset size, training epochs, etc. These task features will identify the task, indicating the specific type of task and dataset, facilitating task classification, while the number of training epochs and... This helps the sampler accurately predict the total execution time of a task. Feature preprocessing converts text data into a numerical representation using one-hot encoding, and normalizes numerical features to obtain the same units and ranges. Node feature extraction includes node name, GPU model, architecture, number of CUDA cores, memory bandwidth, and operating frequency. Feature preprocessing converts categorical features into numerical representations and normalizes numerical features.

[0076] Step S2: The sampler distributes the tasks evenly across available GPU nodes and performs a mixed sampling of the task iteration execution time;

[0077] Furthermore, the sampler distributes new tasks without historical data evenly across available nodes, and each task undergoes a mixed sampling on its corresponding node. First, the sampler collects the "cold start" execution time (initialization) of the task from startup to training. Second, the sampler collects the execution time during the unstable iteration phase at the beginning of training; this phase, the task's iteration process has not yet reached a stable state and therefore needs to be considered separately and included in the total JCT calculation. Third, the sampler collects the execution time when the task's training iterations are stable; the task has entered a stable state, and its iteration execution time reflects the overall iteration execution time of the task thereafter. The sampler then uses a formula to combine the mixed data. Predicted The task is in the corresponding Total JCT on the node. Until all tasks have a corresponding execution time data, which is used as a reference for prediction by the decision tree model.

[0078] Step S3: Using the estimated execution time of each task as a reference, use a regression decision tree model to predict the execution time of all tasks on various types of GPU nodes;

[0079] Furthermore, the execution data of historical deep learning tasks in the cluster are used as the training set to train the regression decision tree model at intervals. New task execution data from the cluster is added to the training set over time, and the decision tree is retrained to obtain the latest prediction model. The decision tree uses the mean absolute error as the splitting criterion for each node. When splitting features into nodes, the mean absolute error between the two nodes after the split is calculated. Select the minimized node The optimal partition for a node is determined by the minimum error, and the process stops at this minimum. The regression decision tree model uses the execution time of mixed task sampling as a reference, improving the accuracy of the decision tree prediction results. Using task features and node features as the test set, the estimated execution time of each task on each available node is output.

[0080] Step S4: Distribute long tasks according to the computing power of the nodes;

[0081] Furthermore, since the cost search algorithm adopts a short-job total JCT priority strategy, long tasks will be allocated at the end of the allocation process. However, long tasks have long execution times and large differences in execution time between tasks, which can easily lead to some nodes being assigned long-running tasks in the final stage of node allocation, resulting in a large total JCT for that batch of deep learning tasks. Therefore, the current step first allocates long tasks to each node in order of their computing power to avoid severe load imbalance between nodes.

[0082] Furthermore, such as Figure 2 As shown, for unassigned tasks, following the shortest job total JCT priority strategy, a deep learning task is cyclically allocated according to the node's computing power from smallest to largest. The allocation decision with the minimum cost at each step is searched, and compared with the total JCT, the node allocation path with the shortest JCT is selected. This cost-search node allocation algorithm is repeatedly executed for all unassigned tasks until all tasks are allocated. After cost-search node allocation is completed, each node has its own task sequence. .

[0083] Step S5: Determine if there are any task delays in the task sequence of each node;

[0084] Furthermore, the deadline for deep learning tasks with a specified timeframe is... After all tasks have been assigned to nodes, the start time and JCT of each task on the corresponding node are compared with the deadline of the task to check whether there are any task delays in the task sequence of each node.

[0085] If there is a delay, the delayed task will be moved to the front of the sequence. If there is still a delay, the task will be swapped with a node with a more lenient deadline.

[0086] Furthermore, to ensure timely completion of tasks, delayed tasks are moved to the front of the task sequence to advance their completion time. If task delays persist, they are swapped with tasks that have more lenient deadlines to minimize delays.

[0087] Step S6: Periodically monitor whether any additional nodes have been released in the cluster. If so, add the node to the available node queue, classify unexecuted tasks as unassigned, and trigger the node allocation process.

[0088] Furthermore, in order to make timely use of the released GPU node computing resources, the system needs to monitor in real time whether there are any additional nodes in the cluster whose computing resources are released. If there are new available GPU resources, they are added to the available node queue, and unexecuted tasks are classified as unexecuted, thus triggering the cost search node allocation module.

[0089] like Figure 1 As shown, based on the same concept as the above method embodiments, this application also proposes a time-aware computing power network heterogeneous GPU cluster deep learning task node allocation system, including:

[0090] Deep Learning Task Preprocessing Module: When users submit a batch of new deep learning tasks, to estimate the execution time of the task at the corresponding node after mixed sampling of task iteration execution times, and to meet the input requirements of the regression decision tree model, the tasks are preprocessed. This includes task feature extraction and preprocessing, as well as feature extraction and preprocessing of each node in the available node queue. Task feature extraction includes task name, task type, model structure, dataset type, dataset size, training epochs, etc. Task feature preprocessing includes converting textual data such as task name, task type, and model structure into numerical representations using one-hot encoding. For numerical features (such as dataset size, training epochs, etc.), ... Features such as GPU name, GPU model, GPU architecture, number of CUDA cores, memory bandwidth, and operating frequency are normalized to ensure consistent dimensions and range, thereby improving model accuracy. Node feature extraction includes node name, GPU model, GPU architecture, number of CUDA cores, memory bandwidth, and operating frequency. Node feature preprocessing involves converting categorical features into numerical representations using one-hot encoding, and normalizing numerical features (such as memory, number of cores, and frequency). These features are used for training and prediction in the decision tree model.

[0091] Iterative execution time sampling module: Distribute all new tasks without historical data evenly across nodes for mixed sampling. The first step collects the task startup overhead time, and the second step collects the time after the task starts training. The execution time of the unstable iteration and the third step of data collection. The stable iteration execution time is formed by the combination of these three parts in the entire sampling process. After sampling is completed, the execution of this task ends, and the next task's mixed sampling is performed. At the same time, the data obtained from the three-step mixed sampling is used to calculate the estimated execution time of the task at this node. Hybrid sampling takes into account the "cold start" overhead of deep learning tasks and the unstable iteration execution time of the head of the training process, and can accurately reflect the execution time of the task at a specific node.

[0092] Decision tree model prediction module: The regression decision tree model uses the execution data of historical deep learning tasks in the cluster as the training set, and performs predictions at intervals. New task execution data is then added to the training set for repeated training to update the decision tree model. Mean absolute error (MAE) is used as the criterion for node splitting. For each possible split point of each feature, the MAE of the two child nodes after the split is calculated. The feature and split point that minimizes the MAE of the node samples are selected as the optimal split, with minimum error as the stopping criterion. The decision tree model uses the execution time estimated by each task through mixed sampling as a reference, and the task features and available node features of each task as test set inputs to predict the execution time of each task on each available node. .

[0093] Cost Search Node Allocation Module: Based on the results of the decision tree prediction module, long tasks are first allocated to nodes in descending order of computing power to prevent excessive differences in the total JCT (Job Computation Time) across nodes due to the allocation of long tasks at the end. Using a short job total JCT priority strategy, tasks are first deployed on GPU nodes with lower computing performance, and then allocated to all nodes sequentially, while maintaining a minimum total task completion time. Each step of the node traversal selects the allocation scheme that minimizes the total completion time, until all tasks are allocated, generating the task sequence for each node. This search algorithm can significantly reduce the search scope and lower search costs. After the search is completed, the timeframe is determined based on the task deadline. The algorithm checks for task delays in the task sequence of each node. If a delay is found, the task is moved to the front of the sequence or swapped with another node that has a more lenient deadline, ensuring the task is completed on time. The final output is the optimal node allocation scheme with the minimum task delay and total JCT.

[0094] Real-time resource monitoring module: Heterogeneous GPU clusters typically have complex and variable operating environments, making it difficult to predict the performance of each node manually. The resource monitoring module proposed in this invention monitors the resource usage of each node in the cluster in real time. Once a new node is released and confirmed to be available, it is added to the available node queue. Furthermore, the cost-search node allocation algorithm is re-executed for unexecuted tasks, further reducing task delays, lowering the total JCT, and improving cluster resource utilization.

[0095] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0096] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for allocating deep learning task nodes in a heterogeneous GPU cluster with time-aware computing power, characterized in that, Includes the following steps: Based on the deep learning tasks submitted by users, extract and preprocess their task features and available node features; The sampler distributes new tasks without historical data evenly across the available nodes, and each node performs mixed sampling of the tasks until all tasks have estimated execution time data. Using the execution time data as a reference, existing historical data as a training set, and the task features and node features as a test set, a regression decision tree model is used to predict the execution time of the task at each node. When predicting the execution time, a cost search algorithm is used to allocate tasks to each node with a shortest job total JCT priority strategy; Periodically monitor the released node resources in the cluster. If a new available node is found, add it to the queue, mark unexecuted tasks as pending allocation, and trigger the node task allocation process to obtain the optimal node allocation result.

2. The method according to claim 1, characterized in that, The task features include task name, task type, model structure, dataset type, dataset size, training epochs, and... ; The node characteristics include node name, GPU model, GPU architecture, number of CUDA cores, memory bandwidth, and operating frequency.

3. The method according to claim 1 or 2, characterized in that, The preprocessing includes converting task features or node features into numerical representations through one-hot encoding, and normalizing the numerical features.

4. The method according to claim 1, characterized in that, Predicting the execution time of a task at each node using the regression decision tree model includes the following steps: a. Input the task features and node features into the regression decision tree model, and compare the task features layer by layer starting from the root node; b. Traverse the decision tree according to the node partitioning rules, and return the average target value as the predicted value after reaching a leaf node; c. Repeat steps a and b to predict the execution time of each task on each node.

5. The method according to claim 1 or 4, characterized in that, The estimated execution time of the task at this node is calculated based on the data obtained from the hybrid sampling, including: The first step is the startup time of the data collection task; The second step involves collecting data on the unstable execution time during the initial iterative training phase of the task. The third step is to collect data on the training and iteration of stable execution times for the task.

6. The method according to claim 4, characterized in that, The regression decision tree model uses the mean absolute error (MAE) as the node splitting criterion. By calculating the MAE of the child nodes corresponding to all possible split points for each feature, it selects the feature and split point that minimizes the sample MAE as the optimal split, and uses the minimum error as the stopping condition.

7. The method according to claim 1, characterized in that, The cost search algorithm includes the following steps: Distribute long tasks according to the computing power of the nodes; Unassigned tasks are allocated in a loop based on the node's computing power, from smallest to largest, with each step making the allocation decision based on the minimum cost, until all tasks are allocated.

8. The method according to claim 7, characterized in that, In a heterogeneous GPU cluster, the cost search algorithm captures released nodes and adds them to the available resource queue through a real-time resource awareness module, and performs a new round of cost search on unexecuted tasks to obtain the minimum cost allocation scheme.

9. The method according to claim 1 or 8, characterized in that, After the tasks are assigned, the start time, JCT and deadline of each node task are compared to check if there are any delays in the node sequence. If a task is delayed, it is moved to the front of the sequence. If it is still delayed, it is swapped with a node task with a more lenient deadline.

10. A time-aware computing power network heterogeneous GPU cluster deep learning task node allocation system, characterized in that, include: The deep learning task preprocessing module is used to extract and preprocess the task features and available node features based on the deep learning tasks submitted by users. The iterative execution time sampling module is used by the sampler to evenly distribute new tasks without historical data to the available nodes. Each node performs mixed sampling of tasks until the execution time data of all tasks is estimated. The decision tree model prediction module is used to predict the execution time of the task at each node using the execution time data as a reference, existing historical data as a training set, and the task features and node features as a test set. The cost search node allocation module is used to allocate tasks to each node using a heuristic cost search algorithm with a shortest job total JCT priority strategy when predicting the execution time. The real-time resource monitoring module is used to periodically monitor the node resources released in the cluster. If a new available node is found, it is added to the queue, and unexecuted tasks are marked as pending allocation, triggering the node task allocation process to obtain the optimal node allocation result.

Images