Task scheduling methods, devices, storage media and electronic equipment

Abstract

This application discloses a task scheduling method, apparatus, storage medium, and electronic device, relating to the field of cloud computing. The method includes: constructing a task model based on a target training task, wherein the task model characterizes the attributes and resource requirements of the target training task; predicting the performance results of the target training task executed on N computing nodes based on the task model, obtaining N performance results, where N is an integer greater than 1, and the performance results include the execution efficiency and resource utilization of the target training task on the computing nodes; filtering the N computing nodes according to the N performance results and network environment information to obtain S first computing nodes, and determining a target node from the S first computing nodes; and executing the target training task through the target node. This application solves the technical problem of low scheduling efficiency for deep learning training tasks in the prior art.

Description

Technical Field

[0001] This application relates to the field of cloud computing, and more specifically, to a task scheduling method, apparatus, storage medium, and electronic device. Background Technology

[0002] In the context of current cloud computing and big data processing, efficient scheduling of deep learning training tasks has become a key bottleneck restricting the rapid iteration and deployment of AI models. Traditional cluster job scheduling algorithms, such as First-Come, First-Served (FCFS), Shortest Job First (SJF), or priority-based scheduling, while performing well in general computing tasks, are inefficient when handling deep learning tasks. This is mainly because these algorithms fail to fully consider the dynamic and heterogeneous resource requirements of deep learning tasks, as well as the complexity of inter-task communication. For example, large model training often requires high computing power and large GPU memory, while real-time lightweight inference tasks emphasize low latency and network stability. Existing technologies often employ static strategies in resource allocation, failing to respond to the real-time status of tasks and resources, leading to resource waste and task delays, severely impacting overall training efficiency and user experience.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a task scheduling method, apparatus, storage medium, and electronic device to at least solve the technical problem of low scheduling efficiency of deep learning training tasks in the prior art.

[0005] According to one aspect of this application, a task scheduling method is provided, comprising: constructing a task model based on a target training task, wherein the task model is used to characterize the attributes and resource requirements of the target training task; predicting the performance results of the target training task when executed on N computing nodes based on the task model, obtaining N performance results, wherein N is an integer greater than 1, and the performance results include the execution efficiency and resource utilization of the target training task on the computing nodes; filtering the N computing nodes according to the N performance results and network environment information to obtain S first computing nodes, and determining a target node from the S first computing nodes, wherein S is an integer greater than or equal to 1 and less than or equal to N, the first computing nodes are used to characterize computing nodes that meet the network environment information, and the target node is used to characterize first computing nodes that meet preset requirements, wherein the preset requirements are used to constrain the duration of the computing nodes executing the target training task; and executing the target training task through the target node.

[0006] Optionally, based on the task model, the performance results of the target training task executed on N computing nodes are predicted to obtain N performance results, including: based on the task model, a first value of the target training task on each computing node is predicted to obtain N first values, wherein the first value is used to characterize the computing resource consumption of the graphics processor in each computing node when executing the target training task; based on the task model, a second value of the target training task on each computing node is predicted to obtain N second values, wherein the second values ​​are used to characterize the transmission time generated by data exchange between computing nodes; based on the task model, a third value of the target training task on each computing node is predicted to obtain N third values, wherein the third values ​​are used to characterize the video memory space required by the graphics processor in each computing node when executing the target training task; and N performance results are determined based on the N first values, N second values, and N third values.

[0007] Optionally, the task model includes the target neural network for the training task, the number of iterations, and the batch size. The batch size is used to characterize the amount of data processed by the target neural network in each round of training. Based on the task model, the first value of the target training task on each computing node is predicted to obtain N first values, including: when there are M network layers in the target neural network, the computational cost of each network layer is determined according to the target information of each network layer and the batch size to obtain M computational costs. The target information is used to characterize the parameter information required by each network layer when performing the target training task; the computational cost of the target neural network is determined according to the M computational costs and the number of iterations; the first value corresponding to each computing node is determined according to the computational cost of the target neural network and the number of graphics processors corresponding to each computing node to obtain N first values.

[0008] Optionally, the task model also includes a parallel approach, wherein the parallel approach is used to characterize the resource allocation and computational task partitioning strategy during the training of the target neural network. Based on the task model, the second value of the target training task on each computing node is predicted to obtain N second values, including: when the parallel approach is data parallel, the second value corresponding to each computing node is determined according to the number of graphics processors corresponding to each computing node, the number of parameters of each network layer in the M network layers, and the number of iterations, to obtain N second values; when the parallel approach is tensor parallel, the second value corresponding to each computing node is determined according to the number of graphics processors corresponding to each computing node, the feature tensor output by each network layer in the M network layers, the batch size, and the number of iterations, to obtain N second values.

[0009] Optionally, based on the task model, a third value for the target training task is predicted at each computing node to obtain N third values, including: determining an optimizer, wherein the optimizer is used to adjust and store the model parameters of the target neural network; determining a first influence value of the optimizer on the GPU memory, wherein the first influence value is used to quantify the GPU memory space occupied by the optimizer storing data; determining a second influence value according to the task type of the target training task, wherein the second influence value is used to quantify the GPU memory occupancy of the intermediate feature map output by any network layer in the target neural network; and determining the third value corresponding to each computing node based on the target information of any network layer among the M network layers, the first influence value, the second influence value, and the batch size to obtain N third values.

[0010] Optionally, the N computing nodes are filtered based on N performance results and network environment information to obtain S first computing nodes, and the target node is determined from the S first computing nodes. This includes: filtering the N computing nodes based on N third values ​​corresponding to the N computing nodes in the N performance results and network environment information to obtain S first computing nodes; calculating a fourth value for each first computing node based on the first value, second value, and average utilization of the graphics processor, to obtain S fourth values, where the fourth value is used to characterize the total time consumed by each first computing node in executing the target training task; and selecting the first computing node corresponding to the fourth value less than a preset threshold from the S fourth values ​​as the target node.

[0011] Optionally, the N computing nodes are filtered based on the N third values ​​corresponding to the N computing nodes in the N performance results and the network environment information to obtain S first computing nodes. This includes: detecting whether the third value corresponding to each computing node in the N performance results is greater than or equal to the video memory capacity corresponding to that computing node in the network environment information; filtering the computing nodes whose third values ​​are greater than or equal to the video memory capacity corresponding to that computing node in the network environment information; and selecting the computing nodes whose third values ​​are less than the video memory capacity corresponding to that computing node in the network environment information as first computing nodes, thus obtaining S first computing nodes.

[0012] According to another aspect of this application, a task scheduling apparatus is also provided, comprising: a construction unit, configured to construct a task model based on a target training task, wherein the task model is used to characterize the attributes and resource requirements of the target training task; a prediction unit, configured to predict the performance results of the target training task executed on N computing nodes based on the task model, obtaining N performance results, wherein N is an integer greater than 1, and the performance results include the execution efficiency and resource utilization of the target training task on the computing nodes; a determination unit, configured to filter the N computing nodes according to the N performance results and network environment information, obtaining S first computing nodes, and determining a target node from the S first computing nodes, wherein S is an integer greater than or equal to 1 and less than or equal to N, the first computing nodes are used to characterize computing nodes that meet the network environment information, and the target node is used to characterize first computing nodes that meet preset requirements, wherein the preset requirements are used to constrain the duration for which the computing nodes execute the target training task; and an execution unit, configured to execute the target training task through the target node.

[0013] According to another aspect of this application, a computer-readable storage medium is also provided, which includes a stored executable program, wherein the executable program controls the device where the computer-readable storage medium is located to perform the above-described task scheduling method when it runs.

[0014] According to another aspect of this application, an electronic device is also provided, including one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the task scheduling method described above.

[0015] In this application, firstly, a task model is constructed based on the target training task. This task model characterizes the attributes and resource requirements of the target training task. Secondly, based on the task model, the performance results of the target training task executed on N computing nodes are predicted, resulting in N performance results, where N is an integer greater than 1. These performance results include the execution efficiency and resource utilization of the target training task on the computing nodes. Then, based on the N performance results and network environment information, the N computing nodes are filtered to obtain S first computing nodes. Finally, the target node is determined from these S first computing nodes, where S is an integer greater than or equal to 1 and less than or equal to N. A node is used to represent a computing node that meets the network environment information, and a target node is used to represent a first computing node that meets the preset requirements. The preset requirements are used to constrain the time for computing nodes to execute the target training task. Finally, the target training task is executed through the target node. That is, by constructing the task model, predicting performance, and filtering and selecting computing nodes based on network environment information, the goal of accurately matching computing resources with the needs of deep learning training tasks is achieved. This realizes the technical effect of improving the scheduling efficiency of deep learning training tasks and optimizing the utilization of computing resources, thereby solving the technical problem of low scheduling efficiency of deep learning training tasks in the prior art. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a flowchart of an optional task scheduling method according to an embodiment of this application;

[0018] Figure 2 This is a structural diagram of an optional task scheduling method according to an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of an optional task scheduling method according to an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of an optional task scheduling device according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0024] According to an embodiment of this application, a method embodiment of a task scheduling method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0025] It should be noted that an intelligent scheduling system can serve as the execution entity for the task scheduling method in this application embodiment. It is understood that the task scheduling method provided in this application embodiment can also be executed by other systems or devices, and this application embodiment does not specifically limit this.

[0026] Figure 1 This is a flowchart of an optional task scheduling method according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0027] Step S101: Construct a task model based on the target training task.

[0028] In step S101, the task model is used to characterize the attributes and resource requirements of the target training task.

[0029] Optionally, the target training task refers to the deep learning training task submitted by the user, which has a specific model structure, algorithm hyperparameters, dataset size, and expected training objective.

[0030] Optionally, the intelligent scheduling system receives and parses the task request information submitted by the user, including model structure, batch size, number of iterations, neural network structure and parallel mode, etc. This information is compiled and modeled into a formatted model describing task attributes and resource requirements, i.e., task model.

[0031] Optionally, the task model serves as the basis for subsequent resource requirement assessment and task scheduling. It ensures that the intelligent scheduling system can accurately understand the attributes and resource requirements of the task, providing precise data input for subsequent steps.

[0032] Step S102: Based on the task model, predict the performance results of the target training task when it is executed on N computing nodes, and obtain N performance results.

[0033] In step S102, N is an integer greater than 1, and the performance results include the execution efficiency and resource utilization of the target training task on the computing node.

[0034] Optionally, N computing nodes, where N represents the total number of available nodes in the intelligent scheduling system, including various GPU (Graphics Processing Unit) accelerated nodes, each with different computing power, video memory size and network bandwidth.

[0035] Optionally, performance results refer to the expected execution efficiency (such as the computational load and communication overhead of the task) and resource utilization (such as memory usage) of the deep learning training task when executed on each computing node.

[0036] Optionally, the intelligent scheduling system can predict the expected performance of a task on different nodes in advance, which helps to filter out unsuitable nodes and provides a basis for decision-making for subsequent task scheduling.

[0037] Step S103: Filter the N computing nodes based on the N performance results and network environment information to obtain S first computing nodes, and determine the target node from the S first computing nodes.

[0038] In step S103, S is an integer greater than or equal to 1 and less than or equal to N. The first computing node is used to represent the computing node that meets the network environment information, and the target node is used to represent the first computing node that meets the preset requirements.

[0039] In step S103, preset requirements are used to constrain the duration for computing nodes to execute the target training task.

[0040] Optionally, network environment information refers to a series of metrics data related to the network status and computing node status of the intelligent scheduling system, collected and monitored in real time by the GPU cluster management module. This information plays a crucial role in the scheduling and resource allocation of deep learning training tasks, ensuring optimal task performance and efficient utilization of cluster resources.

[0041] Optionally, network environment information includes, but is not limited to, GPU resource status (including the utilization, idle status, memory usage, and computing performance of each GPU, used to evaluate the GPU's ability and availability to execute tasks), CPU resource status (CPU utilization, number of idle cores, memory usage, etc. Although deep learning tasks mainly rely on GPUs, the CPU resource status also affects task preparation and data processing speed), network status (involving key performance indicators such as network bandwidth, latency, and packet loss rate between computing nodes. For distributed deep learning tasks, network status directly affects the synchronization efficiency of model parameters and data transmission speed), storage status (including the read / write speed and remaining space of storage devices. For data-intensive deep learning tasks, storage performance directly affects data loading and storage efficiency), and node health status (the online status, fault records, temperature, and power status of nodes, ensuring that computing nodes are in normal working condition and avoiding task scheduling to faulty or unstable nodes).

[0042] Optionally, the S first computing nodes are selected by the system from those nodes that can effectively perform deep learning tasks and whose network conditions meet the requirements by comparing the performance results of the computing nodes with the network environment information.

[0043] Optionally, the target node refers to the node that the intelligent scheduling system further selects from the S first computing nodes that meets the preset requirements, that is, the first computing node that can execute the target training task with minimal time cost.

[0044] Optionally, the intelligent scheduling system further narrows down the range of node selection by using network environment information and performance prediction results, ensuring that the selected nodes not only have sufficient computing resources but also excellent network conditions, thereby reducing the overall time cost of task execution.

[0045] Step S104: Execute the target training task through the target node.

[0046] Optionally, the target training task is assigned to the target node for execution. The intelligent scheduling system distributes resources such as task configuration files and datasets to the target node through the task scheduling module, starts the container environment and executes the deep learning task, and monitors the task status and resource usage in real time to ensure the smooth operation of the task.

[0047] As can be seen from steps S101 to S104, in this application, firstly, a task model is constructed based on the target training task. This task model characterizes the attributes and resource requirements of the target training task. Secondly, based on the task model, the performance results of the target training task executed on N computing nodes are predicted, resulting in N performance results, where N is an integer greater than 1. These performance results include the execution efficiency and resource utilization of the target training task on the computing nodes. Then, the N computing nodes are filtered based on the N performance results and network environment information to obtain S first computing nodes. Finally, the target node is determined from these S first computing nodes, where S is greater than or equal to 1 and less than or equal to 1. The integer N represents the first computing node, which is used to represent the computing node that meets the network environment information. The target node represents the first computing node that meets the preset requirements. The preset requirements are used to constrain the time for the computing node to execute the target training task. Finally, the target training task is executed through the target node. That is, by constructing the task model, predicting performance, and filtering and selecting computing nodes based on network environment information, the goal of accurately matching computing resources with the needs of deep learning training tasks is achieved. This achieves the technical effect of improving the scheduling efficiency of deep learning training tasks and optimizing the utilization of computing resources, thereby solving the technical problem of low scheduling efficiency of deep learning training tasks in the prior art.

[0048] In one optional embodiment, the intelligent scheduling system predicts a first value of the target training task on each computing node based on the task model, obtaining N first values, wherein the first values ​​are used to characterize the computing resource consumption of the graphics processor in each computing node when executing the target training task; predicts a second value of the target training task on each computing node based on the task model, obtaining N second values, wherein the second values ​​are used to characterize the transmission time generated by data exchange between computing nodes; and predicts a third value of the target training task on each computing node based on the task model, obtaining N third values, wherein the third values ​​are used to characterize the video memory space required by the graphics processor in each computing node when executing the target training task. Then, N performance results are determined based on the N first values, N second values, and N third values.

[0049] Optionally, a graphics processor refers to a GPU accelerator, the core processing unit for parallel computing of deep learning tasks, which has high parallel computing capabilities and large-scale video memory.

[0050] Optionally, the first value refers to the predicted computational resource consumption of the GPU in each computing node when executing the target training task, that is, reflecting the computational overhead of the task on different nodes.

[0051] Optionally, the second value refers to the transmission time generated by the predicted data exchange (such as gradient synchronization) between computing nodes, which is used to evaluate the communication efficiency of the task, i.e. the communication overhead of the task.

[0052] Optionally, the third value refers to the predicted memory space required by the GPU to perform tasks in each computing node, ensuring the reasonable allocation of memory resources to avoid overflow.

[0053] Optionally, firstly, the intelligent scheduling system constructs a task model based on the user-uploaded deep learning task description, including model structure (such as convolutional layers, fully connected layers, etc.), algorithm parameters (such as batch size, number of iterations), and parallelism strategy (data parallelism or tensor parallelism). Next, the system calls API commands provided by the PyTorch framework (a machine learning library), such as `torch.summary`, to analyze the model structure and parameter configuration, calculate the number of parameters, computational cost, and size of intermediate feature maps for each layer, and then predict the computational resource consumption of the GPU executing the task on each computing node, obtaining N first values. Based on the Ring-AllReduce architecture (a communication mode for distributed deep learning training), the system calculates the synchronization communication overhead between GPU nodes under different parallelism strategies (data parallelism or tensor parallelism), including intra-layer and inter-layer communication volume and transmission time, obtaining N second values. Based on the task model, the system evaluates the memory requirements of model parameters, intermediate feature maps, and optimizer selection (such as SGD (Stochastic Gradient Descent) and Adam (Adaptive Moment Estimation)). Considering the impact of different task types (training or inference) on memory usage, it predicts the memory space required by the GPU when executing tasks in each computing node, and obtains N third values.

[0054] Optionally, the intelligent scheduling system combines N first values, N second values, and N third values ​​to comprehensively analyze the task execution status on each computing node and calculates N performance results through algorithms. These results not only take into account computational overhead but also communication latency and memory usage, thus comprehensively reflecting the expected performance of the task on different nodes.

[0055] As described above, the intelligent scheduling system, through task models, can make fine-grained predictions of the resource requirements of deep learning tasks, down to computational overhead, communication time, and memory usage. Furthermore, it can achieve comprehensive evaluation of task performance and dynamic optimization of resource allocation, ultimately significantly improving the execution efficiency and resource utilization of deep learning tasks.

[0056] In one optional embodiment, when there are M network layers in the target neural network, the intelligent scheduling system determines the computational cost of each network layer based on the target information of each network layer and the batch size, thus obtaining M computational costs. The target information is used to characterize the parameter information required by each network layer when performing the target training task. Then, the computational cost of the target neural network is determined based on the M computational costs and the number of iterations. Finally, the first value corresponding to each computing node is determined based on the computational cost of the target neural network and the number of graphics processors corresponding to each computing node, thus obtaining N first values.

[0057] Optionally, the target neural network is a neural network model submitted by the user for deep learning training, and its structure and parameters will serve as the basis for scheduling decisions; M network layers refer to the set of all layers contained in the target neural network, where M is the number of network layers, such as convolutional layers and fully connected layers; target information: describes the parameter information required by each network layer when performing the target training task, such as convolutional layers, including but not limited to the number of input channels, the height, width and number of channels of the output feature map, and the width of the convolutional kernel; computational cost is the computational requirement of the graphics processor for the computational task, including the number of multiplications and additions of convolutional and fully connected layers and the number of gradient updates; N first values: N is the total number of computing nodes, and each node corresponds to a first value, reflecting its GPU's computational overhead when processing deep learning tasks.

[0058] Optionally, the intelligent scheduling system first analyzes the structure of the target neural network, extracting target information for each network layer, including the number of parameters and the size of the input and output features, laying the foundation for subsequent computational cost prediction. Based on the target information and batch size, the system predicts the computational cost of each of the M network layers. For example, for convolutional layers, the computational cost of each layer is calculated based on the number of input channels, the height, width, and number of channels of the output feature map, and the width of the convolutional kernel. For fully connected layers, the computational cost is predicted based on the number of input neurons, the number of output neurons, and the batch size. This yields M computational costs, each corresponding to a layer in the network. The system further determines the total computational cost of the entire target neural network based on the M computational costs and the number of iterations. The total computational cost equals the sum of the computational costs of each layer multiplied by the number of iterations, reflecting the computational resources required to complete one training cycle.

[0059] Optionally, considering the varying number of GPUs per computing node in distributed training, the system predicts the computational resource consumption (i.e., the first value) of each GPU in each computing node when executing deep learning tasks, based on the computational complexity of the target neural network and the number of GPUs on each computing node. The parallel processing capabilities of the GPUs are taken into account during the calculation to ensure that the prediction results match the actual situation. The system performs the above prediction process on all N computing nodes, ultimately obtaining N first values, each clearly reflecting the computational resource consumption of the GPUs on the corresponding computing node when executing deep learning tasks.

[0060] As can be seen from the above, by extracting the target information of the network layers layer by layer and combining the batch size and the number of iterations, the intelligent scheduling system can accurately predict the computational load of the target neural network in each computational layer, thereby accurately assessing the computational resource consumption of each computing node. Furthermore, by predicting the computational resource consumption of the computing nodes, the system can significantly improve the execution efficiency of deep learning tasks, optimize the allocation of GPU resources, avoid resource waste, and improve the resource utilization of the entire computing cluster.

[0061] In an optional embodiment, the task model further includes a parallel mode, wherein the parallel mode is used to characterize the strategy of resource allocation and computational task partitioning during the training of the target neural network. Based on the task model, a second value of the target training task on each computing node is predicted to obtain N second values, including: when the parallel mode is data parallelism, the intelligent scheduling system determines the second value corresponding to each computing node according to the number of graphics processors corresponding to each computing node, the number of parameters of each network layer in the M network layers, and the number of iterations, to obtain N second values; when the parallel mode is tensor parallelism, the intelligent scheduling system determines the second value corresponding to each computing node according to the number of graphics processors corresponding to each computing node, the feature tensor output by each network layer in the M network layers, the batch size, and the number of iterations, to obtain N second values.

[0062] Optionally, the parallel approach is used to describe the strategy for resource allocation and computational task partitioning during the training of the target neural network, and is mainly divided into two types: data parallelism and tensor parallelism.

[0063] Alternatively, data parallelism refers to dividing the training dataset into several parts, with each part independently performing the same model computations on different computing nodes, and finally summing the computation results from all nodes. This approach is often used for training tasks with relatively small model parameters and a large amount of data.

[0064] Alternatively, tensor parallelism refers to dividing the model weights or feature maps into multiple tensors, computing them on GPUs on different computing nodes, and finally updating the model by aggregating the tensor results through communication. This approach is suitable for scenarios with large model parameters and heavy computational load.

[0065] Optionally, the intelligent scheduling system first determines whether the target training task adopts a data parallelism or tensor parallelism strategy based on the parallelism field in the task model.

[0066] Optionally, if the parallelism is data parallelism, the intelligent scheduling system will predict the time required for data exchange based on the number of GPUs on each computing node, the number of parameters in each network layer, and the number of iterations. Specifically, when each network layer synchronizes gradients between GPUs via the Ring-AllReduce mechanism, there is a fixed amount of communication, which depends on the number of model parameters, the number of GPUs, and the number of iterations. The intelligent scheduling system calculates the data exchange and transmission time between computing nodes based on these factors, i.e., the second value corresponding to each node, ultimately obtaining N second values.

[0067] Optionally, if tensor parallelism is used, the intelligent scheduling system predicts the communication time based on the number of GPUs on each computing node, the size of the feature tensors output by each network layer, the batch size, and the number of iterations. In tensor parallelism, calculating the communication time involves effectively segmenting and reassembling model parameters or intermediate feature maps. The intelligent scheduling system combines this data with network bandwidth and the number of GPUs to predict the transmission time required for each computing node to synchronize feature tensors, thus obtaining N second values.

[0068] As described above, the intelligent scheduling system, based on the specific parallelism method (data parallelism or tensor parallelism), can refine the prediction of the second value of communication overhead during parallel training, ensuring the accuracy of the prediction results. This helps in the advance planning and optimization of data transmission strategies. By predicting the communication latency between computing nodes, the system can better coordinate parallelism strategies and resource allocation, ensuring that data transmission does not become a bottleneck and improving the overall execution efficiency of deep learning tasks. Accurately predicting the communication time of parallel training helps the intelligent scheduling system optimize task placement strategies, reduce unnecessary communication latency, and thus significantly improve the performance of distributed deep learning training tasks and shorten task completion time.

[0069] In one optional embodiment, the third value of the target training task on each computing node is predicted based on the task model to obtain N third values, including: First, the intelligent scheduling system determines the optimizer, wherein the optimizer is used to adjust and store the model parameters of the target neural network; then, the first influence value of the optimizer on the video memory in the graphics processor is determined, wherein the first influence value is used to quantify the video memory space occupied by the optimizer storing data; then, the second influence value is determined according to the task type of the target training task, wherein the second influence value is used to quantify the occupation of the intermediate feature map output by any network layer in the target neural network in the video memory of the graphics processor; finally, the third value corresponding to each computing node is determined according to the target information of any network layer in the M network layers, the first influence value, the second influence value, and the batch size to obtain N third values.

[0070] Alternatively, an optimizer refers to an algorithm used to adjust model parameters during deep learning training, such as SGD and Adam. Different optimizers have a direct impact on GPU memory usage.

[0071] Optionally, the first influence value is used to quantify the GPU memory usage of the data stored by the optimizer (such as gradients, momentum, etc.). Different optimizer choices will result in different first influence values. The second influence value is used to quantify the GPU memory usage of the intermediate feature maps output by any network layer in the target neural network, and is used to evaluate the dynamic memory requirements during task execution.

[0072] Optionally, the task type includes training tasks and inference tasks. Since the intermediate feature map storage requirements of the two are different during the execution process, the task type directly affects the determination of the second influence value.

[0073] Optionally, the intelligent scheduling system first determines the type of optimizer used in the target training task, such as SGD or Adam. Then, it calculates the first impact value on GPU memory based on the optimizer type. For example, SGD typically needs to store gradient values ​​equivalent to the number of parameters, while the Adam optimizer needs to additionally store first and second-order momentum, thus occupying four times the memory space of the number of parameters. The system quantifies the optimizer's contribution to memory usage through this calculation.

[0074] Optionally, the intelligent scheduling system determines the second influence value based on the type of the target training task (training or inference). Training tasks require saving intermediate feature maps during execution, so their second influence value (intermediate feature map memory usage) is typically twice the feature map size. Inference tasks, on the other hand, do not require saving these intermediate results, so their second influence value is relatively low, typically one time the feature map size. This difference reflects the specific impact of task type on memory requirements.

[0075] Optionally, the intelligent scheduling system further calculates the total GPU memory requirement for each computing node when executing the target training task, based on the target information of each of the M network layers, the first influence value of the optimizer, the second influence value of the task type, and the batch size. This includes the memory usage of model parameters, optimizer data, and intermediate feature maps, and takes into account the different impacts of data parallelism and tensor parallelism on memory requirements in distributed training. Finally, the system obtains N third values, which are the predicted memory requirements for the N computing nodes. Through the above steps, the intelligent scheduling system performs memory requirement prediction on all N computing nodes, obtaining N third values ​​that reflect the memory usage of different computing nodes when executing deep learning training tasks.

[0076] As described above, the intelligent scheduling system can precisely predict the memory requirements of the target training task on each computing node based on optimizer type, task type, network layer parameter information, and batch size. This avoids over-allocation or under-allocation of memory resources, ensuring successful task execution. Furthermore, by accurately predicting this third factor, the intelligent scheduling system can dynamically adjust resource allocation strategies according to the memory requirements of the training task, selecting the most suitable computing node for each task, reducing the risk of memory overflow, and improving GPU memory utilization efficiency. In addition, accurate prediction and reasonable allocation of memory requirements significantly improve the execution reliability of deep learning tasks while reducing unnecessary waiting time and resource waste.

[0077] In one optional embodiment, the intelligent scheduling system filters the N computing nodes based on the N third values ​​corresponding to the N computing nodes in the N performance results and network environment information to obtain S first computing nodes. Then, it calculates the fourth value of each first computing node based on the first value, the second value, and the average utilization rate of the graphics processor, to obtain S fourth values. The fourth values ​​are used to characterize the total time consumption of each first computing node in executing the target training task. Finally, the first computing node corresponding to the fourth value among the S fourth values ​​that is less than a preset threshold is taken as the target node.

[0078] Optionally, the intelligent scheduling system, based on the memory requirements of the N computing nodes (the third value) and network environment information, selects computing nodes that can meet the memory requirements of the target training task and have superior network conditions, resulting in S first computing nodes. The system then further evaluates these S first computing nodes. For each first computing node, the intelligent scheduling system combines the first value, the second value, and the current average utilization rate of the graphics processor to calculate the total expected time consumption for executing the target training task, i.e., the fourth value. This value reflects the total execution time of the task on a specific node, including computation time, communication time, and waiting time.

[0079] Optionally, the system sets a preset threshold to limit the maximum time consumption of task execution. The first computing node corresponding to the S fourth values ​​that are lower than or equal to the preset threshold is selected as the final target node. These nodes are considered to be the most suitable for performing the target training task because they can complete the task in a reasonable time while maintaining efficient resource utilization.

[0080] As described above, the intelligent scheduling system, through initial filtering based on memory and network conditions, and comprehensive evaluation of computational resource consumption, data transfer time, and GPU utilization, ensures efficient resource utilization and significantly improves the execution efficiency of deep learning tasks. The introduction of preset thresholds allows the system to accurately select computing nodes capable of completing tasks with optimal time consumption, avoiding resource waste and excessively long task execution times. Through a series of screening and evaluation processes, the system ensures that deep learning training tasks are intelligently allocated to the most suitable computing nodes, reducing task completion time and improving the efficiency of computing resource utilization. This provides users with faster and more stable deep learning training services while optimizing the overall operational performance of the cluster.

[0081] In one optional embodiment, the intelligent scheduling system detects whether the third value corresponding to each computing node in the N performance results is greater than or equal to the video memory capacity corresponding to that computing node in the network environment information. The computing nodes whose third values ​​are greater than or equal to the video memory capacity corresponding to that computing node in the network environment information are filtered out, and the computing nodes whose third values ​​are less than the video memory capacity corresponding to that computing node in the network environment information are selected as the first computing nodes, thus obtaining S first computing nodes.

[0082] Optionally, video memory capacity refers to the total video memory of the GPU in the computing node, which is used to store model parameters, intermediate calculation results, etc.

[0083] Optionally, the intelligent scheduling system first obtains N performance results generated by the resource requirement assessment module. Each result includes the synchronization communication time cost, computation time cost, and GPU memory usage requirement (third value) of the target training task on different computing nodes. The system checks the third value in each of the N performance results, i.e., the GPU memory space required by each computing node to execute the deep learning task, and compares it with the memory capacity of the same computing node recorded in the network environment information. For computing nodes whose third value (memory requirement) is greater than or equal to the memory capacity in the network environment information, the intelligent scheduling system will exclude them from the candidate list to ensure that the system will not assign tasks to nodes with insufficient memory resources, avoiding task failure due to memory overflow. After filtering, the system retains computing nodes whose third value (memory requirement) is less than the memory capacity in the network environment information, i.e., the GPU memory capacity of these nodes is sufficient to support the execution of the deep learning task, and marks these nodes as first computing nodes, for a total of S first computing nodes.

[0084] Optionally, the intelligent scheduling system, through fine-grained analysis of user deep learning tasks, is equivalent to constructing a time-priority task scheduling framework. This enables online prediction of the computing resource requirements of different deep learning tasks and the changes in task performance under different placement strategies, thereby improving the task execution efficiency and cluster resource utilization of the deep learning R&D platform.

[0085] As described above, the intelligent scheduling system can automatically compare the memory requirements of a task with the memory capacity of the computing nodes, ensuring that tasks are assigned to nodes with sufficient memory resources for execution. This avoids task failures or delays caused by insufficient memory, improving the accuracy and efficiency of resource scheduling. By filtering computing nodes with insufficient memory capacity, the system can select the most suitable node for executing deep learning tasks, i.e., the first computing node. This not only reduces invalid options during scheduling but also optimizes the task execution environment, avoiding resource waste. Furthermore, ensuring that the memory resources of computing nodes match the task requirements improves the execution efficiency of deep learning tasks, reduces waiting time during task execution, and improves the overall resource utilization and throughput of the system, providing strong support for the rapid completion of deep learning tasks.

[0086] In one alternative embodiment, Figure 2 This is a structural diagram of an optional task scheduling method according to an embodiment of this application, such as... Figure 2 As shown, it includes: a training task management module, a GPU cluster management module, a training task resource requirement assessment module, and a training task request scheduling module. The training task management module is used to calculate task requests, including model structure, algorithm hyperparameters, input datasets, etc. Specifically, it is mainly responsible for managing the uploading, parsing, and model information construction of deep learning training tasks. It accepts deep learning training task requests uploaded by users through an API interface, constructs a user task request information model based on the task description file (such as a YAML file), and uploads it to the training task resource requirement assessment module. The user task request information is modeled as follows: ,in, This represents the model used in the deep learning training task. These represent the batch size (B), number of iterations (I), neural network structure (D), and parallelism method, respectively. ); This represents the amount of data in the user task input dataset. The standardization of user-submitted tasks and their descriptions is crucial to the system's scheduling efficiency and accuracy. The model used for the task can be selected from packets within an AI framework, or users can customize it, but it must conform to the AI ​​framework specifications and containerized Docker packaging standards to facilitate the system's rapid parsing and processing of the model, reducing scheduling failures or resource waste caused by formatting errors or unclear descriptions.

[0087] The GPU cluster management module is used to acquire system monitoring metrics, such as graphics card configuration, video memory, and network bandwidth. Specifically, it is primarily responsible for monitoring and managing cluster resources. By deploying monitoring software, it enables real-time detection of the resources and link status of each node, providing computational and network awareness parameters for the algorithm. The monitoring software is mainly responsible for metric collection and data persistence, and supports PromQL for flexible metric querying. In practical applications, different types of Exporter scraping addresses and related parameters need to be added to the monitoring software configuration file. For example, deploying `node_exporter` to monitor basic server metrics (CPU, memory, network interfaces, etc.) and deploying `snmp_exporter` to collect network device metrics exposed via SNMP (Simple Network Management Protocol), such as bandwidth utilization and latency of routers and switches. Because distributed training involves high-performance computing and large-scale parameter transmission between GPUs, the GPU cluster management module models the GPU-accelerated node resource information managed by the cluster as... The data is uploaded to the deep learning task resource requirement assessment module and the deep learning task request scheduling module. This indicates resource information for GPU-accelerated nodes. These represent the number of GPUs and the floating-point computing performance on the i-th computing node, respectively; This represents the amount of video memory that can be allocated to each GPU in the current computing node i, which is the actual remaining video memory minus the video memory reserved for other tasks. This represents the PCL-E bus bandwidth within compute node i. This indicates the current GPU utilization rate.

[0088] The training task resource requirement assessment module is used to analyze task requirements, including computational load and memory usage. Specifically, it is responsible for estimating the computational and storage resource requirements of the deep learning training task uploaded by the user based on the structure and hyperparameters of the neural network model and the real-time status information of the computing nodes in the cluster, and inputting the calculation results into the training task request scheduling module.

[0089] The training task request scheduling module will filter out nodes with insufficient video memory resources based on the resource requirements of the training task and the real-time status information of the cluster. Then, it will further execute a scheduling algorithm based on minimizing the model training time to determine the target node, and at the same time complete the activation of computing power and network resources.

[0090] In one optional embodiment, the PyTorch framework is used as the deep learning training platform. First, the training task resource requirement assessment module calls the `torch.summary` API command provided in PyTorch's extension library, inputting the task model structure and parameter configuration to obtain basic information about the model structure. The function call format is: `torch.summary(model, input_size)`, and the output is the number of parameters for each layer of the model. Size of intermediate feature maps The computational cost of one forward propagation of the model Memory requirements for model parameters And the memory requirements for one forward propagation of the model. ,in, This indicates which layer in the model it belongs to. These represent the number of channels, height, and width of the intermediate feature map, respectively. Based on the output of the extended torch.summary function, the total GPU memory requirements for executing user tasks under different placement strategies are predicted, along with the computational overhead and communication overhead introduced by multiple GPUs due to synchronous updates. The specific calculation method is as follows:

[0091] 1) Computational overhead of task requests:

[0092] Deep learning models primarily consist of convolutional and fully connected layers. Other layers involve only simple floating-point calculations or no calculations at all, and their computational cost is negligible compared to convolutional and fully connected layers. For convolutional layers, convolution is implemented using convolutional kernels. Assume... This is the number of input channels. , and It is the height, width, and number of channels of the output feature map. This is the width of the convolution kernel (assuming the kernel is symmetrical). The computational cost of a feature point in the output feature map should include the number of multiplications and additions in the convolution operation. (Including bias terms), and the size of the output feature map is Therefore, the total computational cost of a convolutional layer is For a fully connected layer, assuming its input neuron count is... The number of output neurons is A fully connected layer can also be viewed as a convolutional layer with a kernel size of... A special case of convolutional layers, then its number of input channels Equal to the number of input neurons The number of convolution kernels is equal to the number of output neurons. .

[0093] Therefore, the total computational cost of a single-layer neural network is shown in formula (1):

[0094] (1)

[0095] in, This is the batch size for the training task; To account for the differences in computational cost due to different task types, for training tasks... For reasoning tasks .

[0096] The total computational cost of the entire neural network model is equal to the sum of the computational costs of each layer multiplied by the total number of iterations. With distributed parallelism, each GPU will share the total computational load equally. Let i represent the computational cost of one forward propagation of the model, then the computational cost per GPU in computing node i. As shown in formula (2):

[0097] (2)

[0098] in, This represents the computational cost of performing a forward propagation at the i-th computation node. This indicates which layer in the model it belongs to. These represent the number of GPUs and the floating-point computing performance on the i-th computing node, respectively.

[0099] 2) Synchronous communication overhead for task requests:

[0100] In distributed parallel computing based on the Ring-AllReduce architecture, GPU accelerators are connected in a ring topology, continuously sending and receiving data to aggregate gradients and update model parameters locally. The synchronous communication overhead of distributed tasks under the Ring-AllReduce mechanism is divided into intra-layer and inter-layer communication overhead, which is related to the parallelism method. These will be analyzed in turn below:

[0101] Intra-layer communication overhead is generated by performing a single Ring-AllReduce operation on a single layer of the model, and is used for model gradient aggregation. The total communication overhead per card is... ,in This represents the number of GPUs.

[0102] For data parallelism, , For this layer of the model ( ) number of parameters;

[0103] For tensor parallelism , This is the product of the dimensions of the output results corresponding to a single sample.

[0104] Therefore, the intra-layer communication overhead of computing node i corresponding to data parallelism The calculation formula is shown in formula (3):

[0105] (3)

[0106] in, Representation Model The number of parameters of the layer (i.e.) ), This represents the total number of iterations.

[0107] Intra-layer communication overhead for computation node i in tensor parallelism The calculation formula is shown in formula (4):

[0108] (4)

[0109] in, , The size of the feature map ( The number of channels in the feature map (representing the height and width of the feature map), i.e. and .

[0110] Inter-layer communication overhead arises from data processing during the transfer of intermediate computation results between layers. Data parallelism does not involve inter-layer communication costs, while tensor parallelism corresponds to a single-card communication overhead of [missing information]. ,in Therefore, the inter-layer communication overhead corresponding to tensor parallelism... The calculation formula is shown in formula (5):

[0111] (5)

[0112] Summing the communication overhead within and between layers yields the synchronization communication overhead for data parallelism and tensor parallelism:

[0113] The communication overhead of computing node i corresponding to data parallelism is shown in formula (6):

[0114] (6)

[0115] The communication overhead of computing node i corresponding to tensor parallelism is shown in equation (7):

[0116] (7)

[0117] 3) Memory usage requirements of task requests:

[0118] The GPU memory used by neural network models primarily stores the model's own weight parameters, gradient values, and intermediate output results. Common layer types that include weight parameters include convolutional layers, fully connected layers, batchNorm layers, and embedding layers. Non-linear activation layers, pooling layers, and Dropout layers only have hyperparameters and no weight parameters, therefore they do not occupy GPU memory. Assume the total number of parameters in a convolutional layer is... The total number of convolution weights is The total number of bias parameters is The number of input channels in this layer is The kernel width is Quantity is (The number of convolutional kernels is equal to the number of output channels of this layer) Based on the computational characteristics of convolutional layers, the total number of parameters of the entire convolutional layer can be calculated as shown in formulas (8)-(10):

[0119] (8)

[0120] (9)

[0121] (10)

[0122] Similarly, for a fully connected layer, the number of output neurons is A fully connected layer can also be viewed as a convolutional layer with a kernel size of... A special case of a convolutional layer. Its input channel count... Equal to the number of input neurons The number of convolution kernels is equal to the number of output neurons. The total number of parameters for a fully connected layer is shown in formula (11):

[0123] (11)

[0124] Because GPU memory also needs to store gradients and momentum from the model computation process, the choice of optimizer also has a certain impact on GPU memory usage. For example, the typical stochastic gradient descent algorithm SGD needs to store the same number of gradients as the number of weight parameters, and the total GPU memory usage is twice the memory space occupied by the parameters. Momentum-SGD, on the other hand, has a total GPU memory usage of three times the size of the weight parameters. For more complex optimizers, such as the Adam optimizer, the total GPU memory usage is four times the size of the weight parameters.

[0125] In the computation process of deep learning tasks, for each input data sample, the forward inference process often requires caching intermediate computation results between layers, i.e., intermediate feature maps. The size of the intermediate output tensor stored by the convolutional layer is equal to the size of the output feature map of that layer. (H is the height of the output feature map of this layer, W is the width, (This refers to the number of channels). The output tensor of a fully connected layer is a one-dimensional vector, i.e. Number of output channels Equal to the number of output neurons Simply substitute the values ​​into the above formula to calculate.

[0126] make Represents the model weight parameters. Let i represent the memory usage of a single forward propagation of the model. The formula for calculating the total memory usage of a single layer network at node i is shown in formula (12):

[0127] (12)

[0128] in, This indicates the impact of optimizer selection on the memory usage of model parameters. This indicates the impact of different task types on the memory usage of intermediate feature maps, for the training task. For reasoning tasks .and The storage space required to represent different numeric types, when using single-precision floating-point numbers. That is, each single-precision floating-point number occupies 4 bytes of video memory. This represents the weight parameters of the i-th computation node in the model. This represents the amount of video memory used when the model performs one forward propagation at the i-th computing node.

[0129] In this data-parallel strategy, a complete copy of the model is loaded onto each GPU, while the training data is divided into mini-batches and distributed to each GPU for individual processing. Since each GPU stores a complete copy of the model, the memory usage of the weight parameters on each GPU is significantly reduced. The space occupied by the intermediate feature map remains unchanged. The data is evenly distributed; in the tensor parallel approach, the entire dataset is loaded, but the model itself is divided into multiple parts, each running on a different GPU. Therefore, the space occupied by the weight parameters on each GPU is significant. The space is divided equally, while the space occupied by the intermediate feature map is... Same as the original model.

[0130] Therefore, the memory usage requirement of computing node i under the data parallel mode is as shown in formula (13):

[0131] (13)

[0132] The memory usage requirement of computing node i under tensor parallelism is shown in formula (14):

[0133] (14)

[0134] The resource requirements assessment for the task can be summarized as follows: 'fun' represents the algorithm for evaluating the resource requirements of a training task. The inputs are user request information (service) and node resource information (node). The outputs are the synchronization communication time cost, computation time cost, and the model memory space requirement on a single GPU under the placement strategy corresponding to the node.

[0135] The training task request scheduling module is mainly divided into two stages: screening and scheduling. The screening stage filters out nodes with insufficient GPU memory or hardware failures based on the resource requirements of the training task and cluster status information, ensuring that the selected nodes have enough GPU memory to support the model's computational needs. Once a node is confirmed to meet the GPU memory requirements, the scheduling module calculates the total time cost of executing the user task on that node (node ​​i). As shown in formula (15):

[0136] (15)

[0137] in, The startup time of worker nodes is also negligible because starting a container / virtual machine takes very little time (within a few seconds). This is due to GPU utilization. This refers to the proportion of time the GPU actually uses to execute computational tasks. Furthermore, computation and communication within a GPU node cannot be parallelized; the next layer's computation can only begin after the synchronization communication at that layer is complete. Therefore, the total time overhead is the sum of these two. After traversing all available nodes, the nodes are sorted in ascending order based on their time overhead. The node with the shortest task execution time is selected for task scheduling, and the computing power scheduling platform completes the activation of computing resources and network connectivity.

[0138] In one alternative embodiment, Figure 3 This is a schematic diagram of an optional task scheduling method according to an embodiment of this application, such as... Figure 3 As shown, the user first submits a deep learning training task request (AI training task request) through the API interface. After receiving the user's task request, the training task management module parses the task description file submitted by the user, extracts the key requirement indicators of the task, and constructs a user task request information model based on the task description file. This includes uploading batch size, number of iterations, neural network structure, parallelization method, and the amount of input dataset to the training task resource requirement assessment module. Next, the GPU cluster management module is responsible for real-time monitoring and management of cluster resources. When a user task request arrives, the cluster management module will collect node resource status and network status monitoring information. The data is uploaded to the training task resource requirement assessment module. Then, the scheduling engine receives the user task request information model uploaded by the training task management module, obtains the cluster's computing metrics and real-time node status information from the GPU cluster management module, and executes the resource requirement assessment algorithm. The system estimates resource requirements by considering the synchronization communication time overhead, computation time overhead, and the memory space requirements of the model on a single GPU under different placement strategies for the user's computational task. Based on the resource requirement assessment results and resource metrics uploaded by the GPU cluster management module, the training task request scheduling module filters out nodes that do not meet resource constraints, calculates the expected total execution time for the remaining selectable nodes, and selects the node with the shortest total execution time as the target node. Once the target node is obtained, the cluster master node distributes the task's configuration file and other data to the target node and routes the task request to the selected target node, where the deep learning training task is executed. During task execution, the GPU cluster management module monitors the task execution status in real time, collects task execution data (such as resource usage and execution time), and provides feedback on the task execution status and results to the user. Nodes are represented as Node1...Nodex.

[0139] This application also provides a task scheduling device. It should be noted that the task scheduling device of this application can be used to execute the task scheduling method provided in this application. The task scheduling device provided in this application is described below.

[0140] According to an embodiment of this application, an apparatus for implementing the above-described task scheduling method is also provided. Figure 4 This is a schematic diagram of an optional task scheduling device according to an embodiment of this application, such as... Figure 4 As shown, it includes: a construction unit 401, a prediction unit 402, a determination unit 403, and an execution unit 404.

[0141] Optionally, the construction unit 401 is used to construct a task model based on the target training task, wherein the task model is used to characterize the attributes and resource requirements of the target training task; the prediction unit 402 is used to predict the performance results of the target training task when executed on N computing nodes based on the task model, and obtain N performance results, wherein N is an integer greater than 1, and the performance results include the execution efficiency and resource utilization of the target training task on the computing nodes; the determination unit 403 is used to filter the N computing nodes according to the N performance results and network environment information, obtain S first computing nodes, and determine the target node from the S first computing nodes, wherein S is an integer greater than or equal to 1 and less than or equal to N, the first computing nodes are used to characterize computing nodes that meet the network environment information, and the target node is used to characterize first computing nodes that meet preset requirements, wherein the preset requirements are used to constrain the execution time of the computing node for the target training task; and the execution unit 404 is used to execute the target training task through the target node.

[0142] Optionally, the prediction unit 402 includes: a first prediction subunit, a second prediction subunit, a third prediction subunit, and a first determination subunit. The first prediction subunit is used to predict a first value of the target training task on each computing node based on the task model, obtaining N first values, where the first values ​​characterize the computing resource consumption of the graphics processor in each computing node when executing the target training task; the second prediction subunit is used to predict a second value of the target training task on each computing node based on the task model, obtaining N second values, where the second values ​​characterize the transmission time generated by data exchange between computing nodes; the third prediction subunit is used to predict a third value of the target training task on each computing node based on the task model, obtaining N third values, where the third values ​​characterize the video memory space required by the graphics processor in each computing node when executing the target training task; the first determination subunit is used to determine N performance results based on the N first values, N second values, and N third values.

[0143] Optionally, the first prediction subunit includes: a first determining module, a second determining module, and a third determining module. The first determining module is used to determine the computational cost of each of the M network layers based on the target information of each network layer and the batch size, when there are M network layers in the target neural network, thus obtaining M computational costs. The target information is used to characterize the parameter information required by each network layer when performing the target training task. The second determining module is used to determine the computational cost of the target neural network based on the M computational costs and the number of iterations. The third determining module is used to determine a first value corresponding to each computing node based on the computational cost of the target neural network and the number of graphics processors corresponding to each computing node, thus obtaining N first values.

[0144] Optionally, the second prediction subunit includes a fourth determining module and a fifth determining module. The fourth determining module is used to determine the second value corresponding to each computing node based on the number of graphics processors corresponding to each computing node, the number of parameters in each of the M network layers, and the number of iterations when the parallelism mode is data parallelism, thus obtaining N second values. The fifth determining module is used to determine the second value corresponding to each computing node based on the number of graphics processors corresponding to each computing node, the feature tensors output by each of the M network layers, the batch size, and the number of iterations when the parallelism mode is tensor parallelism, thus obtaining N second values.

[0145] Optionally, the third prediction subunit includes: a sixth determining module, a seventh determining module, an eighth determining module, and a ninth determining module. The sixth determining module is used to determine the optimizer, wherein the optimizer is used to adjust and store the model parameters of the target neural network; the seventh determining module is used to determine the first influence value of the optimizer on the GPU memory, wherein the first influence value is used to quantify the GPU memory space occupied by the optimizer storing data; the eighth determining module is used to determine a second influence value based on the task type of the target training task, wherein the second influence value is used to quantify the GPU memory usage of the intermediate feature map output by any network layer in the target neural network; the ninth determining module is used to determine a third value corresponding to each computing node based on the target information of any network layer among the M network layers, the first influence value, the second influence value, and the batch size, obtaining N third values.

[0146] Optionally, the determining unit 403 includes: a first filtering subunit, a first calculation subunit, and a second determining subunit. The first filtering subunit is used to filter the N computing nodes based on N third values ​​corresponding to the N computing nodes in the N performance results and network environment information, obtaining S first computing nodes; the first calculation subunit is used to calculate a fourth value for each of the S first computing nodes based on a first value, a second value, and the average utilization rate of the graphics processor, obtaining S fourth values, where the fourth values ​​characterize the total time consumed by each first computing node in executing the target training task; the second determining subunit is used to select the first computing nodes corresponding to the fourth values ​​less than a preset threshold from the S fourth values ​​as target nodes.

[0147] Optionally, the determining unit 403 includes: a first detection subunit, a second filtering subunit, and a third determining subunit. The first detection subunit is used to detect whether the third value corresponding to each computing node in the N performance results is greater than or equal to the video memory capacity corresponding to that computing node in the network environment information; the second filtering subunit is used to filter out computing nodes whose N third values ​​corresponding to the N computing nodes are greater than or equal to the video memory capacity corresponding to that computing node in the network environment information; the third determining subunit is used to select computing nodes whose N third values ​​corresponding to the N computing nodes are less than the video memory capacity corresponding to that computing node in the network environment information as first computing nodes, thus obtaining S first computing nodes.

[0148] According to another aspect of this application, a computer-readable storage medium is also provided, which includes a stored executable program, wherein the executable program controls the device where the computer-readable storage medium is located to perform the above-described task scheduling method when it runs.

[0149] According to another aspect of this application, an electronic device is also provided, including one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the task scheduling method described above.

[0150] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0151] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0153] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0155] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0156] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A task scheduling method, characterized in that, include: A task model is constructed based on the target training task. The task model is used to characterize the attributes and resource requirements of the target training task. The task model includes the target neural network for training the target training task, the number of iterations, the batch size, and the parallel mode. The batch size is used to characterize the amount of data processed by the target neural network in each round of training, and the parallel mode is used to characterize the strategy of resource allocation and computation task division during the training of the target neural network. Based on the task model, predict the performance results of the target training task when executed on N computing nodes to obtain N performance results, where N is an integer greater than 1. The performance results include the execution efficiency and resource utilization of the target training task on the computing nodes, and each computing node includes at least a graphics processor node. Specifically, based on the task model, the performance results of the target training task executed on N computing nodes are predicted to obtain N performance results, including: predicting a first value, a second value, and a third value of the target training task on each computing node based on the task model, wherein the first value is used to characterize the computing resource consumption of the graphics processor in each computing node when executing the target training task, the second value is used to characterize the transmission time generated by data exchange between computing nodes, and the third value is used to characterize the video memory space required by the graphics processor in each computing node when executing the target training task; and determining the N performance results based on the first value, the second value, and the third value of the target training task on each computing node. Based on the N performance results and network environment information, the N computing nodes are filtered to obtain S first computing nodes, and a target node is determined from the S first computing nodes, where S is an integer greater than or equal to 1 and less than or equal to N. The first computing nodes are used to represent computing nodes that meet the network environment information, and the target node is used to represent the first computing nodes that meet preset requirements, where the preset requirements are used to constrain the duration for the computing nodes to execute the target training task. The network environment information includes at least the utilization and computing performance information of each computing node, the network bandwidth information between the N computing nodes, and the memory capacity information of each computing node. The target training task is executed through the target node.

2. The task scheduling method according to claim 1, characterized in that, Based on the task model, predict the first value of the target training task on each computing node to obtain N first values, including: When there are M network layers in the target neural network, the computational cost of each network layer is determined according to the target information of each of the M network layers and the batch size, resulting in M ​​computational costs. The target information is used to characterize the parameter information required by each network layer when performing the target training task. The computational complexity of the target neural network is determined based on the M computational quantities and the number of iterations. Based on the computational complexity of the target neural network and the number of graphics processors corresponding to each computing node, a first value corresponding to each computing node is determined, resulting in N first values.

3. The task scheduling method according to claim 2, characterized in that, Based on the task model, predict the second value of the target training task at each computing node to obtain N second values, including: When the parallel mode is data parallel, the second value corresponding to each computing node is determined according to the number of graphics processors corresponding to each computing node, the number of parameters of each of the M network layers, and the number of iterations, so as to obtain N second values; When the parallel mode is tensor parallelism, the second value corresponding to each computing node is determined according to the number of graphics processors corresponding to each computing node, the feature tensor output by each of the M network layers, the batch size, and the number of iterations, so as to obtain N second values.

4. The task scheduling method according to claim 2, characterized in that, Based on the task model, predict the third value of the target training task at each of the computing nodes to obtain N third values, including: Determine an optimizer, wherein the optimizer is used to adjust and store the model parameters of the target neural network; Determine a first influence value of the optimizer on the video memory in the graphics processor, wherein the first influence value is used to quantify the video memory space occupied by the optimizer storing data; A second influence value is determined based on the task type of the target training task, wherein the second influence value is used to quantify the occupancy of the intermediate feature map output by any one of the network layers in the target neural network in the graphics processor's memory. Based on the target information of any one of the M network layers, the first influence value, the second influence value, and the batch size, determine the third value corresponding to each computing node to obtain N third values.

5. The task scheduling method according to claim 1, characterized in that, Based on the N performance results and network environment information, the N computing nodes are filtered to obtain S first computing nodes, and the target node is determined from the S first computing nodes, including: Based on the N third values ​​corresponding to the N computing nodes in the N performance results and the network environment information, the N computing nodes are filtered to obtain S first computing nodes; Based on the first value, the second value, and the average utilization of the graphics processor for each of the S first computing nodes, a fourth value is calculated for each of the first computing nodes, resulting in S fourth values. The fourth values ​​are used to characterize the total time consumed by each of the first computing nodes in executing the target training task. The first calculation node corresponding to the fourth value that is less than a preset threshold among the S fourth values ​​is taken as the target node.

6. The task scheduling method according to claim 5, characterized in that, Based on the N third values ​​corresponding to the N computing nodes in the N performance results and the network environment information, the N computing nodes are filtered to obtain S first computing nodes, including: Detect whether the third value corresponding to each computing node in the N performance results is greater than or equal to the video memory capacity corresponding to that computing node in the network environment information; Filter out computing nodes whose video memory capacity is greater than or equal to that of the computing node in the network environment information from among the N third values ​​corresponding to the N computing nodes; The computing node whose third value is less than the video memory capacity corresponding to the computing node in the network environment information is selected as the first computing node, thus obtaining S first computing nodes.

7. A task scheduling device, characterized in that, include: A construction unit is used to construct a task model based on a target training task. The task model is used to characterize the attributes and resource requirements of the target training task. The task model includes a target neural network for training the target training task, the number of iterations, the batch size, and the parallel mode. The batch size is used to characterize the amount of data processed by the target neural network in each round of training, and the parallel mode is used to characterize the strategy of resource allocation and computation task division during the training of the target neural network. The prediction unit is used to predict the performance results of the target training task when it is executed on N computing nodes based on the task model, and obtain N performance results, where N is an integer greater than 1. The performance results include the execution efficiency and resource utilization of the target training task on the computing nodes, and each computing node includes at least a graphics processor node. The prediction unit further includes: predicting a first value, a second value, and a third value of the target training task on each computing node based on the task model, wherein the first value is used to characterize the computing resource consumption of the graphics processor in each computing node when executing the target training task, the second value is used to characterize the transmission time generated by data exchange between computing nodes, and the third value is used to characterize the video memory space required by the graphics processor in each computing node when executing the target training task; and determining N performance results based on the first value, the second value, and the third value of the target training task on each computing node. A determining unit is configured to filter N computing nodes based on N performance results and network environment information to obtain S first computing nodes, and determine a target node from the S first computing nodes, wherein S is an integer greater than or equal to 1 and less than or equal to N, the first computing nodes are used to characterize computing nodes that satisfy the network environment information, and the target node is used to characterize the first computing nodes that satisfy preset requirements, wherein the preset requirements are used to constrain the duration for the computing nodes to execute the target training task, and the network environment information includes at least the utilization rate and computing performance information of each computing node, the network bandwidth information among the N computing nodes, and the video memory capacity information of each computing node; An execution unit is used to execute the target training task through the target node.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device in which the computer-readable storage medium is located performs the task scheduling method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the task scheduling method according to any one of claims 1 to 6.

Images