Traffic scheduling method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a traffic scheduling method and apparatus, an electronic device, a storage medium and a program product. The method comprises the steps of determining a plurality of to-be-trained tasks; determining the shortest completion time for training any one of the to-be-trained tasks, and performing queue allocation on any one of the to-be-trained tasks according to the shortest completion time; in response to determining that the queue of the to-be-trained tasks is allocated to the time-sensitive queue, determining a plurality of to-be-trained sub-tasks in the to-be-trained tasks, performing priority ranking on the plurality of to-be-trained sub-tasks, and determining the priority of any one of the to-be-trained sub-tasks; and carrying out bandwidth allocation on the to-be-trained task according to the priority to complete traffic scheduling. According to the method, the to-be-trained tasks are subjected to priority ranking, bandwidth allocation is performed on the to-be-trained tasks according to the priority ranking, and flow scheduling is completed, so that the overdue risk of time-sensitive tasks is reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a traffic scheduling method, device, electronic device, storage medium and program product. Background Art

[0002] In recent years, deep learning technology has been widely used and rapidly developed in many fields such as computer vision and speech recognition.

[0003] However, the applicant found that when conducting deep learning training in a multi-tenant AI cluster, it was impossible to allocate network resources for training tasks with different requirements in a timely manner, resulting in tasks with strict training deadline requirements not being trained in a timely manner, resulting in task overdue. Summary of the invention

[0004] In view of this, the purpose of the present application is to propose a traffic scheduling method, device, electronic device, storage medium and program product.

[0005] Based on the above purpose, the present application provides a traffic scheduling method for scheduling traffic resources in multi-tenant cluster training, including:

[0006] Determine a number of tasks to be trained;

[0007] Determine the shortest completion time for training any of the tasks to be trained, and allocate a queue for any of the training tasks according to the shortest completion time;

[0008] In response to determining that the queue of the task to be trained is allocated to the time-sensitive queue, determining a number of subtasks to be trained in the task to be trained, prioritizing the subtasks to be trained, and determining a priority of any subtask to be trained;

[0009] Bandwidth is allocated to the task to be trained according to the priority level to complete traffic scheduling.

[0010] Optionally, the shortest completion time is the training time required when the task to be trained exclusively uses the network bandwidth;

[0011] The determining of the required training duration of any of the tasks to be trained comprises:

[0012] According to the formula shown below, the required training time T is determined n :

[0013] T n =(T comp +T comm ))*N iter

[0014] Among them, Tcomp is the computing time, Tcomm is the communication time, and Niter is the number of iterations of the task.

[0015] Optionally, allocating a queue for any of the training tasks according to the shortest completion time includes:

[0016] Determine a preset deadline of any of the training tasks, compare the preset deadline of any of the training tasks with the shortest completion time, and allocate a queue for any of the training tasks according to the comparison result;

[0017] In response to determining that the shortest completion time is less than or equal to the preset deadline, assigning the training task to the time-sensitive queue;

[0018] In response to determining that the shortest completion time is greater than the preset deadline, the training task is assigned to a time-insensitive queue.

[0019] Optionally, the prioritizing the plurality of subtasks to be trained and determining the priority of any subtask to be trained includes:

[0020] Determine an occupied node ID of any of the subtasks to be trained, and determine a competition factor of any of the subtasks to be trained according to the occupied node ID;

[0021] Priority ranking is performed on the plurality of subtasks to be trained according to the competition factor to determine the priority of any subtask to be trained; wherein, the larger the competition factor, the lower the priority, and the smaller the competition factor, the higher the priority.

[0022] Optionally, determining the competition factor of any of the subtasks to be trained according to the occupied node ID includes:

[0023] Determine, according to the occupied node ID, the number of nodes for which port contention exists between the target subtask to be trained and a plurality of subtasks to be trained other than the target subtask to be trained;

[0024] The competition factor is determined according to the number of nodes with port competition; wherein, the fewer the number of nodes with port competition, the greater the competition factor, and the more the number of nodes with port competition, the smaller the competition factor.

[0025] Optionally, allocating bandwidth for the task to be trained according to the priority to complete traffic scheduling includes:

[0026] Determine the full bandwidth quota and the reserved bandwidth quota;

[0027] Determine the bandwidth quota to be allocated according to the full bandwidth quota and the reserved bandwidth quota; wherein the ratio of the reserved bandwidth quota to the bandwidth quota to be allocated is 1:9;

[0028] Bandwidth is allocated for the task to be trained according to the priority and the bandwidth quota to be allocated, thereby completing traffic scheduling.

[0029] Based on the same inventive concept, the embodiment of the present application further provides a traffic scheduling device, including:

[0030] A determination module, configured to determine a number of tasks to be trained;

[0031] A queue allocation module is configured to determine the shortest completion time of training any of the tasks to be trained, and allocate a queue to any of the training tasks according to the shortest completion time;

[0032] a sorting module, configured to, in response to determining that the queue of the task to be trained is allocated to the time-sensitive queue, determine a number of subtasks to be trained in the task to be trained, prioritize the subtasks to be trained, and determine the priority of any subtask to be trained;

[0033] The bandwidth allocation module is configured to allocate bandwidth for the task to be trained according to the priority level to complete traffic scheduling.

[0034] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a traffic scheduling method as described in any one of the above items is implemented.

[0035] Based on the same inventive concept, an embodiment of the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned traffic scheduling methods.

[0036] Based on the same inventive concept, an embodiment of the present application also provides a computer program product, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes any of the above-mentioned traffic scheduling methods.

[0037] From the above description, it can be seen that the traffic scheduling method, device, electronic device, storage medium and program product provided by the present application complete traffic scheduling by prioritizing the tasks to be trained, allocating bandwidth to the tasks to be trained according to the priority ranking, thereby reducing the risk of overdue time-sensitive tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 This is a schematic diagram of the distributed training server architecture of an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a prior art scheduling method according to an embodiment of the present application;

[0041] Figure 3 A flow chart of a traffic scheduling method according to an embodiment of the present application;

[0042] Figure 4 This is a schematic diagram of the Allot architecture of an embodiment of the present application;

[0043] Figure 5 A schematic diagram of a flow scheduling device according to an embodiment of the present application;

[0044] Figure 6 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0046] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0047] In order to facilitate understanding of the technical solutions of the present disclosure, some technical terms involved in the present disclosure are introduced below.

[0048] DML, distributed machine learning training, also known as distributed training, refers to a training method that decomposes the training task into multiple subtasks and performs training in parallel on multiple computing devices.

[0049] In order to make the technical solution of the present disclosure clearer and easier to understand, the traffic scheduling method provided by the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0050] Coflow, collaborative flow, is a group of data flows associated between two groups of servers caused by parallel computing. That is, a group of data flows with common performance goals are brought together to meet specific performance requirements in different scenarios. Among them, coflow adopts a new data flow model and scheduling method. As described in the background technology section, in recent years, deep learning technology has been widely used and rapidly developed in many fields such as computer vision and speech recognition. However, the applicant found that when performing deep learning training through distributed training in a cluster, it is impossible to allocate network resources for training tasks with different requirements in a timely manner, resulting in tasks with strict deadline requirements not being trained in a timely manner, resulting in task overdue.

[0051] Furthermore, the reasons why the applicant's distributed training resulted in tasks with strict deadline requirements not being trained in a timely manner, resulting in task overdues, include: best-effort tasks and tasks that could not be completed within the deadline (i.e., time-insensitive tasks) consumed too much bandwidth resources, resulting in deadline-sensitive tasks being overdue; and link competition between tasks resulting in task overdues.

[0052] Specifically, tasks for deep learning training in a cluster have different latency requirements. One type of task has strict deadline requirements, usually commercial deep learning tasks; the other type of task is a best-effort task without clear deadline requirements, mainly used for model debugging and experiments. In actual training clusters, these two types of tasks often exist at the same time. Therefore, how to effectively allocate network resources for tasks with different user requirements has become an unresolved problem.

[0053] In the existing technology, the traffic scheduling schemes in the current data center cluster are divided into general traffic scheduling and traffic scheduling for DML tasks. General traffic scheduling technology is mainly aimed at traditional applications, scheduling traffic in the cluster with different priorities to optimize the network transmission of the data center. However, this type of scheduling method does not take into account the unique communication characteristics of distributed training, such as the determinism of gradient traffic and the periodicity of communication patterns. Therefore, in machine learning clusters, these general methods are difficult to fully exert their effects.

[0054] Among them, the traffic scheduling of DML tasks can be divided into single-task traffic scheduling (traffic management and allocation for a single task or application in the network) and multi-task traffic scheduling (traffic management and allocation for multiple tasks or applications in the network at the same time). Therefore, for single-task scheduling, the gradient close to the input layer is usually prioritized to achieve overlap between computing and communication, thereby shortening the training time. However, this method does not consider multi-tenant clusters where multiple tasks coexist. Current multi-task traffic scheduling usually adopts principles such as communication interleaving and shortest task priority to achieve priority scheduling, but these schemes mainly focus on shortening the average task time, without considering that some tasks have strict deadline requirements, and it is difficult to fully meet the needs of users in the cluster.

[0055] The applicant of this application has discovered through research that, for applications in distributed training, in order to cope with training tasks of large-scale data sets and complex models, DML breaks through the limitations of single-machine training by distributing tasks to multiple nodes. The most common parallel strategy in DML is data parallelism, which distributes training data to multiple workers (Workers receive data shards or subtasks assigned by the Master (or parameter server) and execute machine learning algorithms locally). Each worker trains independently and performs gradient exchange and aggregation. Among them, the parameter server architecture has been widely used in many machine learning systems, such as TensorFlow and MXNet (both are deep learning frameworks). Figure 1 As shown in the figure, an overview of the parameter server architecture is shown. Each worker stores the complete training model and divides the data set into N parts and distributes them to N workers for training.

[0056] Completing a DML task usually requires multiple iterations, each of which includes the following four steps:

[0057] (1) Local training: Each worker is trained independently based on its data;

[0058] (2) Parameter push: Each worker sends the gradient after a round of training to the parameter server;

[0059] (3) Gradient aggregation: The parameter server aggregates the gradients from multiple workers;

[0060] (4) Parameter pull: The parameter server sends the aggregated gradients back to each worker.

[0061] Furthermore, in a multi-tenant AI cluster, multiple DML tasks are usually run in parallel, and each task has different training deadline requirements. In order to maximize the needs of users in the cluster, it is necessary to ensure that all tasks can complete training within the deadline as much as possible. Figure 2 As shown in the figure, when there are three arriving tasks JOB1, JOB2, and JOB3 competing for the inlet port at the same time, the preset training deadlines are 5s, 7s, and 7s respectively. Among them, JOB1 has two coflows (C1 and C2), C1 has four flows, both of which are 2 in size, and C2 also has four flows, both of which are 2 in size. C2 cannot start before C1 is completed. JOB2 has two coflows (C3 and C4), C3 has two flows of size 2 and 1, and C4 has two flows of size 3 and 4. Similarly, JOB3 contains two coflows (C5 and C6), C5 has two flows of size 2 and 3, and C6 has two flows of size 4 and 3. Among them, each inlet port and outlet port can only transmit one unit per unit time.

[0062] like Figure 2 As shown, the preset training time of JOB1, JOB2, and JOB3 is abnormally arranged from small to large: JOB1>JOB2=JOB3. Therefore, in the prior art, JOB1 with a preset training deadline of 5s is scheduled first, and the scheduling order is: JOB1, JOB2, and JOB3. Then, under this scheduling method, J1 is scheduled first with a completion time of 4s, J2 and J3 are completed in 9s and 10s respectively, and only one task meets the deadline. This is because the priority scheduling of J1 causes J2 and J3 to be unable to allocate network bandwidth in time, thus missing the deadline. The number of tasks competing with J1 is two (J2, J3), and the number of tasks competing with J2 and J3 at the same time is one (J1). Therefore, considering the competition factor of the task, the scheduling order (J2, J3, J1) is adopted. The completion time of J2 and J3 is 5s and 6s respectively, and the completion time of J1 is 10s. It can be seen that two tasks meet the deadline.

[0063] It can be seen that priority scheduling based solely on deadlines is not able to better improve the rate of meeting task deadlines. Therefore, the present invention proposes an online scheduler. The scheduler solves problem one by allocating queues to prevent tasks without deadline requirements or that cannot meet deadlines from occupying too much bandwidth resources. For problem two, the scheduler adopts a competition-aware priority scheduling method to reasonably allocate bandwidth resources, reduce the mutual influence between tasks, and ensure that more tasks are completed on time.

[0064] As described above, in order to solve the problem in the prior art that tasks that cannot be completed within the deadline lead to overdue deadline-sensitive tasks, the embodiments of the present application provide a traffic scheduling method, device, electronic device, storage medium and program product, which prioritizes the tasks to be trained through an online scheduler named Allot, allocates bandwidth for the tasks to be trained according to the priority ranking, and completes traffic scheduling, thereby reducing the risk of overdue time-sensitive tasks. Among them, the Allot scheduler includes a queue allocation part, a calculation sorting part and a bandwidth allocation part. Whenever a new task arrives or a task is completed, the Allot scheduler starts to execute and maintains two priority queues in real time: a deadline-sensitive task queue and a deadline-insensitive task queue, that is, a best-effort task queue.

[0065] It should be noted that the determination of the deadline-sensitive task queue and the deadline-insensitive task queue is done by calculating the shortest completion time of each deadline-sensitive training task and judging whether the shortest time can meet the deadline requirement. Among them, the tasks that can meet the deadline requirement are placed in the deadline-sensitive queue, and all other tasks are placed in the best-effort queue.

[0066] like Figure 3 As shown, the traffic scheduling method includes:

[0067] Step S302, determining a number of tasks to be trained;

[0068] Step S304, determining the shortest completion time of training any of the tasks to be trained, and assigning a queue to any of the training tasks according to the shortest completion time;

[0069] Step S306: in response to determining that the queue of the task to be trained is allocated to the time-sensitive queue, determining a number of subtasks to be trained in the task to be trained, and prioritizing the subtasks to be trained to determine the priority of any subtask to be trained;

[0070] Step S308: Allocate bandwidth for the task to be trained according to the priority to complete traffic scheduling.

[0071] In step S302, the task to be trained is as follows: Figure 4 Job1, Job2, Job3, Job4, Job5 are shown.

[0072] In some embodiments, Figure 4 As shown, the shortest completion time for training any of the tasks to be trained is determined through the queue allocation part, and the training duration in the queue allocation for any of the training tasks according to the shortest completion time is the training duration required when the task to be trained exclusively enjoys the network bandwidth.

[0073] Specifically, since the traffic pattern of distributed training tasks has periodic and deterministic characteristics, we can infer that when the task has exclusive network bandwidth, the time required to complete the task training is T n :

[0074] T n =(T comp +T comm ))*N iter

[0075] Among them, T comp is the calculation time, T comm is the communication time, N iter is the number of task iterations.

[0076] The computation time depends on the GPU's computing power and the time spent on each iteration is constant. I was able to obtain the computation time for each iteration through pre-training. The communication time is given by Figure 1 The time taken to transfer the gradients in the pull and push phases is shown in the figure. The completion time of the slowest flow determines the completion time of the entire Coflow. For example, when the task to be trained JobX includes two data flow sets Coflow (Coflow1, Coflow2), Coflow1 includes two data flow flows (flow1, flow2), and Coflow2 includes two data flow flows (flow3, flow4).

[0077] In some embodiments, since the number of model parameters is fixed, we can easily obtain the amount of data V required to be transmitted for each gradient synchronization. i,j,n,k , we can get (communication time) T comm :

[0078]

[0079] Where i is the inlet port, j is the outlet port, V i,j,n,k The amount of data that needs to be transferred from i to j for the k-th coflow generated for task n, U i is the bandwidth capacity of port i, U j is the bandwidth capacity of port j.

[0080] In some embodiments, based on the above calculation formula, we can infer the shortest completion time of the task and determine whether it can meet the deadline of the task. Tasks that can meet the deadline will be placed in the deadline-sensitive task queue, while tasks that cannot meet the deadline and best-effort tasks are placed in the best-effort task queue.

[0081] Computational sorting: In order to effectively maximize the number of jobs that meet deadline requirements, we propose a heuristic algorithm SDCF, which takes into account the impact of both deadlines and competition factors. The competition factor of task n is defined as the number of tasks that will be affected by task n, that is, the number of tasks that compete with the task for ports and have a lower priority than the task. This module uses SDCF to sort deadline-sensitive queues. Figure 4 As shown, first sort the deadline-sensitive task queue according to the deadline. We can obtain the node set N occupied by task n n , and the set of nodes N occupied by task m m , we can find out whether there is port competition between tasks m and n mn :

[0082]

[0083] By traversing other tasks in the queue with lower priority than task n, calculate the competition factor δ of task n n for:

[0084]

[0085] Where J is the task set. The competition factor represents the number of nodes in the queue that will be affected by the task. The fewer the number of nodes with port competition, the larger the competition factor, and the more the number of nodes with port competition, the smaller the competition factor. Therefore, when determining the size of the competition factor and prioritizing according to the competition factor, we move the tasks with high competition factors to the end of the task queue. It can be seen that by reducing the priority of tasks with high competition factors, the risk of overdue of other tasks is reduced.

[0086] In some embodiments, for bandwidth allocation, 90% of the network bandwidth is dedicated to deadline-sensitive tasks by dynamically allocating bandwidth to each job. Bandwidth allocation follows a greedy approach based on queue priority until 90% of the network bandwidth is utilized. We reserve 10% of the bandwidth for each port, which will be shared fairly with all flows in the link. This ensures that all tasks in the cluster can complete the training of the task within a certain period of time, so that there will be no task starvation. By reserving a fixed amount of shared bandwidth on the link, task starvation is effectively avoided and no tasks are trained.

[0087] In some embodiments, the processing flow of Allot is as follows:

[0088] Step (1) When a new task arrives, Allot first determines whether the task is a deadline-sensitive task. If so, it infers the shortest time required to complete the task. Otherwise, it is placed in the best-effort task queue. When a task completes training, it directly goes to step (3).

[0089] Step (2) infers the shortest completion time of the task. If the task cannot meet the deadline in the shortest time, the task is placed in the best effort task queue.

[0090] Step (3) The deadline-sensitive queue is pre-sorted according to the deadline of the task. The earlier the deadline, the higher the priority of the task.

[0091] Step (4) can calculate the competition factor of each task according to the node occupancy of each task, and move the tasks with high competition factors to the end of the queue.

[0092] Step (5) The scheduler maintains a global deadline-sensitive queue, and each link greedily allocates 90% of the bandwidth to each task based on the priority of the queue.

[0093] Step (6) The remaining bandwidth of each link is equally divided among all flows in the link. In some embodiments, for the Allot scheduler, we use a flow-level network simulator to evaluate the performance of Allot. The simulation uses an AI training cluster consisting of 128 servers, with every four servers connected to a ToR switch. The bandwidth of its inlet / outlet links is set to 40 Gbps. We use real network loads and artificially synthesized network loads to implement simulations. Microsoft's job trace is used to simulate real workloads. Considering that the most popular and practical models have more than millions of parameters, such as VGG-16 with about 138M parameters, ResNet-50 with about 23M parameters, and AlexNet with about 61M parameters. Therefore, we artificially synthesized dozens of parameters evenly distributed in [10 7 ,10 9 ] task load.

[0094] Through simulation, we evaluate the success rate of Allot in meeting task deadlines and compare it with existing traffic scheduling schemes. For the sake of generality, we perform 20 experiments with the same parameter settings and select the average value as the result. Based on the simulation results of two types of loads, we can conclude that Allot's deadline satisfaction rate is more than 1.4 times higher than other traffic scheduling schemes.

[0095] In order to observe the impact of deadline distribution on simulation results, we set the deadline of each task to (1.2+k)*T dur , where T duris the training duration of the task in the real load, and the value of k follows the uniform distribution U[0,0.2x]. We simulated and compared the performance in different deadline intervals by changing the value of x. The results show that Allot's success rate in meeting task deadlines is better than other scheduling schemes. We also compared the success rate of meeting task deadlines in different scenarios by adjusting the proportion of deadline-sensitive tasks and the size of network load. According to a large number of simulation numerical results, it can be concluded that Allot's success rate significantly exceeds that of other scheduling schemes. In summary, Allot can better meet the needs of different users in the cluster and ensure that as many tasks as possible can complete training before the deadline.

[0096] From the above, it can be seen that the traffic scheduling method provided by the present application completes traffic scheduling by prioritizing the tasks to be trained and allocating bandwidth to the tasks to be trained according to the priority ranking, thereby reducing the risk of overdue time-sensitive tasks.

[0097] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0098] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a traffic scheduling device.

[0100] refer to Figure 5 , the flow scheduling device comprises:

[0101] A determination module 502 is configured to determine a number of tasks to be trained;

[0102] The queue allocation module 504 is configured to determine the shortest completion time of training any of the tasks to be trained, and allocate a queue to any of the training tasks according to the shortest completion time;

[0103] A sorting module 506 is configured to, in response to determining that the queue of the task to be trained is assigned to the time-sensitive queue, determine a number of subtasks to be trained in the task to be trained, prioritize the subtasks to be trained, and determine the priority of any subtask to be trained;

[0104] The bandwidth allocation module 508 is configured to allocate bandwidth for the task to be trained according to the priority level to complete traffic scheduling.

[0105] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0106] The device of the above embodiment is used to implement the corresponding traffic scheduling method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0107] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the traffic scheduling method described in any of the above embodiments is implemented.

[0108] Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0109] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0110] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0111] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0112] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0113] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0114] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0115] The electronic device of the above-mentioned embodiment is used to implement the corresponding traffic scheduling method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0116] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the traffic scheduling method described in any of the above embodiments.

[0117] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0118] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the traffic scheduling method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0119] Based on the same inventive concept, corresponding to the traffic scheduling method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor executes the traffic scheduling method. Corresponding to the execution subject corresponding to each step in each embodiment of the traffic scheduling method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0120] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the traffic scheduling method described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0121] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0122] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0123] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0124] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A traffic scheduling method, characterized in that: Used to schedule traffic resources in multi-tenant cluster training, including: Determine a number of tasks to be trained; Determine the shortest completion time for training any of the tasks to be trained, and allocate a queue for any of the training tasks according to the shortest completion time; In response to determining that the queue of the task to be trained is allocated to the time-sensitive queue, determining a number of subtasks to be trained in the task to be trained, and prioritizing the subtasks to be trained to determine the priority of any subtask to be trained; Bandwidth is allocated to the task to be trained according to the priority level to complete traffic scheduling.

2. The method according to claim 1, characterized in that The shortest completion time is the training time required when the task to be trained exclusively uses the network bandwidth; The determining of the required training duration of any of the tasks to be trained comprises: According to the formula shown below, the required training time T is determined n : Tn=(Tcomp+Tcomm))*Niter Among them, Tcomp is the computing time, Tcomm is the communication time, and Niter is the number of task iterations.

3. The method according to claim 1, characterized in that The queue allocation for any of the training tasks according to the shortest completion time includes: Determine a preset deadline of any of the training tasks, compare the preset deadline of any of the training tasks with the shortest completion time, and allocate a queue for any of the training tasks according to the comparison result; In response to determining that the shortest completion time is less than or equal to the preset deadline, assigning the training task to the time-sensitive queue; In response to determining that the shortest completion time is greater than the preset deadline, the training task is assigned to a time-insensitive queue.

4. The method according to claim 1, characterized in that: The step of prioritizing the plurality of subtasks to be trained and determining the priority of any subtask to be trained includes: Determine an occupied node ID of any of the subtasks to be trained, and determine a competition factor of any of the subtasks to be trained according to the occupied node ID; Priority ranking is performed on the plurality of subtasks to be trained according to the competition factor to determine the priority of any subtask to be trained; wherein, the larger the competition factor, the lower the priority, and the smaller the competition factor, the higher the priority.

5. The method according to claim 4, characterized in that The determining, according to the occupied node ID, a competition factor of any of the subtasks to be trained comprises: Determine, according to the occupied node ID, the number of nodes for which port contention exists between the target subtask to be trained and a plurality of subtasks to be trained other than the target subtask to be trained; The competition factor is determined according to the number of nodes with port competition; wherein, the fewer the number of nodes with port competition, the greater the competition factor, and the more the number of nodes with port competition, the smaller the competition factor.

6. The method according to claim 1, characterized in that The allocating bandwidth for the task to be trained according to the priority to complete traffic scheduling includes: Determine the full bandwidth quota and the reserved bandwidth quota; Determine the bandwidth quota to be allocated according to the full bandwidth quota and the reserved bandwidth quota; wherein the ratio of the reserved bandwidth quota to the bandwidth quota to be allocated is 1:9; Bandwidth is allocated for the task to be trained according to the priority and the bandwidth quota to be allocated, thereby completing traffic scheduling.

7. A flow scheduling device, characterized in that: include: A determination module, configured to determine a number of tasks to be trained; A queue allocation module is configured to determine the shortest completion time of training any of the tasks to be trained, and allocate a queue to any of the training tasks according to the shortest completion time; a sorting module, configured to, in response to determining that the queue of the task to be trained is allocated to the time-sensitive queue, determine a number of subtasks to be trained in the task to be trained, prioritize the subtasks to be trained, and determine the priority of any subtask to be trained; The bandwidth allocation module is configured to allocate bandwidth for the task to be trained according to the priority level to complete traffic scheduling.

8. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

9. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 6.

Images