Task scheduling method, device, electronic device, storage medium and program product

Abstract

The present disclosure provides a task scheduling method, device, electronic device, storage medium and program product, which relate to the field of artificial intelligence technology, and in particular to the field of distributed data processing, deep learning and large model technology. The method includes: obtaining a scheduling strategy for model training, the scheduling strategy is used to allocate multiple computing tasks to multiple computing units based on an interleaved pipeline mechanism, and the multiple computing tasks are sequentially allocated execution time intervals; determining a specified forward computing task and a target reverse computing task from at least one computing task related to the same batch of data, and the number of intervals of the execution time interval between the target reverse computing task and the specified forward computing task meets a preset condition; adjusting the execution time interval of the target reverse computing task until the adjusted number of intervals is less than a preset number threshold, thereby obtaining a target scheduling strategy; and scheduling the computing units to execute the computing tasks based on the target scheduling strategy.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of distributed data processing, deep learning, and large model technology. Background Art

[0002] In the field of deep learning, training tasks across computing units can be scheduled based on distributed strategies to improve the training efficiency of deep learning models. For example, distributed strategies can include interleaved pipeline strategies, which can further optimize the task execution efficiency of multiple computing units and reduce bubble occupancy during training. Summary of the Invention

[0003] The present disclosure provides a task scheduling method, device, electronic device, storage medium and program product.

[0004] According to one aspect of the present disclosure, a task scheduling method is provided, including: obtaining a scheduling strategy for model training, the scheduling strategy being used to allocate multiple computing tasks to multiple computing units based on an interleaved pipeline mechanism, the multiple computing tasks being sequentially allocated execution time intervals, the computing tasks being used for at least one processing layer of the model to process batch data; determining a specified forward computing task and a target reverse computing task from at least one computing task associated with the same batch data, the specified forward computing task and the target reverse computing task being associated with a specified processing layer of the model, the number of intervals of the execution time intervals between the target reverse computing task and the specified forward computing task meeting a preset condition; adjusting the execution time interval of the target reverse computing task until the adjusted number of intervals is less than a preset number threshold, thereby obtaining a target scheduling strategy; and scheduling the computing units to perform computing tasks based on the target scheduling strategy.

[0005] According to another aspect of the present disclosure, a task scheduling device is provided, including: an acquisition module for acquiring a scheduling strategy for model training, the scheduling strategy being used to allocate multiple computing tasks to multiple computing units based on an interleaved pipeline mechanism, the multiple computing tasks being sequentially allocated execution time intervals, the computing tasks being used for at least one processing layer of the model to process batch data; a first determination module for determining a specified forward computing task and a target reverse computing task from at least one computing task associated with the same batch data, the specified forward computing task and the target reverse computing task being associated with a specified processing layer of the model, the number of intervals of the execution time intervals between the target reverse computing task and the specified forward computing task meeting a preset condition; an adjustment module for adjusting the execution time interval of the target reverse computing task until the adjusted number of intervals is less than a preset number threshold, thereby obtaining a target scheduling strategy; and an execution module for scheduling the computing units to execute computing tasks based on the target scheduling strategy.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided by an embodiment of the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method provided by an embodiment of the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the method provided in the embodiment of the present disclosure when executed by a processor.

[0009] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0011] Figure 1 Schematically illustrates an exemplary system architecture to which the task scheduling method and apparatus according to an embodiment of the present disclosure can be applied;

[0012] Figure 2 The following schematically shows a flowchart of a task scheduling method according to an embodiment of the present disclosure;

[0013] Figure 3 The following schematically shows an application scenario diagram of the task scheduling method according to an embodiment of the present disclosure;

[0014] Figure 4A A schematic diagram schematically illustrates a scheduling strategy according to an embodiment of the present disclosure;

[0015] Figure 4B A schematic diagram schematically illustrates a target scheduling strategy according to an embodiment of the present disclosure;

[0016] Figure 5 A flowchart of scheduling a computing unit to perform a computing task based on a target scheduling strategy according to an embodiment of the present disclosure is schematically shown;

[0017] Figure 6 A block diagram schematically illustrates a task scheduling device according to an embodiment of the present disclosure; and

[0018] Figure 7 A schematic block diagram of an example electronic device that can be used to implement the task scheduling method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0021] The inventors found that scheduling the training process of deep learning models based on a distributed strategy has problems such as low computational efficiency of computing units and long training time.

[0022] Embodiments of the present disclosure provide a task scheduling method, apparatus, electronic device, storage medium, and program product. The task scheduling method includes: obtaining a scheduling strategy for model training, the scheduling strategy being used to allocate multiple computing tasks to multiple computing units based on an interleaved pipeline mechanism, the multiple computing tasks being sequentially allocated execution time intervals, the computing tasks being used by at least one processing layer of the model to process batch data; determining a specified forward computing task and a target reverse computing task from at least one computing task associated with the same batch data, the specified forward computing task and the target reverse computing task being associated with a specified processing layer of the model, the number of intervals of the execution time interval between the target reverse computing task and the specified forward computing task meeting a preset condition; adjusting the execution time interval of the target reverse computing task until the adjusted number of intervals is less than a preset number threshold, thereby obtaining a target scheduling strategy; and scheduling the computing units to execute computing tasks based on the target scheduling strategy.

[0023] According to an embodiment of the present disclosure, by determining the designated forward computing task and the target reverse computing task from the scheduling strategy determined based on the staggered pipeline mechanism, it can be preliminarily determined that the factors affecting the computing efficiency of the computing unit may include that after the designated forward computing task is completed, the computing unit executing the designated forward computing task stores the calculation result of the designated forward computing task in a plurality of execution time intervals that meet preset conditions, resulting in an imbalance in storage space between multiple computing units and affecting computing efficiency. By adjusting the execution time interval of the target reverse computing task so that the number of intervals between the target reverse computing task and the designated forward computing task is less than a preset number threshold, the storage time of the calculation result of the designated forward computing task in the computing unit can be reduced, thereby avoiding redundant storage resource settings caused by storage imbalance between multiple computing units during model training, and avoiding the reduction in model training efficiency due to storage resource overload of the computing unit, thereby improving the overall training efficiency of distributed model training.

[0024] Figure 1 An exemplary system architecture to which the task scheduling method and apparatus according to an embodiment of the present disclosure can be applied is schematically shown.

[0025] It should be noted that Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure. This does not mean that the embodiments of the present disclosure cannot be applied to other devices, systems, environments, or scenarios. For example, in another embodiment, an exemplary system architecture to which the task scheduling method and apparatus may be applied may include a terminal device, but the terminal device may implement the task scheduling method and apparatus provided by the embodiments of the present disclosure without interacting with a server.

[0026] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0027] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0028] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0029] Server 105 may be a server that provides various services, such as a background management server (for example only) that supports content browsed by users using terminal devices 101, 102, and 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal device.

[0030] Server 105 may be a cloud server, also known as a cloud computing server or cloud host. This server is a host product within a cloud computing service system that addresses the management difficulties and limited scalability of traditional physical hosts and VPS (Virtual Private Server) services. The server may also be a distributed system server or a server integrated with blockchain technology.

[0031] It should be noted that the task scheduling method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the task scheduling apparatus provided in the embodiment of the present disclosure can generally be set in the server 105. The task scheduling method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the task scheduling apparatus provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0032] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0033] Figure 2 The flowchart of the task scheduling method according to the embodiment of the present disclosure is schematically shown.

[0034] like Figure 2 As shown, the task scheduling method includes operations S210 to S240.

[0035] In operation S210 , a scheduling strategy for model training is obtained.

[0036] In operation S220 , a designated forward computing task and a target backward computing task are determined from at least one computing task associated with the same batch of data.

[0037] In operation S230 , the execution time interval of the target reverse calculation task is adjusted until the number of intervals after the adjustment is less than a preset number threshold, thereby obtaining a target scheduling strategy.

[0038] In operation S240 , the computing units are scheduled to execute computing tasks based on the target scheduling policy.

[0039] According to an embodiment of the present disclosure, a scheduling strategy is used to allocate multiple computing tasks to multiple computing units based on an interleaved pipeline mechanism, and the multiple computing tasks are sequentially allocated execution time intervals.

[0040] According to embodiments of the present disclosure, a computing task is used by at least one processing layer of a model to process batch data. Batch data can be obtained by partitioning sample data and can include batch data or micro-batch data. Embodiments of the present disclosure do not limit the dimensionality or data volume of batch data.

[0041] According to an embodiment of the present disclosure, the model may include a deep learning model constructed based on any algorithm.

[0042] In one example, the model may be a large model built based on a deep learning algorithm, such as a large language model (LLM), or the model may be other large models with a large number of parameters.

[0043] According to embodiments of the present disclosure, a computing unit may include a processor for performing computing tasks, such as a graphics processing unit (GPU), a central processing unit (CPU), and the like. Alternatively, the computing unit may be a processor module of a processor, for example, the same GPU may be divided into multiple computing units. Alternatively, the computing unit may be a collection of multiple processors or computing devices, for example, multiple GPUs may be divided into the same computing unit. The embodiments of the present disclosure do not limit the specific configuration of the computing unit.

[0044] According to an embodiment of the present disclosure, a scheduling strategy constructed based on an interleaved pipeline mechanism can divide the model into multiple different processing layers and assign multiple processing layers to multiple computing units. The sample data used to train the model can be divided into multiple batches of data, and multiple computing units can train the model by executing the computing tasks of multiple batches of data in parallel. Intermediate results can be transmitted between multiple computing units to execute the computing tasks of each processing layer. Among them, the computing tasks can include forward computing tasks and backward computing tasks. The execution time interval can be understood as the time step of the computing task. The computing unit can execute the computing task in the execution time interval to obtain the intermediate results.

[0045] In one example, the scheduling policy can be used to indicate the computational tasks for each batch of data assigned to each computing unit. Computational tasks can be arranged based on execution time intervals (or time steps). By assigning computational tasks for multiple batches of data to multiple computing units using an interleaved pipeline mechanism, multiple computing units can execute computational tasks in parallel, thereby improving model training efficiency.

[0046] According to an embodiment of the present disclosure, the specified forward computing task and the target reverse computing task are related to the specified processing layer of the model, and the number of intervals of the execution time intervals between the target reverse computing task and the specified forward computing task meets the preset conditions, for example, the number of intervals of the execution time intervals between the target reverse computing task and the specified forward computing task is greater than or equal to the preset number threshold.

[0047] According to an embodiment of the present disclosure, the designated processing layer may include any one or more of the multiple processing layers of the model. The designated forward computation task may be a forward computation task for the designated processing layer to perform the i-th batch of data, and the target reverse computation task may be a reverse computation task for the designated processing layer to perform the i-th batch of data. The designated forward computation task and the target reverse computation task may be computation tasks of the same computation unit.

[0048] According to an embodiment of the present disclosure, the intermediate result obtained by the computing unit after executing the specified forward computing task can be stored in a specified storage area. For example, it can be cached in the cache area of ​​the computing unit. When the number of intervals of the execution time intervals between the target reverse computing task and the specified forward computing task meets the preset conditions, it can be indicated that the intermediate result of the specified forward computing task is stored in the cache area of ​​the computing unit for the number of intervals of execution time intervals. When the number of intervals meets the preset conditions, it can be indicated that the storage time of the intermediate result of the specified forward computing task is too long, which will result in the storage space of other intermediate results of the computing unit executing other computing tasks being limited, and may also cause the computing unit to affect computing performance due to limited cache space during the execution of other computing tasks.

[0049] According to an embodiment of the present disclosure, by adjusting the execution time interval of the target reverse calculation task, the number of execution time intervals between the associated specified forward calculation task and the target reverse calculation task in the target scheduling strategy is made less than a preset number threshold, so that after the execution of the specified forward calculation task is completed, the intermediate results can be stored in a smaller number of execution time intervals. And after a certain number of execution time intervals, the computing unit uses the intermediate results to execute the target reverse calculation task, so as to release the intermediate results of the specified forward calculation task in a timely manner. Reduce the storage time of the calculation result of the specified forward calculation task in the computing unit, and avoid the imbalance of storage space between multiple computing units affecting the execution of computing tasks and the transmission of intermediate results between multiple computing units.

[0050] In one example, the model to be trained can be a deep learning model built based on an attention network algorithm. The deep learning model can include multiple attention sub-layers and a normalization layer connected after the last attention sub-layer. The designated processing layer includes the attention network sub-layer and the normalization sub-layer. The normalization sub-layer can output the model's calculation results, which can include any type of data, such as text, images, and vectors.

[0051] According to an embodiment of the present disclosure, the batch data includes at least one of the following: image data, audio data, and text data.

[0052] In one example, the batch data may include a batch of image data. The model calculation result may include an object detection result for the image data. Alternatively, the model calculation result may also include a generated new image.

[0053] In one example, audio data can be encoded based on a model to obtain audio features, and the audio features can be processed through multiple processing layers to obtain model calculation results. The model calculation results can be, for example, text recognition results for the audio data.

[0054] In one example, text data may include any type of text, such as question text and news text, and the model calculation result may include text data, such as answer text and news rewrite text.

[0055] It should be noted that the above examples are only for illustrative purposes only, illustrating the model calculation results obtained after the model and calculation tasks involved in the embodiments of the present disclosure process different types of batch data, and are not intended to limit the actual processing capabilities of the model.

[0056] According to an embodiment of the present disclosure, the ratio between the number of batches of batch data and the number of computing units is a non-integer. The number of batches can represent the cumulative number of times of the interleaved pipeline parallel mechanism, and the number of computing units can represent the interleaved pipeline segmentation dimension of the interleaved pipeline parallel mechanism. In the case that the ratio between the number of batches of batch data and the number of computing units is a non-integer, it can be understood that it means that the number of batches cannot be divided evenly by the number of computing units, resulting in a scheduling strategy constructed based on the interleaved pipeline mechanism. It is necessary to use the storage interval of the computing unit to store the intermediate results of the specified forward computing task for a longer time. By adjusting the execution time interval of the target reverse computing task in the scheduling strategy, the execution interval between the specified forward computing task and the target reverse computing task can be shortened, fully releasing the computing power resources and storage space of the computing unit.

[0057] According to an embodiment of the present disclosure, adjusting the execution time interval of the target reverse computing task until the number of adjusted intervals is less than a preset number threshold, and obtaining the target scheduling strategy may include: adjusting the execution time interval of the target reverse computing task to obtain the Nth reverse execution time interval of the target reverse computing task; and adjusting the execution order of the target reverse computing task based on the Nth reverse execution time interval to obtain the target scheduling strategy.

[0058] According to an embodiment of the present disclosure, adjusting the execution order of the target reverse computing task based on the Nth reverse execution time interval can include moving the target reverse computing task in the task scheduling timing diagram represented by the scheduling strategy to a position corresponding to the Nth reverse execution time interval to obtain a task scheduling timing diagram representing the target scheduling strategy.

[0059] According to an embodiment of the present disclosure, the number of intervals between the Nth reverse execution time interval and the execution time interval of the specified forward computing task is less than a preset threshold. Therefore, the intermediate results of the specified forward computing task in the target scheduling policy can be cached for a smaller number of execution time intervals, thereby allowing the computing unit to free up its storage interval by executing the target reverse computing task.

[0060] In one example, a specified forward computing task is associated with the last processing layer among multiple processing layers, and the intermediate result of the specified forward computing task may include a model computing result. However, this is not limited to the above, and the intermediate result of the specified forward computing task may also include a loss value obtained by processing the model computing result with a loss function. The model computing result and loss value of the specified forward computing task in the target scheduling strategy can be cached for a smaller number of execution time intervals to avoid storage space imbalance among multiple computing units and improve the computing power resource balance and storage space balance of multiple computing units.

[0061] According to an embodiment of the present disclosure, the number of execution time intervals between the forward computing task and the target reverse computing task specified in the target scheduling policy is 0. Therefore, based on the target scheduling policy, the Nth computing unit can start executing the target reverse computing task in the next execution time interval after executing the forward computing task. After the target reverse computing task is completed, the Nth computing unit sends the intermediate result of the target reverse computing task to the N-1th computing unit. In this way, the storage interval of the Nth computing unit can be released in time, avoiding the intermediate computing result being stored in the Nth computing unit for a long time, resulting in an imbalance in the storage space between multiple computing units.

[0062] According to an embodiment of the present disclosure, N computing units have a unit order attribute, which is related to the execution order of computing tasks of the computing units. N>1, and the computing task of the Nth computing unit for the i-th batch of data includes a target reverse computing task. i is any integer greater than or equal to 1.

[0063] Figure 3 The following schematically illustrates an application scenario diagram of a task scheduling method according to an embodiment of the present disclosure.

[0064] like Figure 3 As shown, the application scenario includes a first computing unit 301, a second computing unit 302, a third computing unit 303, and a fourth computing unit 304 with a unit order attribute. The fourth computing unit may be associated with a specified forward computing task and a target reverse computing task of the i-th batch of data.

[0065] The target scheduling policy 310 can control and instruct the first computing unit 301, the second computing unit 302, the third computing unit 303, and the fourth computing unit 304 to execute computing tasks. The target scheduling policy 310 includes a designated forward task block J311 corresponding to the designated forward computing task for the sixth batch of data, and a target reverse task block J312 corresponding to the target reverse computing task for the sixth batch of data. The execution time interval between the designated forward task block J311 and the target reverse task block J312 is 0. Task scheduling for the fourth computing unit 304 based on the target scheduling policy 310 can cause the fourth computing unit 304 to execute the target reverse computing task for the sixth batch of data in the next execution time interval after executing the designated forward computing task for the sixth batch of data. This frees up the storage interval of the fourth computing unit 304 by transferring the intermediate results of the target reverse computing task for the sixth batch of data to the third computing unit 303.

[0066] According to an embodiment of the present disclosure, adjusting the execution order of the target reverse computing tasks based on the Nth reverse execution time interval to obtain a target scheduling strategy includes: adjusting the execution order of the target reverse computing tasks based on the Nth reverse execution time interval to obtain an intermediate scheduling strategy; and adjusting the execution order of the reverse computing tasks of the n-1th computing unit based on the execution order of the target reverse computing tasks in the intermediate scheduling strategy to obtain a target scheduling strategy.

[0067] According to an embodiment of the present disclosure, the execution order of the reverse calculation tasks of the N-1th to 1st computing units in the intermediate scheduling strategy is adjusted based on the staggered pipeline to obtain a target scheduling strategy. After the target reverse calculation unit is executed, the N-1th to 1st computing units will execute their respective reverse calculation tasks in turn in a staggered pipeline manner, thereby preventing the N-1th to 1st computing units from storing the intermediate results for too long, thereby improving the overall efficiency of model training.

[0068] According to an embodiment of the present disclosure, adjusting the execution order of the reverse calculation tasks of the n-1th computing unit based on the execution order of the target reverse calculation tasks in the intermediate scheduling strategy includes: adjusting the execution order of the n-1th associated reverse calculation tasks of the n-1th computing unit based on the execution order of the target reverse calculation tasks in the intermediate scheduling strategy, wherein the target reverse calculation task to the 1st associated reverse calculation task represents that multiple processing layers arranged consecutively in the model execute reverse calculation tasks in sequence.

[0069] It should be noted that N≥n>1, and n is an integer.

[0070] Figure 4A A schematic diagram schematically illustrates a scheduling strategy according to an embodiment of the present disclosure.

[0071] Figure 4B A schematic diagram of a target scheduling strategy according to an embodiment of the present disclosure is schematically shown.

[0072] like Figure 4A As shown, scheduling strategy 410 can allocate computing tasks for each of the nine batches of data based on an interleaved pipeline mechanism, resulting in a computing task timing diagram. The computing units used for model training may include a first computing device D1, a second computing device D2, a third computing device D3, and a fourth computing device D4, each having a unit sequence attribute. The first computing device D1 can execute the first forward computing task for the first batch of data during execution time interval 1, and the second computing device D2 can execute the second forward computing task for the second batch of data during execution time interval 2. Simultaneously, the fourth computing device can execute the 12th forward computing task for the sixth batch of data during execution time interval t1. The fourth computing device can execute the 9th backward computing task for the fourth batch of data during execution time interval t2. The fourth computing device can also execute the 1st backward computing task for the sixth batch of data during execution time interval t6. The number of execution time intervals between the 12th forward computing task for the sixth batch of data and the first backward computing task for the sixth batch of data is 4, satisfying a preset condition. Therefore, the 12th forward computing task of the 6th batch of data may be the designated forward computing task of the 6th batch of data. The 1st backward computing task of the 6th batch of data may be the target backward computing task of the 6th batch of data.

[0073] Combine Figure 4A and Figure 4B As shown, when the preset threshold value indicates that the number of intervals between execution time intervals is 1, the execution time interval of the first backward computation task of the sixth batch of data can be adjusted to obtain the fourth backward execution time interval t2. By adjusting the first backward computation task of the sixth batch of data to the execution order corresponding to execution time interval t2 using the fourth backward execution time interval, an intermediate scheduling strategy can be obtained. In the row corresponding to the third computing device D3, the second backward computation task of the sixth batch of data in execution time interval t8 is the third associated backward computation task of the third computing device D3. In the row corresponding to the second computing device D2, the third backward computation task of the sixth batch of data in execution time interval t9 is the second associated backward computation task of the second computing device D2. In the row corresponding to the first computing device D1, the fourth backward computation task of the sixth batch of data in execution time interval t10 is the first associated backward computation task of the first computing device D1. The third associated reverse calculation task, the second associated reverse calculation task, and the first associated reverse calculation task are all represented by white square task blocks in the first dotted area 411 .

[0074] The execution order of the 1st backward computing task based on the 6th batch of data in the intermediate scheduling strategy is expressed based on the execution time interval t2. The positions of the white square task blocks corresponding to the 3rd associated reverse computing task, the 2nd associated reverse computing task and the 1st associated reverse computing task in the first dotted area 411 can be adjusted so that the white square task blocks corresponding to the 3rd associated reverse computing task, the 2nd associated reverse computing task and the 1st associated reverse computing task in the dotted area 411 are located at the positions corresponding to the second dotted area 421, thereby adjusting the execution order of the 3rd associated reverse computing task, the 2nd associated reverse computing task and the 1st associated reverse computing task.

[0075] Combine Figure 4A and Figure 4B As shown, by adjusting the target reverse calculation tasks of the 7th batch of data, the 8th batch of data and the 9th batch of data, and adjusting the 3rd associated reverse calculation task and the 2nd associated reverse calculation task of the target reverse calculation tasks of the 7th batch of data, the 8th batch of data and the 9th batch of data to the 1st associated reverse calculation task, the target adjustment strategy 420 can be obtained.

[0076] From the target reverse calculation task, the third associated reverse calculation task, the second associated reverse calculation task to the first associated reverse calculation task, it can be represented that multiple processing layers arranged continuously in the model perform reverse calculation tasks in sequence.

[0077] It should be understood that Figure 4A and Figure 4B The middle rectangular task block represents the forward calculation task, and the square task block represents the reverse calculation task.

[0078] It should be noted that Figure 4A and Figure 4B The ellipsis “…” in the table is used to indicate omitted parts of the scheduling policy and target scheduling policy.

[0079] According to an embodiment of the present disclosure, the designated forward computing task includes the Nth forward computing subtask and the loss computing task, and the Nth forward computing subtask represents that the Nth computing unit uses the designated processing layer to process the N-1th forward intermediate result sent by the N-1th computing unit.

[0080] In one example, the designated processing layer may include the last processing layer after which the model is partitioned. For the i-th batch of data, the designated processing layer processes the N-1th forward intermediate result sent by the N-1th computing unit to perform the forward computation subtask for the i-th batch of data, thereby obtaining the model computation result for the i-th batch of data.

[0081] According to an embodiment of the present disclosure, a loss calculation task represents the loss value of a batch of data calculated by the Nth computing unit based on the model calculation results output by the specified processing layer. In the target scheduling strategy, the number of execution time intervals between the specified forward computing task and the target reverse computing task is less than a preset number threshold. Therefore, the Nth computing unit can execute the target reverse computing task in a timely manner and send the reverse calculation result of the target reverse computing task to the N-1th computing unit, thereby improving the utilization rate of the storage interval of the Nth computing unit, avoiding long-term caching of intermediate results, and improving the storage space balance between multiple computing units.

[0082] In one example, the model calculation results of the i-th batch of data can be processed based on the loss function to obtain the loss value of the i-th batch of data.

[0083] According to an embodiment of the present disclosure, the target number of intervals between the forward computing task and the target reverse computing task specified in the target scheduling strategy is K, K≥0, and is less than a preset number threshold.

[0084] In one example, the preset number threshold can be 1, the target interval number between the forward computing task and the target reverse computing task specified in the target scheduling policy is K, and the target interval number is the number of execution time intervals between the forward computing task and the target reverse computing task specified in the target scheduling policy.

[0085] Figure 5 The flowchart of scheduling computing units to execute computing tasks based on a target scheduling strategy according to an embodiment of the present disclosure is schematically shown.

[0086] like Figure 5 As shown, scheduling the computing unit to execute the computing task based on the target scheduling policy includes operations S510 to S540.

[0087] In operation S510 , based on a specified benchmark execution time interval of a forward computing task, the Nth computing unit calls the model computing result stored in the Nth storage area to perform a loss computing subtask and obtains a loss value stored in the Nth storage area.

[0088] In operation S520 , in a reference execution time interval that is K execution time intervals away from the base execution time interval, the Nth calculation unit performs the target reverse calculation task based on the loss value, and obtains an Nth reverse intermediate result stored in the Nth storage area;

[0089] In operation S530 , the Nth reverse intermediate result is transmitted to the N−1th computing unit.

[0090] In operation S540 , the Nth storage area is released.

[0091] According to an embodiment of the present disclosure, the benchmark execution time interval may be the execution time interval of a specified forward computation task for the i-th batch of data in the target scheduling policy. During the benchmark execution time interval, the Nth computation unit invokes the model computation result stored in the Nth storage area to execute the loss computation subtask, obtaining a loss value, which may be stored in the Nth storage area.

[0092] According to an embodiment of the present disclosure, the reference execution time interval may be the execution time interval of the target reverse calculation task for the i-th batch of data. During the reference execution time interval, the N-th computing unit may retrieve the model calculation results for the i-th batch of data from the N-th storage area to obtain a loss value for the i-th batch of data. The loss value is transmitted as the N-th reverse intermediate result to the N-1-th computing unit.

[0093] In one example, the computing unit may be a GPU unit, and the Nth storage area may be a video memory space of the GPU unit. Releasing the Nth storage area may include deleting a loss value of the Nth storage area to avoid accumulation of activation loss values ​​calculated by the loss function due to the number of batches of data not being divisible by the number of computing units, thereby preventing imbalance of video memory space among multiple computing units.

[0094] According to an embodiment of the present disclosure, releasing the Nth storage area includes: deleting the loss value and the Nth reverse intermediate result stored in the Nth storage area during the process of transmitting the Nth reverse intermediate result to the N-1th computing unit.

[0095] According to an embodiment of the present disclosure, by deleting the loss value and the Nth reverse intermediate result stored in the Nth storage area during the process of transmitting the Nth reverse intermediate result to the N-1th computing unit, the Nth computing unit can promptly convert the Nth storage area into a storage idle state, so that the Nth computing unit can execute the next computing task more promptly, avoiding the accumulation of activation values ​​calculated by the loss function, thereby improving the overall training efficiency of the model training.

[0096] According to an embodiment of the present disclosure, the Nth storage area may be the cache area of ​​the Nth computing unit. For example, the Nth storage area may be the video memory module of a GPU chip. The task scheduling method provided by the embodiment of the present disclosure can release the storage area of ​​the computing unit in a more timely manner, thereby improving the memory usage of the graphics card. In one example, the task scheduling method based on the embodiment of the present disclosure can save 10GB of video memory space.

[0097] According to an embodiment of the present disclosure, the N-1th forward intermediate result sent by the N-1th computing unit is related to the i-th batch of data.

[0098] According to an embodiment of the present disclosure, scheduling computing units to perform computing tasks based on a target scheduling strategy also includes: in the next execution time interval of the reference execution time interval, the Nth computing unit uses a designated processing layer to process the N-1th forward intermediate result related to the i+1th batch of data to obtain a model calculation result related to the i+1th batch of data.

[0099] According to an embodiment of the present disclosure, the model calculation results associated with the (i+1)th batch of data are stored in the Nth storage area. By releasing the Nth storage area during the process of transmitting the Nth reverse intermediate result to the (N-1)th computing unit, the Nth computing unit can be promptly switched to an idle state during the next execution time interval after the reference execution time interval, thereby utilizing the relatively sufficient free storage interval to execute the designated forward calculation task for the (i+1)th batch of data and obtain the model calculation results associated with the (i+1)th batch of data. This improves the efficiency of executing calculation tasks for different batches of data.

[0100] In one example, the Nth storage area can also execute the loss calculation subtask using the model calculation results related to the (i+1)th batch of data during the execution time interval following the reference execution time interval, thereby obtaining the loss value of the (i+1)th batch of data. Thus, based on the method provided in the embodiments of the present disclosure, the target reverse calculation results for the (i+1)th batch of data can be calculated relatively promptly after K intervals of execution time intervals, and the Nth storage area can be released promptly.

[0101] Figure 6 The block diagram of the task scheduling device according to an embodiment of the present disclosure is schematically shown.

[0102] like Figure 6 As shown, the task scheduling device 600 includes: an acquisition module 610 , a first determination module 620 , an adjustment module 630 and an execution module 640 .

[0103] an acquisition module 610 for acquiring a scheduling policy for model training, the scheduling policy being used to allocate multiple computing tasks to multiple computing units based on an interleaved pipeline mechanism, the multiple computing tasks being sequentially assigned execution time intervals, the computing tasks being used to process batch data by at least one processing layer of the model;

[0104] A first determining module 620 is configured to determine, from at least one computing task associated with the same batch of data, a specified forward computing task and a target reverse computing task, wherein the specified forward computing task and the target reverse computing task are associated with a specified processing layer of the model, and the number of intervals between the execution time intervals of the target reverse computing task and the specified forward computing task satisfies a preset condition;

[0105] An adjustment module 630 is configured to adjust the execution time interval of the target reverse calculation task until the number of intervals after the adjustment is less than a preset number threshold, thereby obtaining a target scheduling strategy; and

[0106] The execution module 640 is used to schedule the computing unit to execute the computing task based on the target scheduling policy.

[0107] According to an embodiment of the present disclosure, the ratio between the number of batches of batch data and the number of computing units is a non-integer.

[0108] According to an embodiment of the present disclosure, N computing units have a unit order attribute, which is related to the execution order of computing tasks of the computing units, N>1, and a specified forward computing task includes an Nth forward computing subtask and a loss computing task, and the Nth forward computing subtask represents that the Nth computing unit processes the N-1th forward intermediate result sent by the N-1th computing unit using a specified processing layer;

[0109] The loss calculation task represents that the Nth computing unit calculates the loss value of the batch data according to the model calculation results output by the specified processing layer.

[0110] According to an embodiment of the present disclosure, the target number of intervals between the forward computing task and the target reverse computing task specified in the target scheduling strategy is K, K≥0, and is less than a preset number threshold.

[0111] According to an embodiment of the present disclosure, the execution module includes: a first execution submodule, a second execution submodule, a transmission submodule, and a release submodule.

[0112] The first execution submodule is used to call the model calculation result stored in the Nth storage area to execute the loss calculation subtask based on the benchmark execution time interval of the specified forward calculation task, and obtain the loss value stored in the Nth storage area.

[0113] The second execution submodule is used to perform the target reverse calculation task based on the loss value by the Nth calculation unit in a reference execution time interval that is K execution time intervals away from the benchmark execution time interval, and obtain the Nth reverse intermediate result stored in the Nth storage area.

[0114] The transmission submodule is used to transmit the Nth reverse intermediate result to the N-1th computing unit.

[0115] The release submodule is used to release the Nth storage area.

[0116] According to an embodiment of the present disclosure, the release submodule includes a release submodule.

[0117] The deleting unit is used to delete the loss value and the Nth reverse intermediate result stored in the Nth storage area during the process of transmitting the Nth reverse intermediate result to the N-1th calculation unit.

[0118] According to an embodiment of the present disclosure, the N-1th forward intermediate result sent by the N-1th computing unit is related to the i-th batch of data.

[0119] According to an embodiment of the present disclosure, the execution module further includes a processing submodule.

[0120] The processing submodule is configured to process, in an execution time interval next to the reference execution time interval, the Nth computing unit using a designated processing layer to process the N-1th forward intermediate result associated with the (i+1)th batch of data, to obtain a model calculation result associated with the (i+1)th batch of data, wherein the model calculation result associated with the (i+1)th batch of data is stored in the Nth storage area.

[0121] According to an embodiment of the present disclosure, the number of execution time intervals between the forward computing task and the target reverse computing task specified in the target scheduling policy is 0.

[0122] According to an embodiment of the present disclosure, the adjustment module includes: a first adjustment submodule and a second adjustment submodule.

[0123] The first adjustment submodule is used to adjust the execution time interval of the target reverse computing task to obtain the Nth reverse execution time interval of the target reverse computing task, wherein the number of intervals between the Nth reverse execution time interval and the execution time interval of the specified forward computing task is less than a preset number threshold.

[0124] The second adjustment submodule is used to adjust the execution order of the target reverse computing tasks based on the Nth reverse execution time interval to obtain a target scheduling strategy.

[0125] According to an embodiment of the present disclosure, N computing units have a unit order attribute, the unit order attribute is related to the execution order of computing tasks of the computing units, N>1, and the computing task of the Nth computing unit for the i-th batch of data includes a target reverse computing task;

[0126] According to an embodiment of the present disclosure, the second adjustment submodule includes: a first adjustment unit and a second adjustment unit.

[0127] The first adjustment unit is used to adjust the execution order of the target reverse computing tasks based on the Nth reverse execution time interval to obtain an intermediate scheduling strategy.

[0128] The second adjustment unit is used to adjust the execution order of the reverse calculation tasks of the (n-1)th calculation unit based on the execution order of the target reverse calculation tasks in the intermediate scheduling strategy to obtain the target scheduling strategy.

[0129] According to an embodiment of the present disclosure, the second adjustment unit includes an adjustment subunit.

[0130] The adjustment subunit is used to adjust the execution order of the n-1th associated reverse calculation task of the n-1th calculation unit based on the execution order of the target reverse calculation task in the intermediate scheduling strategy, wherein the target reverse calculation task to the first associated reverse calculation task represents that multiple processing layers arranged continuously in the model execute reverse calculation tasks in sequence.

[0131] According to an embodiment of the present disclosure, the designated processing layer includes an attention network sublayer and a normalization sublayer.

[0132] According to an embodiment of the present disclosure, the batch data includes at least one of the following: image data, audio data, and text data.

[0133] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0134] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0135] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method described above.

[0136] According to an embodiment of the present disclosure, a computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the method described above.

[0137] Figure 7 A schematic block diagram of an example electronic device that can be used to implement the task scheduling method of an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0138] like Figure 7As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. Computing unit 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to bus 704.

[0139] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0140] The computing unit 701 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the task scheduling method. For example, in some embodiments, the task scheduling method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the task scheduling method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the task scheduling method in any other suitable manner (e.g., via firmware).

[0141] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0145] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0146] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0147] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0148] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A task scheduling method, comprising: Obtaining a scheduling policy for model training, the scheduling policy being used to assign a plurality of computing tasks to a plurality of computing units based on an interleaved pipeline mechanism, the plurality of computing tasks being sequentially assigned execution time intervals, the computing tasks being used to process batch data by at least one processing layer of the model; Determining, from at least one computing task associated with the same batch of data, a specified forward computing task and a target reverse computing task, wherein the specified forward computing task and the target reverse computing task are associated with a specified processing layer of the model, and a number of intervals between execution time intervals of the target reverse computing task and the specified forward computing task satisfies a preset condition; Adjusting the execution time interval of the target reverse calculation task until the number of intervals after adjustment is less than a preset number threshold, thereby obtaining a target scheduling strategy; and The computing unit is scheduled to execute computing tasks based on the target scheduling policy.

2. The method according to claim 1, wherein The ratio between the number of batches of the batch data and the number of the computing units is a non-integer.

3. The method according to claim 1, wherein N of the computing units have a unit sequence attribute, the unit sequence attribute is related to the execution order of computing tasks of the computing units, N>1, the designated forward computing task includes an Nth forward computing subtask and a loss computing task, the Nth forward computing subtask represents that the Nth computing unit processes an N-1th forward intermediate result sent by the N-1th computing unit using the designated processing layer; The loss calculation task represents that the Nth calculation unit calculates the loss value of the batch data according to the model calculation result output by the designated processing layer.

4. The method according to claim 3, wherein: The target number of intervals between the specified forward computing task and the target reverse computing task in the target scheduling strategy is K, K≥0, and is less than a preset number threshold; The step of scheduling the computing unit to perform the computing task based on the target scheduling policy includes: Based on the benchmark execution time interval of the specified forward computing task, the Nth computing unit calls the model computing result stored in the Nth storage area to execute the loss computing subtask, and obtains the loss value stored in the Nth storage area; In a reference execution time interval that is K execution time intervals away from the benchmark execution time interval, the Nth calculation unit performs the target reverse calculation task based on the loss value, and obtains an Nth reverse intermediate result stored in the Nth storage area; transmitting the Nth reverse intermediate result to the N-1th computing unit; and The Nth storage area is released.

5. The method according to claim 4, wherein The releasing of the Nth storage area comprises: During the process of transmitting the Nth reverse intermediate result to the N-1th calculation unit, the loss value and the Nth reverse intermediate result stored in the Nth storage area are deleted.

6. The method according to claim 4, wherein: The N-1th forward intermediate result sent by the N-1th computing unit is related to the i-th batch of data, where i>1 and i is an integer; Wherein, scheduling the computing unit to perform the computing task based on the target scheduling policy further includes: In an execution time interval next to the reference execution time interval, the Nth calculation unit processes the N-1th forward intermediate result related to the (i+1)th batch of data using the designated processing layer to obtain a model calculation result related to the (i+1)th batch of data, wherein the model calculation result related to the (i+1)th batch of data is stored in the Nth storage area.

7. The method according to any one of claims 4 to 6, wherein The number of execution time intervals between the specified forward computing task and the target reverse computing task in the target scheduling policy is 0.

8. The method according to claim 1, wherein The step of adjusting the execution time interval of the target reverse calculation task until the number of intervals after adjustment is less than a preset number threshold, and obtaining the target scheduling strategy includes: Adjusting the execution time interval of the target reverse computing task to obtain an Nth reverse execution time interval of the target reverse computing task, wherein the number of intervals between the Nth reverse execution time interval and the execution time interval of the specified forward computing task is less than a preset number threshold; and The execution order of the target reverse computing tasks is adjusted based on the Nth reverse execution time interval to obtain the target scheduling strategy.

9. The method according to claim 8, wherein N of the computing units have a unit sequence attribute, the unit sequence attribute being related to the execution order of computing tasks of the computing units, N>1, and the computing task of the Nth computing unit for the i-th batch of data includes the target reverse computing task; The step of adjusting the execution order of the target reverse computing tasks based on the Nth reverse execution time interval to obtain the target scheduling strategy includes: Adjusting the execution order of the target reverse computing tasks based on the Nth reverse execution time interval to obtain an intermediate scheduling strategy; and Based on the execution order of the target reverse computing tasks in the intermediate scheduling strategy, the execution order of the reverse computing tasks of the n-1th computing unit is adjusted to obtain the target scheduling strategy, N≥n>1.

10. The method according to claim 9, wherein: The adjusting the execution order of the reverse computing tasks of the n-1th computing unit based on the execution order of the target reverse computing tasks in the intermediate scheduling strategy includes: Based on the execution order of the target reverse computing task in the intermediate scheduling strategy, the execution order of the n-1th associated reverse computing task of the n-1th computing unit is adjusted, wherein the target reverse computing task to the first associated reverse computing task represents that multiple processing layers arranged consecutively in the model execute reverse computing tasks in sequence.

11. The method according to claim 1, wherein The designated processing layer includes an attention network sublayer and a normalization sublayer.

12. The method according to claim 1, wherein The batch data includes at least one of the following: Image data, audio data, text data.

13. A task scheduling device, comprising: an acquisition module, configured to acquire a scheduling policy for model training, the scheduling policy being configured to allocate a plurality of computing tasks to a plurality of computing units based on an interleaved pipeline mechanism, the plurality of computing tasks being sequentially assigned execution time intervals, the computing tasks being configured to process batch data at at least one processing layer of the model; a first determining module, configured to determine, from at least one computing task associated with the same batch of data, a specified forward computing task and a target reverse computing task, wherein the specified forward computing task and the target reverse computing task are associated with a specified processing layer of the model, and the number of intervals between the execution time intervals of the target reverse computing task and the specified forward computing task satisfies a preset condition; An adjustment module is used to adjust the execution time interval of the target reverse calculation task until the number of intervals after adjustment is less than a preset number threshold, thereby obtaining a target scheduling strategy; as well as An execution module is used to schedule the computing unit to execute computing tasks based on the target scheduling policy.

14. The device according to claim 13, wherein The ratio between the number of batches of the batch data and the number of the computing units is a non-integer.

15. The device according to claim 13, wherein N of the computing units have a unit sequence attribute, the unit sequence attribute is related to the execution order of computing tasks of the computing units, N>1, the designated forward computing task includes an Nth forward computing subtask and a loss computing task, the Nth forward computing subtask represents that the Nth computing unit processes an N-1th forward intermediate result sent by the N-1th computing unit using the designated processing layer; The loss calculation task represents that the Nth calculation unit calculates the loss value of the batch data according to the model calculation result output by the designated processing layer.

16. The device according to claim 15, wherein The target number of intervals between the specified forward computing task and the target reverse computing task in the target scheduling strategy is K, K≥0, and is less than a preset number threshold; Wherein, the execution module includes: a first execution submodule, configured to, based on a benchmark execution time interval of the specified forward computing task, cause the Nth computing unit to call the model computing result stored in the Nth storage area to execute the loss computing subtask, thereby obtaining a loss value stored in the Nth storage area; a second execution submodule, configured to, during a reference execution time interval that is K execution time intervals apart from the benchmark execution time interval, cause the Nth calculation unit to execute the target reverse calculation task based on the loss value, and obtain an Nth reverse intermediate result stored in the Nth storage area; a transmission submodule, configured to transmit the Nth reverse intermediate result to the N-1th computing unit; and The release submodule is used to release the Nth storage area.

17. The device according to claim 16, wherein The release submodule includes: A deleting unit is used to delete the loss value and the Nth reverse intermediate result stored in the Nth storage area during the process of transmitting the Nth reverse intermediate result to the N-1th calculation unit.

18. The device according to claim 16, wherein The N-1th forward intermediate result sent by the N-1th computing unit is related to the i-th batch of data; The execution module further includes: a processing submodule, configured to, in an execution time interval following the reference execution time interval, cause the Nth computing unit to process the N-1th forward intermediate result associated with the (i+1)th batch of data using the designated processing layer to obtain a model calculation result associated with the (i+1)th batch of data, wherein the model calculation result associated with the (i+1)th batch of data is stored in the Nth storage area.

19. The device according to any one of claims 16 to 18, wherein The number of execution time intervals between the specified forward computing task and the target reverse computing task in the target scheduling policy is 0.

20. The apparatus according to claim 13, wherein The adjustment module includes: a first adjustment submodule, configured to adjust the execution time interval of the target reverse computing task to obtain an Nth reverse execution time interval of the target reverse computing task, wherein the number of intervals between the Nth reverse execution time interval and the execution time interval of the specified forward computing task is less than a preset number threshold; and The second adjustment submodule is used to adjust the execution order of the target reverse computing tasks based on the Nth reverse execution time interval to obtain the target scheduling strategy.

21. The device according to claim 20, wherein N of the computing units have a unit sequence attribute, the unit sequence attribute being related to the execution order of computing tasks of the computing units, N>1, and the computing task of the Nth computing unit for the i-th batch of data includes the target reverse computing task; The second adjustment submodule includes: A first adjusting unit is configured to adjust the execution order of the target reverse computing tasks based on the Nth reverse execution time interval to obtain an intermediate scheduling strategy; and The second adjustment unit is used to adjust the execution order of the reverse calculation tasks of the n-1th calculation unit based on the execution order of the target reverse calculation tasks in the intermediate scheduling strategy to obtain the target scheduling strategy, N≥n>1.

22. The device according to claim 21, wherein The second adjustment unit includes: An adjustment subunit is used to adjust the execution order of the n-1th associated reverse calculation task of the n-1th calculation unit based on the execution order of the target reverse calculation task in the intermediate scheduling strategy, wherein the target reverse calculation task to the first associated reverse calculation task represents that multiple processing layers arranged continuously in the model execute reverse calculation tasks in sequence.

23. The apparatus according to claim 13, wherein The designated processing layer includes an attention network sublayer and a normalization sublayer.

24. The apparatus according to claim 13, wherein The batch data includes at least one of the following: Image data, audio data, text data.

25. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

26. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 12.

27. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images