A task scheduling method, device and system

Abstract

The embodiment of the application discloses a kind of task scheduling method, device and system, it is related to chip field.Can efficiently schedule AI chip task, reduce AI algorithm power waste.Specific scheme is: first, processor looks up synchronization information table according to first event identifier, determines event information, event information includes the identifier of to-be-executed event, synchronization information table includes multiple first event identifiers corresponding to at least two accelerators, and the second event identifier corresponding to at least one first event identifier in multiple first event identifiers, there is execution order between second event and first event, and second event executes after first event.Then, processor determines at least one accelerator corresponding to event information in at least two accelerators according to event information.Finally, event information is sent to at least one accelerator.

Description

Technical Field

[0001] This application relates to the field of chips, and more particularly to a task scheduling method, apparatus and system. Background Technology

[0002] Artificial intelligence (AI) computing is characterized by its computational intensity, employing systems composed of accelerators with different instruction sets and architectures for heterogeneous computing. AI chips are specifically designed to handle a large number of tasks in AI applications. How to efficiently schedule and fully utilize the computing power of AI chips is a key issue in the field of AI chip scheduling.

[0003] Currently, AI chip task scheduling is based on stream scheduling. Tasks are split and ordered into streams according to their dependencies, providing intra-stream ordering and inter-stream concurrency scheduling mechanisms. Tasks on the same stream are executed in a first-in, first-out (FIFO) order, fully utilizing the AI ​​chip's computing power through multi-stream concurrency, while also providing inter-stream synchronization scheduling mechanisms. However, when ordering tasks or splitting streams based on dependencies, the limited number of streams, the need to consider task order and concurrency, can easily lead to accelerator idleness or waiting, resulting in wasted AI computing power. Summary of the Invention

[0004] This application provides a task scheduling method, apparatus, and system that can efficiently schedule AI chip tasks and reduce the waste of AI computing power.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] A first aspect of this application provides a task scheduling method, comprising: a processor searching a synchronization information table based on a first event identifier to determine event information, the event information including an identifier of an event to be executed; the synchronization information table including multiple first event identifiers corresponding to at least two accelerators, and a second event identifier corresponding to at least one of the multiple first event identifiers; the second event and the first event have a sequential execution order, and the second event is executed after the first event; the processor, based on the event information, determines at least one accelerator corresponding to the event information from at least two accelerators; and sends the event information to the at least one accelerator.

[0007] Based on this scheme, the processor determines the identifier of the event to be executed by looking up a synchronization information table that indicates the sequential execution order of events based on the first event identifier, and then sends the identifier of the event to be executed to the accelerator. Since the sequential execution order of multiple first events corresponding to at least two accelerators in this scheme can be directly represented by the synchronization information table, tasks can be scheduled directly according to the sequential execution order of multiple tasks by looking up the synchronization information table. Compared with the prior art, where the number of streams is limited and multiple tasks need to be split into a limited number of streams, causing the processor to schedule tasks while ensuring that concurrent tasks across multiple streams are scheduled in sequential order and that multiple tasks within each stream are scheduled according to the first-in-first-out principle, resulting in some accelerators being idle or waiting and wasting AI computing power, the synchronization information table in this application can directly represent the sequential execution order of multiple events and does not introduce the principle of preserving order within a stream (multiple tasks within each stream are scheduled according to the first-in-first-out principle). Therefore, it does not cause the problem of accelerators being idle or waiting, and can reduce the waste of AI computing power.

[0008] For example, consider a scenario where Task 1 and Task 2 have a sequential execution order, with Task 2 executing after Task 1, while Task 3 has no sequential execution order with either Task 1 or Task 2. Tasks 1 and 2 are tasks for accelerator 1, and Task 3 is a task for accelerator 2. According to existing solutions, due to the limited number of streams, Task 1 can be split into stream 1, and Tasks 2 and 3 into stream 2, with Task 2 in stream 2 preceding Task 3. Therefore, when scheduling tasks according to intra-stream ordering and inter-stream concurrency, the processor schedules Task 1 first, then Task 2, and finally Task 3. Consequently, accelerator 2, which executes Task 3, must wait for Tasks 1 and 2 to complete before it can execute, resulting in idle time and wasted computing power. However, in the solution of this application, a synchronization information table is used to indicate the sequential execution order between Task 1 and Task 3. Since Task 3 has no sequential execution order with either Task 1 or Task 2, accelerator 1 can execute Task 1, and accelerator 2 can execute Task 3 simultaneously. After accelerator 1 completes Task 1, it then executes Task 2. Clearly, since the proposed solution does not introduce intra-stream order preservation, the processor can schedule tasks according to the order in which they are executed, thus preventing accelerator idleness and reducing the waste of AI computing power.

[0009] In conjunction with the first aspect, in one implementation, the method further includes: the first event is a composite event, the composite event includes N sub-events, the second event corresponding to the first event and the N sub-events have a sequential execution order, and the second event is executed after the N sub-events, where N is an integer greater than or equal to 2.

[0010] Based on this scheme, a scheduling method is provided to synchronize multiple events with a single event by combining multiple events into a composite event. The task corresponding to the composite event is executed only after the task corresponding to the next event is completed. For example, if events 1, 2, and 3 have a sequential execution order, and the task corresponding to event 3 can only be executed after the tasks corresponding to events 1 and 2 are completed, a composite event can be constructed including events 1 and 2. Once the task corresponding to the composite event is completed, the task corresponding to event 3 will then be executed. Therefore, constructing composite events can solve the task scheduling problem when multiple tasks have a sequential execution order with a single task, and can also reduce the waste of AI computing power.

[0011] In conjunction with the first aspect, in one implementation, the method further includes: the processor receiving first task completion information, which indicates that the task corresponding to the sub-event has been completed. The first sub-event is any one of N sub-events.

[0012] In response to N first task completion messages, the processor sends a second event identifier to the accelerator.

[0013] Based on this scheme, multiple events are combined into a composite event. When the task corresponding to a sub-event in the composite event is completed, the accelerator sends a first task completion message to the processor. When all N sub-events in the composite event are completed, the processor sends a second event identifier to the accelerator. By constructing composite events, the task scheduling problem when multiple tasks and a single task have a sequential execution order can be solved.

[0014] In conjunction with the first aspect, in one implementation, the method further includes: the processor adjusting the value of a counter based on the completion information of each first task. Specifically, when N sub-events have been completed, the counter value is N.

[0015] Based on this scheme, the processor continuously adjusts the value of the counter according to the first task completion information sent by the accelerator. When the tasks corresponding to the sub-events of the composite event are all completed, the processor sends the second event to the accelerator. Therefore, the task scheduling problem when multiple tasks and a single task have a sequential execution order can be solved.

[0016] In conjunction with the first aspect, in one implementation, the method further includes: the second event identifier corresponding to the first event is multiple.

[0017] Based on this solution, multiple events are combined into a composite event. Once the task corresponding to one event is completed, the task corresponding to the composite event is executed. Therefore, this solution can solve the task scheduling problem when there is a sequential execution order between the task corresponding to one event and the tasks corresponding to multiple events.

[0018] In conjunction with the first aspect, in one implementation, the method further includes: the processor receiving second task completion information, the second task completion information being used to indicate that the task corresponding to the first event has been completed.

[0019] In response to the second task completion information, the processor sends multiple second event identifiers to the accelerator.

[0020] Based on this scheme, when the task corresponding to the first event completes, the processor can send multiple second event identifiers to the accelerators. This solves the task scheduling problem when one task and multiple tasks have a sequential execution order. For example, if events 1, 2, and 3 have a sequential execution order, and the tasks corresponding to events 2 and 3 can only be executed after the task corresponding to event 1 has completed, after the task corresponding to event 1 has completed, the processor can determine the accelerators corresponding to events 2 and 3 based on their task types and send the identifiers of events 2 and 3 to those accelerators. Therefore, this scheme can solve the task scheduling problem when one task and multiple tasks have a sequential execution order.

[0021] In conjunction with the first aspect, in one implementation, the method further includes: the processor sending event information to the accelerator via a first queue according to a synchronization information table.

[0022] Based on this scheme, the processor sends event information to the accelerator through the first queue, which can provide a buffer when the number of event information is large.

[0023] In conjunction with the first aspect, in one implementation, the method further includes: the synchronization information table also includes a task identifier corresponding to each first event identifier, the task identifier being used to indicate the task corresponding to the first event identifier, and the task identifier corresponding to at least one of the multiple first event identifiers includes multiple subtask identifiers.

[0024] Based on this solution, the task corresponding to the first event can be broken down into multiple subtasks, which can be homogeneously computed by multiple accelerators of the same type, thereby improving task execution efficiency. For example, taking a matrix accelerator task corresponding to the first event as an example, this matrix accelerator task can be broken down into three matrix accelerator subtasks. These three matrix accelerator subtasks can be executed separately by three matrix accelerators in the AI ​​chip. By having these three matrix accelerators execute the three matrix accelerator subtasks corresponding to the first event simultaneously, the execution efficiency of the task corresponding to the first event can be improved.

[0025] In conjunction with the first aspect, in one implementation, the method further includes: the task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information also includes the first subtask identifier and the second subtask identifier.

[0026] Based on this scheme, the task corresponding to the first event can be broken down into multiple subtasks, which can be computed by multiple accelerators of the same type, thereby improving task execution efficiency.

[0027] A second aspect of this application provides a task scheduling method, comprising: first, an accelerator receiving event information from a processor, the event information including identifiers of events to be executed, the identifiers of which are determined by the processor according to a synchronization information table, the synchronization information table indicating the synchronization relationship between multiple tasks corresponding to at least two accelerators. The synchronization information table includes multiple first event identifiers, a task identifier corresponding to each first event identifier, and a second event identifier corresponding to at least one of the multiple first event identifiers, wherein the second event has a sequential execution order with the first event, and the second event is executed after the first event. Then, the accelerator obtains the task to be executed according to the identifier of the event to be executed. Finally, the accelerator executes the task to be executed.

[0028] Based on this scheme, the accelerator acquires and executes tasks to be executed according to event information. Compared to existing technologies that use stream scheduling, which require splitting the computational tasks of multiple accelerators into streams, thus causing accelerators to idle or wait when the number of streams is limited, resulting in wasted AI computing power, this application's embodiment schedules tasks directly based on the synchronization information table corresponding to the synchronization relationship between the computational tasks of multiple accelerators, without splitting the computational tasks of multiple accelerators into streams. Therefore, it does not introduce the intra-stream order preservation mechanism, thus enabling efficient task scheduling and reducing the waste of AI computing power.

[0029] In conjunction with the second aspect, in one implementation, the method further includes: the first event is a composite event, the composite event includes N sub-events, the second event and the first event have a sequential execution order, and the second event is executed after the first event, where N is an integer greater than or equal to 2.

[0030] Based on this solution, by combining multiple events into a composite event, the task corresponding to the next event is executed only after the task corresponding to the composite event is completed. This provides a scheduling method that synchronizes multiple events with a single event, which can solve the task scheduling problem when multiple tasks and a single task have a sequential execution order.

[0031] In conjunction with the second aspect, in one implementation, the method further includes: after the accelerator completes the task corresponding to each sub-event, the accelerator sends a first task completion message to the processor. When the tasks corresponding to N sub-events are completed, the accelerator receives a second event identifier from the processor.

[0032] Based on this solution, by combining multiple events into a composite event, the task corresponding to the next event is executed only after the task corresponding to the composite event is completed. This provides a scheduling method that synchronizes multiple events with a single event, which can solve the task scheduling problem when multiple tasks and a single task have a sequential execution order.

[0033] In conjunction with the second aspect, in one implementation, the method further includes: the second event identifier corresponding to the first event is multiple.

[0034] Based on this scheme, when the task corresponding to the first event is completed, the accelerator sends the second task completion information to the processor. The processor can send multiple second event identifiers to the accelerator according to the second completion information and the synchronization information table, which can solve the task scheduling problem when one task and multiple tasks have a sequential execution order.

[0035] In conjunction with the second aspect, in one implementation, the method further includes: upon completion of the task corresponding to the first event by the accelerator, the accelerator sends second task completion information to the processor. The accelerator receives multiple second event identifiers from the processor.

[0036] Based on this scheme, when the task corresponding to the first event is completed, the accelerator sends the second task completion information to the processor. The processor can send multiple second event identifiers to the accelerator according to the second completion information and the synchronization information table, which can solve the task scheduling problem when one task and multiple tasks have a sequential execution order.

[0037] In conjunction with the second aspect, in one implementation, the method further includes: the accelerator sending second task completion information to the processor via a second queue.

[0038] Based on this scheme, the accelerator sends event information to the processor through a second queue, which can provide a buffer when there are a large number of task completion messages.

[0039] In conjunction with the second aspect, in one implementation, the method further includes: the synchronization information table also includes a task identifier corresponding to each first event identifier, the task identifier being used to indicate the task corresponding to the first event identifier, and the task identifier corresponding to at least one of the multiple first event identifiers includes multiple subtask identifiers.

[0040] Based on this scheme, the task corresponding to the first event can be broken down into multiple subtasks, which can be computed by multiple accelerators of the same type, thereby improving task execution efficiency.

[0041] In conjunction with the second aspect, in one implementation, the method further includes: the task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information also includes the first subtask identifier and the second subtask identifier.

[0042] Based on this scheme, the task corresponding to the first event can be broken down into multiple subtasks, which can be computed by multiple accelerators of the same type, thereby improving task execution efficiency.

[0043] In conjunction with the second aspect, in one implementation, the method further includes: the identifier of the event to be executed includes a first subtask identifier or a second subtask identifier; the accelerator obtains the task to be executed based on the identifier of the event to be executed, including:

[0044] The accelerator retrieves the first subtask based on the first subtask identifier. Alternatively, the accelerator retrieves the second subtask based on the second subtask identifier. The accelerator executes the task to be executed, including: the accelerator executing the first subtask, or the accelerator executing the second subtask.

[0045] Based on this scheme, the task corresponding to the first event can be broken down into multiple subtasks. Multiple accelerators of the same type can execute any subtask simultaneously for isomorphic computation, which can improve task execution efficiency.

[0046] A third aspect of this application provides a task scheduling device, which includes a processing module and a transceiver module.

[0047] The processing module is used to look up the synchronization information table for the first event identifier, determine the event information, the event information includes the identifier of the event to be executed, the synchronization information table includes multiple first event identifiers, and a second event identifier corresponding to at least one of the multiple first event identifiers, the second event and the first event have a sequential execution order, and the second event is executed after the first event.

[0048] The processing module is also configured to determine, based on the event information, at least one accelerator among at least two accelerators that corresponds to the event information.

[0049] The transceiver module is used to send event information to at least one accelerator.

[0050] In conjunction with the third aspect, in one implementation, the first event is a composite event, which includes N sub-events. The second event corresponding to the first event and the N sub-events have a sequential execution order, and the second event is executed after the N sub-events, where N is an integer greater than or equal to 2.

[0051] In conjunction with the third aspect, in one implementation, the transceiver module is further configured to receive first task completion information, which indicates that the task corresponding to the first sub-event has been completed. The processing module is further configured to, in response to N first task completion messages, send a second event identifier to the accelerator via the transceiver module.

[0052] In conjunction with the third aspect, in one implementation, the processing module is specifically used to adjust the counter value based on the completion information of each first task. Specifically, when the tasks corresponding to N sub-events are completed, the counter value is N.

[0053] In conjunction with the third aspect, in one implementation, the second event identifier corresponding to the first event can be multiple.

[0054] In conjunction with the third aspect, in one implementation, the transceiver module is also used to receive second task completion information, which is used to indicate that the task corresponding to the first event has been completed.

[0055] The processing module is also used to send multiple second event identifiers to the accelerator via the transceiver module in response to the second task completion information.

[0056] In conjunction with the third aspect, in one implementation, the processing module is specifically used to send event information to the accelerator through the first queue according to the synchronization information table.

[0057] In conjunction with the third aspect, in one implementation, the synchronization information table also includes a task identifier corresponding to each first event identifier. The task identifier is used to indicate the task corresponding to the first event identifier, and the task identifier corresponding to at least one of the multiple first event identifiers includes multiple subtask identifiers.

[0058] In conjunction with the third aspect, in one implementation, the task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information also includes the first subtask identifier and the second subtask identifier.

[0059] A fourth aspect of this application provides a task scheduling device, which includes a transceiver module and a processing module. The transceiver module receives event information from a processor. The event information includes an identifier of an event to be executed. The identifier of the event to be executed is determined by the processor by looking up a synchronization information table based on a first event identifier. The synchronization information table includes multiple first event identifiers corresponding to at least two accelerators, and a second event identifier corresponding to at least one of the multiple first event identifiers. The second event and the first event have a sequential execution order, and the second event is executed after the first event. The processing module obtains the task to be executed through the transceiver module based on the identifier of the event to be executed. The processing module is also used to execute the task to be executed.

[0060] In conjunction with the fourth aspect, in one implementation, the first event is a composite event, which includes N sub-events. The second event corresponding to the first event and the N sub-events have a sequential execution order, and the second event is executed after the N sub-events, where N is an integer greater than or equal to 2.

[0061] In conjunction with the fourth aspect, in one implementation, the transceiver module is used to send first task completion information to the processor after the processor has completed the task corresponding to each sub-event.

[0062] The transceiver module is also used to receive a second event identifier from the processor when the processing module completes the tasks corresponding to N sub-events.

[0063] In conjunction with the fourth aspect, in one implementation, the second event identifier corresponding to the first event can be multiple.

[0064] In conjunction with the fourth aspect, in one implementation, the transceiver module is further configured to send second task completion information to the processing module after the processing module has completed the task corresponding to the first event. The transceiver module is also configured to receive multiple second event identifiers from the processor.

[0065] In conjunction with the fourth aspect, in one implementation, the transceiver module is specifically used to send the second task completion information to the processor through the second queue.

[0066] In conjunction with the fourth aspect, in one implementation, the synchronization information table also includes a task identifier corresponding to each first event identifier. The task identifier is used to indicate the task corresponding to the first event identifier, and the task identifier corresponding to at least one of the multiple first event identifiers includes multiple subtask identifiers.

[0067] In conjunction with the fourth aspect, in one implementation, the task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information also includes the first subtask identifier and the second subtask identifier.

[0068] In conjunction with the fourth aspect, in one implementation, the identifier of the event to be executed includes a first subtask identifier or a second subtask identifier. The processing module is specifically used to obtain the first subtask based on the first subtask identifier. Alternatively, the processing module is specifically used to obtain the second subtask based on the second subtask identifier. The processing module is specifically used to execute the first subtask, or the processing module is specifically used to execute the second subtask.

[0069] A fifth aspect of this application provides an artificial intelligence chip, which includes a memory, a processor, and an accelerator, and the memory, processor, and accelerator are coupled together. The memory stores computer program code, which includes computer instructions. When the processor executes the computer instructions, the artificial intelligence chip executes the task scheduling method provided in the first and second aspects described above.

[0070] A sixth aspect of this application provides a computer-readable storage medium including instructions. When the instructions are executed on a computer, they cause the computer to perform the methods provided in the first and second aspects described above.

[0071] According to a seventh aspect of this application, a task scheduling system is provided, which includes the task scheduling device described in any of the third aspects and the task scheduling device described in any of the fourth aspects.

[0072] An eighth aspect of this application provides a computer program product. When the computer program product is run on a computer, it causes the computer to perform the methods provided in the first and second aspects described above.

[0073] The descriptions of the third, fourth, fifth, sixth, seventh, and eighth aspects in this application can be referenced to the detailed descriptions of the first and second aspects; and the beneficial effects of the third, fourth, fifth, sixth, seventh, and eighth aspects can be referenced to the beneficial effect analysis of the first and second aspects, which will not be repeated here. Attached Figure Description

[0074] Figure 1 A flowchart illustrating a task scheduling method provided in an embodiment of this application;

[0075] Figure 2 This is a schematic diagram of the structure of a task scheduling system provided in an embodiment of this application;

[0076] Figure 3 A system architecture diagram illustrating an application scenario of a task scheduling method provided in this application embodiment;

[0077] Figure 4 System architecture diagram for another application scenario of the task scheduling method provided in the embodiments of this application;

[0078] Figure 5 A flowchart illustrating another task scheduling method provided in an embodiment of this application;

[0079] Figure 6 A schematic diagram of another task scheduling system structure provided in this application embodiment;

[0080] Figure 7 A flowchart illustrating yet another task scheduling method provided in an embodiment of this application;

[0081] Figure 8 This is a schematic diagram of a task composition structure provided in an embodiment of this application;

[0082] Figure 9 A schematic diagram of yet another task scheduling system structure provided in this application embodiment;

[0083] Figure 10 A flowchart illustrating another task scheduling method provided in an embodiment of this application;

[0084] Figure 11 A schematic diagram of another task scheduling system structure provided in the embodiments of this application;

[0085] Figure 12 A schematic diagram of another task scheduling system structure provided in the embodiments of this application;

[0086] Figure 13 A flowchart illustrating another task scheduling method provided in an embodiment of this application;

[0087] Figure 14 This is a schematic diagram of the composition of a task scheduling device provided in an embodiment of this application;

[0088] Figure 15 This is a schematic diagram illustrating the composition of another task scheduling device provided in an embodiment of this application. Detailed Implementation

[0089] The technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings. In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple. Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order. For example, in the embodiments of this application, "first" in the first queue and "second" in the second queue are only used to distinguish different queues. The descriptions of "first" and "second" appearing in the embodiments of this application are only for illustration and to distinguish the described objects, and do not indicate any order, nor do they represent a special limitation on the number of devices in the embodiments of this application, and cannot constitute any limitation on the embodiments of this application.

[0090] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0091] Figure 1 This is a schematic diagram illustrating a stream-based AI chip task scheduling method provided in an embodiment of this application. Figure 1As shown, the AI ​​chip includes accelerators such as a cube core (CUBE), a vector core (VEC), and an artificial intelligence central processing unit (AICPU). When scheduling tasks on the AI ​​chip, the processor can split the tasks into three streams (Stream 1, Stream 2, and Stream 3) based on the dependencies between multiple tasks. Each stream can contain multiple tasks, which can be executed by the CUBE, VEC, and AICPU respectively. When scheduling tasks within each stream, the processor can distribute tasks to the accelerators sequentially according to the first-in, first-out (FIFO) principle (this scheduling principle can be called intra-stream order preservation). When scheduling tasks across multiple streams, if the tasks in multiple streams support concurrency, the computing power of the AI ​​chip can be fully utilized through concurrency.

[0092] For example, taking an AI chip with two matrix accelerators as an example, matrix accelerator tasks in stream 1 and stream 3 can be concurrently used to fully utilize the computing power of the AI ​​chip. As another example, taking an AI chip with one AICPU and one appropriate number of accelerators as an example, AICPU and appropriate number of accelerator tasks in different streams can be concurrently used to fully utilize the computing power of the AI ​​chip. This task scheduling method also provides an inter-stream synchronization mechanism to synchronize task processing.

[0093] Based on the scheduling principles described above, task scheduling requires splitting streams and tasks according to their dependencies. However, due to the limited number of streams, when splitting tasks into a finite number of streams, the scheduling of multiple tasks within each stream must follow a first-in, first-out (FIFO) principle. Therefore, if task splitting is not done properly, it can easily lead to accelerator idleness or waiting, resulting in a waste of AI computing power. Moreover, the larger the number of tasks, the more severe the waste of AI computing power may be.

[0094] For example, such as Figure 1As shown, taking the example that the Optimizer 2 task in Stream 2 needs to wait for the Direct Memory Access (DMA) 1 task in Stream 1 to be sent before it can be sent, the scheduling of multiple tasks within each stream must follow the first-in, first-out (FIFO) scheduling principle. Therefore, the Matrix Accelerator 1 task in Stream 2 can only be sent to Matrix Accelerator 1 after the Optimizer 2 task and the DMA task in Stream 2 are sent. If the Matrix Accelerator 1 task and the Optimizer 2 task in Stream 2 do not have a synchronization relationship, but because the in-stream ordering mechanism is introduced when splitting tasks into streams, the Matrix Accelerator 1 task in Stream 2 will have to wait for the Optimizer 2 task and the DMA task in Stream 2 to be sent before it can be sent to Matrix Accelerator 2. Furthermore, there is a synchronization relationship between the Optimizer 2 task in Stream 2 and the DMA task in Stream 1. This will cause the Matrix Accelerator 1 task in Stream 2 to wait for the Matrix Accelerator 1 task, Optimizer 1 task, Direct Memory Access 1 task in Stream 1, and Optimizer 2 task and Direct Memory Access 2 task in Stream 2 to be issued before it can be issued, resulting in the Matrix Accelerator 1 being idle or waiting, leading to a waste of AI computing power.

[0095] To address the problem that stream-based scheduling can easily lead to accelerators being idle or waiting, resulting in wasted AI computing power, this application provides a task scheduling method, apparatus, and system that can reduce the waste of AI computing power by scheduling tasks from multiple accelerators according to a synchronization information table.

[0096] Figure 2 This is a schematic diagram of a task scheduling system provided in an embodiment of this application. Figure 2 As shown, the processor constructs a synchronization information table based on the synchronization relationship between multiple tasks corresponding to multiple accelerators. This synchronization information table is used to indicate the synchronization relationship between the multiple tasks corresponding to the multiple accelerators. The processor can be a microcontroller unit (MCU), a central processing unit (CPU), or other processors. This application embodiment does not limit the specific type of processor; the following embodiment uses an MCU as an example for illustrative purposes.

[0097] When scheduling tasks, the processor first determines the identifier of the event to be executed based on the synchronization information table, and pushes the identifier of the event to be executed to the accelerator through the event queue. Then, the accelerator obtains the task information to be executed based on the identifier of the event to be executed. When the accelerator finishes executing the task, it sends task completion information to the processor. The processor then pushes the next event to the accelerator based on the task completion information.

[0098] like Figure 2As shown, the processor determines the identifier of the event to be executed based on the synchronization information table and pushes the identifier of the event to be executed to the matrix accelerator through the matrix accelerator event queue. The identifier of the event to be executed includes event 1 and event 2. The matrix accelerator obtains the task to be executed based on event 1 and event 2. When the task to be executed is completed, the matrix accelerator sends task completion information to the processor. The processor then pushes the next event to the matrix accelerator based on the task completion information. Alternatively, the processor can push the identifier of the event to be executed to an appropriate accelerator through an appropriate accelerator event queue. The identifier of the event to be executed includes event 8. The appropriate accelerator obtains the task to be executed based on event 8. When the task to be executed is completed, the appropriate accelerator sends task completion information to the processor. The processor then pushes the next event to the appropriate accelerator based on the task completion information. The processor can also push the identifier of the event to be executed to the AICPU through the AICPU event queue. The identifier of the event to be executed includes event 12. The AICPU obtains the task to be executed based on event 12. When the task to be executed is completed, the AICPU sends task completion information to the processor. The processor then pushes the next event to the AICPU based on the task completion information.

[0099] like Figure 3As shown in the embodiments of this application, the task scheduling method can be applied to AI training scenarios. The AI ​​training application system architecture includes a host system (Host) and a device system (Device). The host includes an AI framework adaptation layer, programming languages, operator libraries, drivers, and an operating system. The AI ​​framework adaptation layer is used to interface with various different AI training frameworks, such as: a front-end representation layer, tensor streams, and crawler data packages (Python PyTorch, PyTorch) that provide two functions. The programming languages ​​and operator libraries provide open software interfaces for programming languages ​​and operators based on AI processor hardware. For example, the programming languages ​​include function libraries, graphics engines, ensemble communication library engines, fusion engines, AICPU engines, tensor accelerator engines, and a runtime manager. The operator libraries include AI Core operators, AICPU operators, ensemble communication operators, and main CPU operators. The drivers and operating system provide a basic runtime environment, enabling data interaction with the device. The device provides scheduling and execution capabilities for heterogeneous computing tasks, utilizing local memory for efficient heterogeneous computing and data preprocessing, and also has the ability to schedule computing resources on the host side. The device includes an AI processor and a main CPU. The AI ​​processor includes a task scheduler, an AICore, a digital preprocessing accelerator, a control CPU, and double-data-rate synchronous dynamic random access memory. The main CPU includes a high-speed serial computer expansion bus and a network interface controller. The task scheduling method provided in this application embodiment can be executed by the task scheduler in the AI ​​processor. The task scheduler can schedule computing tasks, utilize local memory for efficient heterogeneous computing and data preprocessing, and can also schedule computing resources on the host side to execute computing tasks.

[0100] like Figure 4 As shown in the embodiments of this application, the task scheduling method can also be applied to inference scenarios. The AI ​​inference application system architecture includes a Host and a Device. The Host includes a programming language, a driver, and an operating system. The Host provides a programming language based on the AI ​​processor hardware, such as a computer language preprocessing function set, a computer language matrix operation function set, a computer language function set, and a runtime function set. The driver and operating system provide a basic runtime environment, enabling data interaction with the Device. The Device provides scheduling and execution capabilities for heterogeneous computing tasks, utilizing local memory for efficient heterogeneous computing and data preprocessing. The Device includes an AI processor, and the IA processor includes a task scheduler, AI Core, AI CPU, data preprocessing, a control CPU, and double-rate synchronous dynamic random access memory. The task scheduling method provided in the embodiments of this application can be executed by the task scheduler in the AI ​​processor. The task scheduler can schedule computing tasks, utilizing local memory for efficient heterogeneous computing and data preprocessing.

[0101] Figure 5 This application provides a task scheduling method, which can be executed by a task scheduling device, such as... Figure 5 As shown, the method includes the following steps S501-S502.

[0102] S501, The processor searches the synchronization information table based on the first event identifier to determine the event information. The event information includes the identifier of the event to be executed. The synchronization information table includes multiple first event identifiers corresponding to at least two accelerators, and a second event identifier corresponding to at least one of the multiple first event identifiers. There is a sequential execution order between the second event and the first event, and the second event is executed after the first event.

[0103] Optionally, the processor can be a general-purpose processor, such as an MCU or a CPU. This application does not limit the specific type of processor; the following embodiments use an MCU as an example for illustrative purposes.

[0104] The number of at least two accelerators mentioned above can be the number of accelerators included in the AI ​​chip. These at least two accelerators may be used to perform the same type of computing task, or they may be used to perform different types of computing tasks. The number of at least two accelerators can be two or more. This application embodiment does not limit the specific type and number of the at least two accelerators.

[0105] For example, if an AI chip includes two accelerators, both accelerators can be cubes. If an AI chip includes eight accelerators, three of them can be cubes and five can be VECs. If an AI chip includes three accelerators, one accelerator is a cube, one is a VEC, and one is an AICPU.

[0106] Optionally, the synchronization information table may also include a task identifier corresponding to each first event identifier, which is used to indicate the task corresponding to the first event identifier.

[0107] In the synchronization information table, the task corresponding to the first event identifier and the task corresponding to the second event identifier can be the same type of task or different types of task. In this application embodiment, the specific type of task corresponding to the first event identifier and the second event identifier is not limited.

[0108] For example, the task corresponding to the first event identifier can be a CUBE task, and the task corresponding to the second event identifier can also be a CUBE task. Alternatively, the task corresponding to the first event identifier can be a CUBE task, and the task corresponding to the second event identifier can be a VEC task.

[0109] The first event identifier may correspond to one task identifier. Alternatively, the first event identifier may correspond to one task identifier and one second event identifier. Alternatively, the first event identifier may correspond to one task identifier and multiple second event identifiers. This application embodiment does not limit the specific number of second event identifiers that the first event identifier corresponds to.

[0110] When a first event identifier corresponds to a second event identifier, the second event is executed after the first event. That is, the second event can only be executed after the first event has finished. When a first event identifier corresponds to multiple second event identifiers, these multiple second events are executed after the first event. That is, these multiple second events can only be executed after the first event has finished.

[0111] Optionally, the synchronization information table may also include information such as task parameters. These task parameters may include, but are not limited to, the program counter (PC) register and parameters required to execute the task; the program counter is used to store instruction addresses.

[0112] For example, taking the synchronization information table, Table 1, as shown in Table 1, the first event identifier 1 corresponds to task identifier 1, and task identifier 1 is the identifier of the task to be executed corresponding to the first event identifier 1. The first event identifier 1 also corresponds to the second event identifier 2 and the second event identifier 3. That is, the tasks corresponding to the second event identifier 2 and the second event identifier 3 can only be executed after the task corresponding to the first event identifier 1 is completed. Task identifier 1 can correspond to task parameters program counter 1 and parameter 1.

[0113] For example, as shown in Table 1, the first event identifier 3 corresponds to task identifier 3, meaning task identifier 3 is the task identifier to be executed corresponding to the first event identifier 3. The first event identifier 3 does not have a corresponding second event identifier; therefore, no task needs to be executed after task 3 corresponding to the first event identifier 3.

[0114] Table 1

[0115]

[0116] The first event can be a composite event. When the first event is a composite event, it can include N sub-events. These N sub-events are synchronized with the corresponding second event, where N is an integer greater than or equal to 2. Synchronization between the N sub-events and the corresponding second event means that the second event executes after the N sub-events. That is, after the N sub-events are executed, the next event to be executed is the second event. This application does not limit the specific number of sub-events corresponding to the composite event. The number of sub-events included in the composite event can be related to the number of accelerators included in the AI ​​chip; the more accelerators the AI ​​chip includes, the more sub-events the first event can include.

[0117] Optionally, when a composite event includes a maximum of M sub-events, if the actual number of events is greater than M (e.g., N is greater than M), a correspondence between multiple layers of composite events can be added.

[0118] For example, taking a composite event that includes a maximum of four sub-events as an example, the relationship between multiple composite events can be represented by the multi-event synchronization table shown in Table 2. Composite event X can include two sub-events, namely event F and event G. Composite event Y includes five sub-events, namely event H, event I, event J, event K, and event L. However, since a maximum of four sub-events are supported, an additional layer of composite event correspondence can be added. For example, in the first layer of composite event correspondence, composite event Y includes four sub-events, namely event H, event I, event J, and composite event Z. In the second layer of composite event correspondence, composite event Z includes two sub-events, namely event K and event L.

[0119] Table 2

[0120] Compound events subevent subevent subevent subevent Composite Event X Event F Event G Composite event Y Event H Event I Event J Composite Event Z Composite Event Z Event K Event L

[0121] Optionally, among the N sub-events included in the composite event, the tasks corresponding to each sub-event can be of the same type or different types of computational tasks. This application embodiment does not limit the specific types of computational tasks corresponding to the N sub-events included in the composite event.

[0122] For example, taking a composite event that includes two sub-events as an example, the tasks corresponding to the two sub-events can both be CUBE tasks, or one of the tasks corresponding to the two sub-events can be a CUBE task and the other can be a VEC task.

[0123] It is understood that the synchronization information table in this application may include Table 1 and Table 2. The multi-event synchronization table shown in Table 2 may be combined with Table 1 in the same table (i.e., the synchronization information table), or the multi-event synchronization table shown in Table 2 may be independent of Table 1.

[0124] The aforementioned event information includes the identifiers of the events to be executed. For example, as shown in Table 1, when task 1 corresponding to the first event identifier 1 is completed, the processor can determine the identifiers of the events to be executed as the second event identifier 2 and the second event identifier 3 based on the synchronization information table shown in Table 1.

[0125] When there are multiple accelerators of the same type, the task identifier corresponding to the identifier of the event to be executed can include multiple subtask identifiers, which can be executed synchronously by multiple accelerators. For example, taking the task identifier corresponding to the identifier of the event to be executed as including a first subtask identifier and a second subtask identifier, the event information also includes the first subtask identifier and the second subtask identifier. The first subtask can be executed by CUBE1, and the second subtask can be executed by CUBE2.

[0126] Optionally, before step S501, the processor may further acquire the aforementioned synchronization information table. Acquiring the synchronization information table may include the processor generating the synchronization information table based on the synchronization relationships between multiple tasks corresponding to at least two accelerators, or the processor receiving the synchronization information table. This application embodiment does not limit the specific method by which the processor acquires the synchronization information table.

[0127] For example, such as Figure 6 As shown, the synchronization information table can be stored in... Figure 6 In the general memory shown, before the MCU determines the identifier of the event to be executed based on the synchronization information table, it can initiate a read operation to prefetch the synchronization information table from memory into a cache (e.g., a cache). The MCU can then read the event information from the cache.

[0128] Compared to existing technologies that require splitting the computational tasks of multiple accelerators into streams when scheduling tasks based on streams, which can lead to accelerator idleness or waiting when the number of streams is limited, resulting in wasted AI computing power, this application's embodiment schedules tasks directly based on the synchronization information table corresponding to the synchronization relationship between the computational tasks of multiple accelerators. It does not split the computational tasks of multiple accelerators into streams, thus avoiding the introduction of intra-stream order preservation. Therefore, it can efficiently schedule tasks and reduce the waste of AI computing power.

[0129] S502. Based on the event information, the processor determines at least one accelerator corresponding to the event information among at least two accelerators, and sends the event information to the at least one accelerator.

[0130] The aforementioned at least one accelerator is an accelerator used to perform the same type of computing task, and the number of at least one accelerator refers to the number of accelerators of that type included in the AI ​​chip. This application embodiment does not limit the number of at least one accelerator.

[0131] Based on the task corresponding to the event information, the processor determines at least one accelerator among at least two accelerators that corresponds to the task, and sends the event information to the at least one accelerator.

[0132] For example, taking an AI chip that includes two accelerators, one of which is a matrix accelerator and the other is a quantitative accelerator, the processor determines the matrix accelerator corresponding to the matrix accelerator task among the two accelerators included in the AI ​​chip according to the matrix accelerator task corresponding to the event information, and sends the event information to that matrix accelerator.

[0133] The processor sends event information to the accelerator, including sending the event information to the accelerator through a first queue. Each accelerator can correspond to a first queue, and the first queues corresponding to different accelerators can be the same or different.

[0134] For example, such as Figure 6 As shown, the MCU can send event information to the accelerator through a first queue. The first queue is an event queue; when the event information includes a large number of first event identifiers, the first queue can be used to buffer these multiple first event identifiers. Optionally, the first queue can be a single register used to buffer event queues corresponding to different accelerators; alternatively, the first queue can be multiple registers, each used to buffer the event queue corresponding to one accelerator. In this embodiment, the specific type and number of the first queue are not limited.

[0135] When the processor sends event information to the accelerator, depending on the number of event identifiers to be executed and the number of subtask identifiers in the event information, the processor can send the corresponding event information to different types of accelerators, or to multiple different accelerators of the same type. Optionally, the accelerator can be a CUBE, VEC, or AICPU, or other types of accelerators. This application embodiment does not limit the specific type of accelerator.

[0136] For example, referring to Table 1, taking an AI chip that includes two matrix accelerators, namely matrix accelerator 1 and matrix accelerator 2, and the event identifiers to be executed as first event identifier 2 and first event identifier 3, the processor can send first event identifier 2 to matrix accelerator 1 and first event identifier 3 to matrix accelerator 2 according to the synchronization information table.

[0137] Optionally, when the processor sends multiple event identifiers to at least two accelerators, the tasks corresponding to these multiple event identifiers can be of the same type or different types of computing tasks. This application embodiment does not limit the specific type of the tasks corresponding to these multiple event identifiers.

[0138] For example, if there are 8 event identifiers to be executed, the tasks corresponding to these 8 event identifiers can be 4 CUBE tasks and 4 VEC tasks, or the tasks corresponding to these 8 event identifiers can be 4 CUBE tasks and 4 AICPU tasks, or the tasks corresponding to these 8 first event identifiers can be 8 CUBE tasks.

[0139] like Figure 7 As shown, the task scheduling method provided in this application embodiment further includes steps S503-S505 after the above steps S501-S502.

[0140] S503, the accelerator receives event information from the processor.

[0141] like Figure 6 The accelerator receives event information from the processor through a first queue, or the accelerator can directly receive event information from the processor. This application embodiment does not limit how the accelerator specifically receives event information.

[0142] S504. The accelerator obtains the task to be executed based on the identifier of the event to be executed.

[0143] The accelerator obtains the task to be executed corresponding to the identifier of the event to be executed and the synchronization information table.

[0144] For example, such as Figure 6 As shown, the accelerator reads the task to be executed corresponding to the identifier of the event to be executed from the cache.

[0145] When the task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information also includes a first subtask identifier and a second subtask identifier, the accelerator obtains the task to be executed based on the identifier of the event to be executed, including: the accelerator obtains the first subtask based on the first subtask identifier; or, the accelerator obtains the second subtask based on the second subtask identifier.

[0146] For example, taking a 2-matrix accelerator as an example, such as Figure 8As shown, the task identifier B corresponding to the event to be executed A can be split into a first subtask and a second subtask by configuring the block identifier BLOCKID when constructing the synchronization information table. The identifiers corresponding to the first subtask and the second subtask can be C and D, respectively. Matrix accelerator 1 can obtain the first subtask based on the first subtask identifier C, and matrix accelerator 2 can obtain the second subtask based on the first subtask identifier D.

[0147] S505, the accelerator executes the task to be performed.

[0148] When the task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information includes a first subtask identifier and a second subtask identifier, the accelerator executes the task to be executed, including: the accelerator executes the first subtask, or the accelerator executes the second subtask.

[0149] For example, such as Figure 9 As shown, taking an AI chip comprising two matrix accelerators, namely Matrix Accelerator 1 and Matrix Accelerator 2, and assuming the task to be executed corresponding to the first event includes a first subtask and a second subtask, one matrix accelerator (e.g., Matrix Accelerator 1) can execute the first subtask. After completing the first subtask, Matrix Accelerator 1 sends a task completion message to the processor, and the counter in the processor is incremented by one. The other matrix accelerator (e.g., Matrix Accelerator 2) can execute the second subtask. After completing the second subtask, Matrix Accelerator 2 sends a task completion message to the processor, and the counter in the processor is incremented by one again. When the counter value is two, the processor determines that both the first and second subtasks corresponding to the first event have been completed, and the processor sends the next event identifier to the accelerators according to the synchronization information table.

[0150] The task scheduling method provided in this application involves a processor determining event information based on a synchronization information table generated from the synchronization relationships between multiple tasks corresponding to at least two accelerators, and sending this event information to the accelerators. The accelerators then acquire and execute the tasks to be executed based on the event information. Compared to existing technologies that use stream-based task scheduling, which require splitting the computational tasks of multiple accelerators into streams, thus causing accelerators to idle or wait when the number of streams is limited, resulting in wasted AI computing power, this application's method directly schedules tasks based on the synchronization information table corresponding to the synchronization relationships between the computational tasks of multiple accelerators, without splitting the computational tasks into streams. Therefore, it avoids introducing the need for intra-stream order preservation, enabling efficient task scheduling and reducing the waste of AI computing power. Furthermore, during the execution of the tasks by the accelerators, dividing the tasks into multiple sub-tasks and having multiple accelerators of the same type perform isomorphic computation can improve task computation efficiency.

[0151] like Figure 10 As shown, when the first event is a composite event, the task scheduling method provided in this application embodiment further includes steps S506-S509 after the above step S505.

[0152] S506. After the accelerator completes the task corresponding to each sub-event, it sends the first task completion information to the processor.

[0153] The first task completion information is used to indicate that the accelerator has completed the sub-event.

[0154] For example, such as Figure 11 As shown, taking the first event as a composite event, this composite event includes four sub-events: Event 1, Event 2, Event 3, and Event 4. The AI ​​chip includes four accelerators, and the tasks corresponding to these four sub-events are executed by the four accelerators respectively. For example, after accelerator 1 completes the task corresponding to Event 1, it sends the first task completion information 1 to the processor. After accelerator 2 completes the task corresponding to Event 2, it sends the first task completion information 2 to the processor. After accelerator 2 completes the task corresponding to Event 3, it sends the first task completion information 3 to the processor. After accelerator 2 completes the task corresponding to Event 4, it sends the first task completion information 4 to the processor.

[0155] like Figure 12 As shown, the accelerator can send the first task completion information to the MCU through the second queue. The second queue is a completion queue; when there are many first task completion messages, the second queue can be used to buffer these multiple messages. This second queue can be a register.

[0156] S507, The processor receives the first task completion information.

[0157] The processor adjusts the counter value based on the completion information of the first task. For example, such as Figure 11 As shown, when the processor receives the first task completion information 1, it adjusts the counter value to 1. When the processor receives the first task completion information 2, it increments the counter value by 1, making the counter value 2. When the processor receives the first task completion information 3, it increments the counter value by 1 again, making the counter value 3. When the processor receives the first task completion information 4, it increments the counter value by 1 again, making the counter value 4.

[0158] like Figure 12 As shown, the processor can receive the first task completion information through the second queue, and the first completion information indicates that the accelerator has completed the sub-event.

[0159] S508: Based on the first task completion information, the processor sends a second event identifier to the accelerator after N sub-events have been completed.

[0160] When N sub-events are completed, the counter value is N.

[0161] For example, such as Figure 11 As shown, after the accelerator completes each sub-event, it sends the first task completion information to the processor. The processor increments the counter value by one based on the first task completion information. When the accelerator completes the tasks corresponding to the four sub-events from event 1 to event 4, the counter value in the processor is 4. The processor then determines the four sub-events and sends the next event identifier to the accelerator.

[0162] Optionally, the second event can be a single event or a composite event. The specific type of the second event is not limited in the embodiments of this application.

[0163] S509, the accelerator receives a second event identifier from the processor.

[0164] For example, such as Figure 12 As shown, when the accelerator has completed N sub-events, the accelerator can receive the second event identifier from the processor through the first queue.

[0165] The task scheduling method provided in this application provides a scheduling method that synchronizes multiple events with one event when the first event is a composite event. This method solves the problem of task execution order when multiple tasks have a synchronization relationship with one task.

[0166] like Figure 13As shown, when a task has a synchronization relationship with multiple tasks, the second event identifier corresponding to the first event is multiple. The task scheduling method provided in this application embodiment further includes steps S510-513 after the above step S505.

[0167] S510. If the accelerator completes the task corresponding to the first event, the accelerator sends the second task completion information to the processor.

[0168] The second task completion information is used to indicate that the first event has been completed.

[0169] For example, such as Figure 12 As shown, when the accelerator executes the first event, the accelerator can send the second task completion information to the processor through the second queue.

[0170] S511, The processor receives the second task completion information.

[0171] For example, such as Figure 12 As shown, the processor can receive the second task completion information through the second queue.

[0172] S512, the processor sends multiple second event identifiers to the accelerator based on the second task completion information.

[0173] Optionally, based on the second person's completion information and the synchronization information table, the processor can send multiple second event identifiers to accelerators of the same type, where the tasks corresponding to these multiple second event identifiers are of the same type. Alternatively, the processor can send multiple second event identifiers to accelerators of different types, where the tasks corresponding to these multiple second event identifiers are of different types.

[0174] For example, as shown in Table 1, taking the first event as the event corresponding to the first event identifier 1, the first event identifier 1 corresponds to two second event identifiers: second event identifier 2 and second event identifier 3. When the accelerator completes task 1, it sends second task completion information to the processor. Based on the second task completion information, the processor sends the second event identifiers 2 and 3 to the accelerator. When the tasks corresponding to second event identifiers 2 and 3 are both matrix accelerator tasks, the processor sends second event identifiers 2 and 3 to both matrix accelerators. When the task corresponding to second event identifier 2 is a matrix accelerator task, and the task corresponding to second event identifier 3 is an appropriate accelerator task, the processor sends second event identifier 2 to the matrix accelerator and second event identifier 3 to the appropriate accelerator.

[0175] S513, the accelerator receives multiple second event identifiers from the processor.

[0176] For example, such as Figure 12As shown, the accelerator can receive multiple second event identifiers from the processor through a first queue.

[0177] The task scheduling method provided in this application embodiment sends multiple second event identifiers to the accelerator after the task corresponding to the first event is completed. This can solve the problem of task execution order when one task has a synchronous relationship with multiple tasks.

[0178] Figure 14 A schematic diagram of a task scheduling device 1400 is shown. The task scheduling device can be the processor in the above embodiments. The task scheduling device 1400 can be used to implement the task scheduling method of any of the above embodiments.

[0179] The task scheduling device 1400 includes a processing module 1401 and a transceiver module 1402. Exemplarily, the processing module 1401 controls and manages the actions of the task scheduling device 1400, and performs the processing carried out by the task scheduling device 1400 in the above embodiments. The transceiver module 1402 supports the task scheduling device in sending event information and receiving task completion information. Optionally, if the task scheduling device 1400 includes a storage unit, the processing module 1401 can also execute programs or instructions stored in the memory, so that the task scheduling device 1400 implements the methods and functions involved in any of the above embodiments.

[0180] For example, the processing module 1401 described above can be used to perform, for example... Figure 5 Step S501 in the document, and / or other processes used in the techniques described herein. The transceiver module 1402 can be used to perform, for example... Figure 5 Step S502 in the above method embodiments, and / or other processes used in the technology described herein. All relevant content regarding each step in the above method embodiments can be referenced to the functional description of the corresponding functional module, and will not be repeated here.

[0181] For example, in hardware implementation, the functions of processing module 1401 can be executed by a processor, and the functions of transceiver module 1402 can be executed by a transceiver (transmitter / receiver) and / or communication interface. Processing module 1401 can be embedded in or independent of the processor of task scheduling device 1400 in hardware form, or it can be stored in the memory of task scheduling device 1400 in software form, so that the processor can call and execute the operations corresponding to the above functional units.

[0182] Figure 15 A schematic diagram of a task scheduling device 1500 is shown. The task scheduling device can be the accelerator in the above embodiments, and the task scheduling device 1500 can be used to implement the task scheduling method of any of the above embodiments.

[0183] The task scheduling device 1500 includes a transceiver module 1501 and a processing module 1502. For example, the transceiver module 1501 supports the task scheduling device in sending task completion information and receiving event information. The processing module 1502 controls and manages the actions of the task scheduling device 1500, and performs the processing carried out by the task scheduling device 1500 in the above embodiments. Optionally, if the task scheduling device 1500 includes a storage unit, the processing module 1502 can also execute programs or instructions stored in the memory, so that the task scheduling device 1500 implements the methods and functions involved in any of the above embodiments.

[0184] For example, the processing module 1502 described above can be used to perform, for example... Figure 7 Steps S504-S505, and / or other processes used in the technology described herein. The transceiver module 1501 can be used to perform, for example... Figure 7 Step S503 in the above method embodiments, and / or other processes used in the technology described herein. All relevant content regarding each step in the above method embodiments can be referenced to the functional description of the corresponding functional module, and will not be repeated here.

[0185] For example, in hardware implementation, the transceiver module 1501 can be implemented by a transceiver (transmitter / receiver) and / or a communication interface, and the processing module 1502 can be implemented by an accelerator. The processing module 1502 can be embedded in or independent of the processor of the task scheduling device 1500 in hardware form, or it can be stored in the memory of the task scheduling device 1500 in software form, so that the processor can call and execute the operations corresponding to the above functional units.

[0186] The steps of the methods or algorithms described in this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a core network interface device. Of course, the processor and storage medium can also exist as discrete components in the core network interface device.

[0187] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0188] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A task scheduling method, characterized in that, Applied to AI chips, the method includes: The processor looks up the synchronization information table based on the first event identifier to determine the event information. The event information includes the identifier of the event to be executed. The synchronization information table includes multiple first event identifiers corresponding to at least two accelerators, and a second event identifier corresponding to at least one of the multiple first event identifiers. There is a sequential execution order between the second event and the first event, and the second event is executed after the first event. The processor determines at least one accelerator corresponding to the event information from among the at least two accelerators based on the event information. The event information is sent to the at least one accelerator.

2. The method according to claim 1, characterized in that, The first event is a composite event, which includes N sub-events. The second event corresponding to the first event and the N sub-events have a sequential execution order, and the second event is executed after the N sub-events. N is an integer greater than or equal to 2.

3. The method according to claim 2, characterized in that, The method further includes: The processor receives first task completion information, which indicates that the task corresponding to the first sub-event has been completed; wherein, the first sub-event is any one of the N sub-events; In response to N instances of the first task completion information, the processor sends the second event identifier to the accelerator.

4. The method according to claim 3, characterized in that, The method further includes: The processor adjusts the value of the counter based on the completion information of each of the first tasks; wherein, when the tasks corresponding to the N sub-events are completed, the value of the counter is N.

5. The method according to any one of claims 1-4, characterized in that, There are multiple second event identifiers corresponding to the first event.

6. The method according to claim 5, characterized in that, The method further includes: The processor receives second task completion information, which is used to indicate that the task corresponding to the first event has been completed. In response to the second task completion information, the processor sends the plurality of second event identifiers to the accelerator.

7. The method according to any one of claims 1-6, characterized in that, The processor sends event information to the accelerator according to the synchronization information table, including: The processor sends the event information to the accelerator through a first queue according to the synchronization information table.

8. The method according to any one of claims 1-7, characterized in that, The synchronization information table also includes a task identifier corresponding to each first event identifier. The task identifier is used to indicate the task corresponding to the first event identifier, and at least one of the multiple first event identifiers includes multiple sub-task identifiers.

9. The method according to claim 8, characterized in that, The task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information also includes the first subtask identifier and the second subtask identifier.

10. A task scheduling method, characterized in that, Applied to AI chips, the method includes: The accelerator receives event information from the processor. The event information includes an identifier of an event to be executed. The identifier of the event to be executed is determined by the processor by looking up a synchronization information table based on a first event identifier. The synchronization information table includes multiple first event identifiers corresponding to at least two accelerators, and a second event identifier corresponding to at least one of the multiple first event identifiers. There is a sequential execution order between the second event and the first event, and the second event is executed after the first event. The accelerator obtains the task to be executed based on the identifier of the event to be executed; The accelerator executes the task to be performed.

11. The method according to claim 10, characterized in that, The first event is a composite event, which includes N sub-events. The second event corresponding to the first event and the N sub-events have a sequential execution order, and the second event is executed after the N sub-events. N is an integer greater than or equal to 2.

12. The method according to claim 11, characterized in that, The method further includes: Each time the accelerator completes the task corresponding to a sub-event, the accelerator sends a first task completion message to the processor. When the tasks corresponding to the N sub-events are completed, the accelerator receives the second event identifier from the processor.

13. The method according to any one of claims 10-12, characterized in that, There are multiple second event identifiers corresponding to the first event.

14. The method according to claim 13, characterized in that, The method further includes: If the accelerator completes the task corresponding to the first event, the accelerator sends a second task completion message to the processor. The accelerator receives the plurality of second event identifiers from the processor.

15. The method according to claim 14, characterized in that, The accelerator sends the second task completion information to the processor, including: The accelerator sends the second task completion information to the processor through a second queue.

16. The method according to any one of claims 10-15, characterized in that, The synchronization information table also includes a task identifier corresponding to each first event identifier. The task identifier is used to indicate the task corresponding to the first event identifier, and at least one of the multiple first event identifiers includes multiple sub-task identifiers.

17. The method according to claim 16, characterized in that, The task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information also includes the first subtask identifier and the second subtask identifier.

18. The method according to claim 17, characterized in that, The identifier of the event to be executed includes either the first subtask identifier or the second subtask identifier. The accelerator obtains the task to be executed based on the identifier of the event to be executed, including: The accelerator obtains the first subtask based on the first subtask identifier; or, the accelerator obtains the second subtask based on the second subtask identifier. The accelerator executes the task to be executed, including: the accelerator executes the first sub-task, or the accelerator executes the second sub-task.

19. A task scheduling device, characterized in that, The device, used in AI chips, includes a processing module and a transceiver module. The processing module is used to determine event information according to a synchronization information table. The event information includes an identifier of an event to be executed. The synchronization information table includes multiple first event identifiers corresponding to at least two accelerators, and a second event identifier corresponding to at least one of the multiple first event identifiers. The second event and the first event have a sequential execution order, and the second event is executed after the first event. The processing module is further configured to determine, based on the event information, at least one accelerator among the at least two accelerators that corresponds to the event information; The transceiver module is used to send the event information to the at least one accelerator.

20. The apparatus according to claim 19, characterized in that, The first event is a composite event, which includes N sub-events. The second event corresponding to the first event and the N sub-events have a sequential execution order, and the second event is executed after the N sub-events. N is an integer greater than or equal to 2.

21. The apparatus according to claim 20, characterized in that, The transceiver module is further configured to receive first task completion information, which is used to indicate that the task corresponding to the first sub-event has been completed. The processing module is further configured to send the second event identifier to the accelerator via the transceiver module in response to N first task completion messages.

22. The apparatus according to claim 21, characterized in that, The processing module is specifically used to adjust the value of the counter based on the completion information of each of the first tasks; wherein, when the tasks corresponding to the N sub-events are completed, the value of the counter is N.

23. The apparatus according to any one of claims 19-22, characterized in that, There are multiple second event identifiers corresponding to the first event.

24. The apparatus according to claim 23, characterized in that, The transceiver module is further configured to receive second task completion information, which indicates that the task corresponding to the first event has been completed. The processing module is further configured to send the plurality of second event identifiers to the accelerator through the transceiver module in response to the second task completion information.

25. The apparatus according to any one of claims 19-24, characterized in that, The processing module is specifically used to send the event information to the accelerator through the first queue according to the synchronization information table.

26. The apparatus according to any one of claims 19-25, characterized in that, The synchronization information table also includes a task identifier corresponding to each first event identifier. The task identifier is used to indicate the task corresponding to the first event identifier, and at least one of the multiple first event identifiers includes multiple sub-task identifiers.

27. The apparatus according to claim 26, characterized in that, The task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information also includes the first subtask identifier and the second subtask identifier.

28. A task scheduling device, characterized in that, The device, used in AI chips, includes a transceiver module and a processing module. The transceiver module is used to receive event information from the processor. The event information includes an identifier of an event to be executed. The identifier of the event to be executed is determined by the processor by looking up a synchronization information table based on a first event identifier. The synchronization information table includes multiple first event identifiers corresponding to at least two accelerators, and a second event identifier corresponding to at least one of the multiple first event identifiers. There is a sequential execution order between the second event and the first event, and the second event is executed after the first event. The processing module is used to obtain the task to be executed through the transceiver module according to the identifier of the event to be executed; The processing module is also used to execute the task to be executed.

29. The apparatus according to claim 28, characterized in that, The first event is a composite event, which includes N sub-events. The second event corresponding to the first event and the N sub-events have a sequential execution order, and the second event is executed after the N sub-events. N is an integer greater than or equal to 2.

30. The apparatus according to claim 29, characterized in that, The transceiver module is used to send a first task completion message to the processor when the processor completes the task corresponding to each of the sub-events. The transceiver module is further configured to receive the second event identifier from the processor when the processing module completes the task corresponding to the N sub-events.

31. The apparatus according to any one of claims 28-30, characterized in that, There are multiple second event identifiers corresponding to the first event.

32. The apparatus according to claim 31, characterized in that, The transceiver module is further configured to send a second task completion message to the processing module when the processing module completes the task corresponding to the first event. The transceiver module is also configured to receive the plurality of second event identifiers from the processor.

33. The apparatus according to claim 32, characterized in that, The transceiver module is specifically used to send the second task completion information to the processor through the second queue.

34. The apparatus according to any one of claims 28-33, characterized in that, The synchronization information table also includes a task identifier corresponding to each first event identifier. The task identifier is used to indicate the task corresponding to the first event identifier, and at least one of the multiple first event identifiers includes multiple sub-task identifiers.

35. The apparatus according to claim 34, characterized in that, The task identifier corresponding to the identifier of the event to be executed includes a first subtask identifier and a second subtask identifier, and the event information also includes the first subtask identifier and the second subtask identifier.

36. The apparatus according to claim 35, characterized in that, The identifier of the event to be executed includes either the first subtask identifier or the second subtask identifier. The processing module is specifically used to obtain the first subtask based on the first subtask identifier; or, the processing module is specifically used to obtain the second subtask based on the second subtask identifier. The processing module is specifically used to execute the first subtask, or the processing module is specifically used to execute the second subtask.

37. A task scheduling device, characterized in that, The device includes a processor and an accelerator; the processor and the accelerator are coupled; when the processor executes computer instructions stored in memory, the device performs the task scheduling method as described in any one of claims 1-9 or 10-18.

38. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a computer or processor, causes the task scheduling device to perform the task scheduling method as described in any one of claims 1-9 or 10-18.

39. A task scheduling system, characterized in that, The task scheduling system includes the task scheduling device as described in any one of claims 19-27, and the task scheduling device as described in any one of claims 28-36.

Images