Npu computing task scheduling method and device, artificial intelligence equipment and medium

Abstract

The present disclosure relates to the field of artificial intelligence, and particularly relates to a method and device for scheduling NPU computing tasks, an artificial intelligence device and a medium. The method comprises: when the task container is not full, a task management module puts the received NPU computing task into a seat of the task container; when the task container is not empty, an NPU management module takes away a corresponding number of NPU computing tasks from the task container based on the processing capacity and bandwidth of at least one NPU unit, and distributes the NPU computing tasks to the at least one NPU unit for processing; after the corresponding number of NPU computing tasks are processed, the processing result of the NPU computing task is placed in the seat bound by the NPU computing task; and the NPU management module sends a taking-away notification to the task management module, so that the task management module takes away the processing result and sends it to the task initiator. The technical solution can solve the problem of high task processing overhead, and is mainly used for task scheduling of edge devices in power Internet of Things.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular to an NPU computing task scheduling method and device, an artificial intelligence device, and a medium. BACKGROUND

[0002] In the field of power Internet of Things, a typical network topology structure includes three levels of cloud-edge-end. Although the edge device does not have powerful computing power as the cloud, it has rich computing power by extending NPU (Neural Processing Unit) units, and can run lightweight artificial intelligence models and tasks. With the development of smart power grids, edge devices implement edge-end collaboration through artificial intelligence models to solve on-site inspection, line loss analysis, and source-load collaboration of power systems, which has become a trend. Intelligent decision-making at the edge can improve response efficiency and reduce the processing pressure and bottleneck problems of servers. Various terminal devices access the edge device, and the edge device provides services to the terminal as a server.

[0003] Various terminal devices can initiate different numbers and scales of requests containing NPU computing tasks to the edge device in different scenarios. A typical processing flow is as follows: the edge device accepts the task request, initiates a request to the NPU unit for the NPU computing task part, transmits data to the NPU unit, waits for the NPU unit to complete processing and return the processing result, and the edge device outputs the processing result. For lightweight tasks, the method of initiating a response once for each task cannot effectively utilize the bandwidth of the NPU unit, and frequent interaction increases the scheduling response overhead of the edge device. SUMMARY

[0004] To solve the problems in the related art, the embodiments of the present disclosure provide an NPU computing task scheduling method and device, an artificial intelligence device, and a medium.

[0005] In a first aspect, the embodiments of the present disclosure provide an NPU computing task scheduling method, which includes:

[0006] receiving an NPU computing task initiated by a task initiator;

[0007] placing, by a task management module, the NPU computing task into a seat of a preset task container when the task container is not full, wherein the NPU computing task is bound one-to-one to the seat of the task container;

[0008] based on the processing capacity and bandwidth of at least one NPU unit, taking away, by an NPU management module, a corresponding number of NPU computing tasks from the task container when the task container is not empty, and distributing the corresponding number of NPU computing tasks to the at least one NPU unit for processing;

[0009] placing, by the NPU management module, a processing result of the NPU computing task in the seat bound by the NPU computing task after the corresponding number of NPU computing tasks are processed;

[0010] sending, by the NPU management module, a taking-away notification to the task management module, the taking-away notification being used to notify the task management module to take away the processing result in the seat bound by the processed NPU computing task from the task container;

[0011] responding, by the task management module, to receiving the taking-away notification, taking away the processing result based on the seat notified by the taking-away notification, sending the processing result to a task initiator, and releasing the seat bound by the processing result;

[0012] wherein, before operating the task container, the task management module and the NPU management module lock the task container, and after operation is completed, the task container is released.

[0013] In a possible implementation, the task capacity of the task container includes one NPU computing task or more than one NPU computing task.

[0014] In a possible implementation, when the at least one NPU unit includes two or more NPU units, the distributing the corresponding number of NPU computing tasks to the at least one NPU unit for processing includes:

[0015] determining the task proportion of each NPU unit according to the processing rate of each NPU unit at the last time;

[0016] distributing the corresponding number of NPU computing tasks to each NPU unit according to the task proportion of each NPU unit.

[0017] In a possible implementation, before the NPU computing task is placed in the seat of the task container, the method further includes:

[0018] placing, by the task holding module, the NPU computing task in the queuing container;

[0019] the placing, by the task management module, the NPU computing task in the seat of the task container when the preset task container is not full includes:

[0020] placing, by the task management module, the NPU computing task in the queuing container in the seat of the task container according to the queuing order of the NPU computing task in the queuing container when the preset task container is not full.

[0021] In a possible implementation, the method further includes:

[0022] The NPU computing tasks in the queuing container are queued according to preset priorities of various tasks by the task management module.

[0023] In a possible implementation, the method further includes:

[0024] Receiving a revocation instruction for the NPU computing task initiated by a task initiator;

[0025] Revoking the NPU computing task when the NPU computing task is in the queuing container.

[0026] In a possible implementation, the method further includes:

[0027] When the task container has no idle seat, automatically closing an entrance of the task container, and when the task container has an idle seat, keeping the entrance of the task container open.

[0028] In a second aspect, the disclosure provides an NPU computing task scheduling device, including:

[0029] A task receiving module configured to receive an NPU computing task initiated by a task initiator;

[0030] A task management module configured to put the NPU computing task into a seat of a preset task container when the task container is not full, wherein the NPU computing task is bound to the seat one-to-one.

[0031] An NPU management module configured to, when the task container is not empty, based on processing capacity and bandwidth of at least one NPU unit, take away a corresponding number of NPU computing tasks from the task container, and distribute the corresponding number of NPU computing tasks to the at least one NPU unit for processing; after the corresponding number of NPU computing tasks are processed, place a processing result of the NPU computing task in the seat bound by the NPU computing task; and send a taking-away notification to the task management module, the taking-away notification being used to notify the task management module to take away the processing result from the seat bound by the NPU computing task that has been processed;

[0032] The task management module is further configured to, in response to receiving the taking-away notification, based on the seat notified by the taking-away notification to take away the processing result, send the processing result to a task initiator and release the seat bound by the processing result;

[0033] The task management module and the NPU management module lock the task container before operating the task container, and release the task container after operation is completed.

[0034] In a possible implementation, the task capacity of the task container includes one NPU computing task or more than one NPU computing task.

[0035] In a possible implementation, when the at least one NPU unit includes two or more NPU units, the part of the NPU management module that distributes the corresponding number of NPU computing tasks to the at least one NPU unit for processing is configured to:

[0036] determine the task proportion of each NPU unit according to the processing rate of the last time of the NPU unit;

[0037] distribute the corresponding number of NPU computing tasks to each NPU unit according to the task proportion of the NPU unit.

[0038] In a possible implementation, before the task management module puts the NPU computing task into the seat of the task container, the apparatus further includes:

[0039] a task holding module configured to put the NPU computing task into a queuing container;

[0040] The part of the task management module that puts the NPU computing task into the seat of the task container when the preset task container is not full is configured to:

[0041] when the preset task container is not full, put the NPU computing task in the queuing container into the seat of the task container in the queuing order of the NPU computing task in the queuing container.

[0042] In a possible implementation, the task management module is configured to queue the NPU computing task in the queuing container according to the priority of each task.

[0043] In a possible implementation, the apparatus further includes:

[0044] a revocation receiving module configured to receive a revocation instruction initiated by a task initiator for the NPU computing task;

[0045] The task holding module is further configured to revoke the NPU computing task when the NPU computing task is in the queuing container.

[0046] In a possible implementation, the apparatus further includes:

[0047] The task container entrance switch module is configured to automatically close the entrance of the task container when there are no empty seats in the task container, and keep the entrance of the task container open when there are empty seats in the task container.

[0048] Thirdly, embodiments of this disclosure provide an artificial intelligence device, including a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method as described in any one of the first aspects.

[0049] Fourthly, this disclosure provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the method as described in any one of the first aspects.

[0050] According to the technical solution provided in this disclosure, after receiving an NPU computing task initiated by a task initiator, the task management module can place the NPU computing task into a seat in the task container when the preset task container is not full, wherein the NPU computing task is bound one-to-one with a seat in the task container; when the task container is not empty, the NPU management module, based on the processing power and bandwidth of at least one NPU unit, retrieves a corresponding number of NPU computing tasks from the task container and distributes the corresponding number of NPU computing tasks to the at least one NPU unit for processing; after the corresponding number of NPU computing tasks have been processed, the NPU management module will... The processing results of NPU computing tasks are placed in the seats bound to the NPU computing tasks. The NPU management module sends a retrieval notification to the task management module, which instructs the task management module to retrieve the processing results from the seats bound to the processed NPU computing tasks in the task container. In response to receiving the retrieval notification, the task management module retrieves the processing results based on the seats specified in the notification, sends the processing results to the task initiator, and releases the seats bound to the processing results. Before operating the task container, both the task management module and the NPU management module lock the task container, and release it after the operation is completed. Thus, by aggregating tasks into the task container, a sufficient number of NPU computing tasks can be retrieved from the task container for processing at once, reducing multiple system overheads to a single overhead. Furthermore, reduced interaction lowers scheduling response overhead, i.e., communication overhead. The NPU unit can process a sufficient number of NPU computing tasks each time, and can idle when there are no tasks, further reducing processing overhead. Therefore, system overhead, communication overhead, and processing overhead can all be reduced to varying degrees, resulting in an overall reduction in overhead.

[0051] In addition, the technical solution provided by the embodiments of the present disclosure can be flexibly self-adapted to sparse or dense task scenarios. For a sparse task scenario, the NPU unit can process all tasks when it is idle. For a dense task scenario, the NPU unit can process the requested tasks at full capacity, which can efficiently utilize the resources of the NPU unit and realize self-balancing of task processing. No matter how many tasks are in the task container, there will be no problem of NPU unit idle waiting when there are tasks, nor will there be a problem of NPU unit processing unsaturation when there are sufficient tasks. The NPU computing task quantity and the uncertainty of the occurrence time can be flexibly adapted, and efficient utilization of NPU resources can be realized.

[0052] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0053] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of the non-limiting embodiments, taken in conjunction with the accompanying drawings. The following is a description of the drawings.

[0054] Figure 1 A flowchart of a method for scheduling NPU computing tasks is shown.

[0055] Figure 2 A schematic diagram of a system for scheduling NPU computing tasks is shown.

[0056] Figure 3 A block diagram of a device for scheduling NPU computing tasks is shown.

[0057] Figure 4 A block diagram of an artificial intelligence device according to an embodiment of the present disclosure is shown.

[0058] Figure 5 A structural schematic diagram of a computer system suitable for implementing the method of the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0059] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily implement them. In addition, parts irrelevant to the description of the exemplary embodiments are omitted in the drawings for the sake of clarity.

[0060] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts or combinations thereof disclosed in the specification, and do not exclude the presence or addition of one or more other features, numbers, steps, actions, components, parts or combinations thereof.

[0061] It should be further noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other in the case of no conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0062] Figure 1 A flowchart of an NPU computing task scheduling method provided by an embodiment of the present disclosure is shown. As shown in the figure, the NPU computing task scheduling method comprises the following steps S101-S106: Figure 1

[0063] In step S101, an NPU computing task initiated by a task initiator is received;

[0064] In step S102, the NPU computing task is put into a seat of a preset task container by a task management module when the task container is not full, wherein the NPU computing task is bound to the seat of the task container one by one;

[0065] In step S103, when the task container is not empty, a corresponding number of NPU computing tasks are taken away from the task container by an NPU management module based on the processing capacity and bandwidth of at least one NPU unit, and the corresponding number of NPU computing tasks are distributed to the at least one NPU unit for processing;

[0066] In step S104, after the corresponding number of NPU computing tasks are processed, the processing results of the NPU computing tasks are placed in the seat bound by the NPU computing tasks by the NPU management module;

[0067] In step S105, the NPU management module sends a taking-away notification to the task management module, and the taking-away notification is used to notify the task management module to take away the processing results on the seat bound by the NPU computing tasks that have been processed from the task container;

[0068] In step S106, the task management module takes away the processing results based on the seat notified by the taking-away notification in response to receiving the taking-away notification, sends the processing results to the task initiator, and releases the seat bound by the processing results.

[0069] In a possible implementation, the NPU computing task scheduling method is applicable to an artificial intelligence device such as a computer, a computing device, a server, a server cluster, etc. that can perform scheduling of NPU computing tasks.

[0070] ​In one possible implementation, an NPU (Neural Processing Unit) is a computing component that can provide accelerated processing capabilities for neural network computing. The NPU unit integrates storage and computing through synaptic weights, thereby improving operating efficiency and completing artificial intelligence computing tasks in collaboration with the CPU.

[0071] In one possible implementation, the method provided in this embodiment can be applied to the power Internet of Things (IoT). The task initiator can be a terminal device in the power IoT, which is a terminal node of power equipment and can monitor parameters such as voltage, current, and power in real time. The edge device of the power IoT can be extended with one or more NPU units to execute NPU computing tasks. This edge device can be an edge gateway, edge computing node, or other edge device in the power IoT, located between the cloud and the terminal, and typically deployed in substations, distribution rooms, and other field locations. The terminal device can send real-time monitored parameters such as voltage and current to the edge device to initiate NPU computing tasks such as power anomaly identification tasks. After receiving the NPU computing task initiated by the task initiator, the edge device can implement the NPU computing task scheduling method provided in this embodiment to schedule the received NPU computing task to the corresponding NPU unit for execution.

[0072] In one possible implementation, Figure 2 This diagram illustrates a scheduling system for NPU computing tasks provided in an embodiment of the present disclosure, as shown below. Figure 2 As shown, edge devices can create task management modules and NPU management modules in the operating system to perform related operations and complete the scheduling of NPU computing tasks.

[0073] In one possible implementation, such as Figure 2 As shown, the edge device has a task container, which contains seats to hold NPU computing tasks to be executed. One seat corresponds to one NPU computing task. After the edge device receives an NPU computing task, if there are still free seats in the task container, it means the task container is not full. At this time, the task management module can place the NPU computing task into a seat in the task container, binding the NPU computing task to the seat one-to-one. This binding can be a one-to-one correspondence between the NPU computing task and the seat number. After the NPU computing task is bound to a seat, there must be tasks that are not free seats. If all seats in the task container are bound to tasks, it means the task container is full. At this time, the NPU computing task can be cached or discarded. If discarded, a notification that the task is rejected because the task container is full needs to be returned to the task initiator so that the task initiator can continue to initiate the NPU computing task.

[0074] In one possible implementation, if no seat in the task container is bound to a task, it indicates that the task container is empty. In this case, the NPU management module can set the NPU unit to an idle state. If a seat in the task container is bound to a task, it indicates that the task container is not empty. In this case, such as Figure 2 As shown, the NPU management module can retrieve NPU computing tasks from the task container. The number of NPU computing tasks retrieved can be determined by the processing power and bandwidth of at least one NPU unit in the edge device. The processing power of the NPU unit determines the scale of parallel computing, while bandwidth affects data supply efficiency. Processing power and bandwidth together determine its capacity in NPU computing tasks. Therefore, the number of NPU computing tasks that at least one NPU unit can process at one time can be determined by its processing power and bandwidth. After retrieving the corresponding number of NPU computing tasks from the task container, the NPU management module can distribute the NPU computing tasks to the at least one NPU unit for processing.

[0075] In one possible implementation, such as Figure 2 As shown, after receiving the distributed NPU computing tasks, the at least one NPU unit processes the received NPU computing tasks. After these NPU computing tasks are processed, the NPU management module can place the processing results of the NPU computing tasks into the seats bound to the NPU computing tasks. It should be noted that during the processing of these NPU computing tasks, the corresponding seats in the task container continue to be bound to the NPU computing tasks to facilitate the subsequent binding of their processing results to the seats.

[0076] In one possible implementation, such as Figure 2 As shown, after placing the processing result, the NPU management module can send a retrieval notification to the task management module. This notification instructs the task management module to retrieve the processing result from the seat associated with the processed NPU computing task within the task container. For example, the retrieval notification can include the seat number associated with the processed NPU computing task. Upon receiving the retrieval notification, the task management module can retrieve the corresponding processing result based on the seat (e.g., seat number) specified in the notification. The task management module can then send this processing result to the task initiator, thus completing the NPU computing task initiated by the task initiator. It should be noted that after retrieving the processing result, the task management module can release the seat associated with that result. Once released, the seat is idle and can be used to place subsequent new NPU computing tasks.

[0077] In a possible implementation, before operating the task container, the task management module and the NPU management module need to lock the task container, only the role that locks the task container can operate the task container, and after operation is completed, the task container is released, so that the task management module and the NPU management module can avoid taking and placing operations of the task at the same time, causing operation confusion. For example, the process of operating the task container by the task management module and the NPU management module can be as follows: When the preset task container is not full, the task management module first locks the task container, places the NPU computing task in the seat of the task container, and binds the NPU computing task and the seat of the task container one by one, and then the task container can be released. During the period in which the task management module locks the task container, the NPU management module cannot operate the task container. When the task container is not empty, the NPU management module can first lock the task container, take a corresponding number of NPU computing tasks from the task container based on the processing capacity and bandwidth of at least one NPU unit, and distribute the corresponding number of NPU computing tasks to the at least one NPU unit for processing; and then the task container can be released. After the corresponding number of NPU computing tasks are processed, the NPU management module can first lock the task container, place the processing result of the NPU computing task in the seat bound by the NPU computing task, and then release the task container. During the period in which the NPU management module locks the task container, the task management module cannot operate the task container. The NPU management module sends a taking notification to the task management module, and the taking notification is used to notify the task management module to take the processing result on the seat bound by the NPU computing task that has been processed from the task container. The task management module locks the task container in response to receiving the taking notification, takes the processing result based on the seat notified by the taking notification, and then releases the task container. The task management module sends the processing result to the task initiator and releases the seat bound by the processing result.

[0078] In a possible implementation, for a scenario in which NPU computing tasks are relatively sparse, as long as an NPU computing task is received, the NPU computing task can be placed in the task container, and the NPU unit is idle to take and process the NPU computing task, without waiting for the task container to have sufficient NPU computing tasks before taking and processing the NPU computing tasks at one time. For a scenario in which NPU computing tasks are relatively dense, sufficient NPU computing tasks can be taken from the task container each time, so that the NPU unit can process the tasks at full capacity, the resources of the NPU unit can be efficiently utilized, and self-balancing of task processing is implemented. Regardless of the number of tasks in the task container, the NPU unit does not wait when there are tasks, and the NPU unit does not have a problem of processing insufficiency when there are sufficient tasks, efficient utilization of NPU resources is implemented.

[0079] The embodiment can, after receiving an NPU computing task initiated by a task initiator, put the NPU computing task into a seat of a preset task container by a task management module when the task container is not full, wherein the NPU computing task is bound to the seat of the task container one-to-one; take away a corresponding number of NPU computing tasks from the task container based on the processing capacity and bandwidth of at least one NPU unit by an NPU management module when the task container is not empty, and distribute the corresponding number of NPU computing tasks to the at least one NPU unit for processing; place the processing result of the NPU computing task in the seat bound by the NPU computing task by the NPU management module after the corresponding number of NPU computing tasks are processed; send a taking-away notification to the task management module by the NPU management module, the taking-away notification being used to notify the task management module to take away the processing result in the seat bound by the processed NPU computing task from the task container; and send the processing result to the task initiator and release the seat bound by the processing result by the task management module in response to receiving the taking-away notification based on the seat notified by the taking-away notification. Before operating the task container, the task management module and the NPU management module lock the task container, and release the task container after the operation is completed. In this way, by converging tasks into a task container, a sufficient number of NPU computing tasks can be taken away from the task container at a time for processing, reducing the system overhead to once, reducing the interaction, and reducing the scheduling response overhead, i.e., the communication overhead to a certain extent. The NPU unit can process a sufficient number of NPU computing tasks at a time, and can be idle when there is no task, so that the processing overhead can also be reduced to a certain extent. Thus, the system overhead, the communication overhead, and the processing overhead can be reduced to different extents, and the overall overhead is reduced.

[0080] In a possible implementation, the task capacity of the task container includes one NPU computing task or more than one NPU computing task.

[0081] In this embodiment, in some scenarios, there is only one NPU unit in the edge device, and each NPU computing task initiated by a terminal device must be processed by occupying one NPU unit, so that the task capacity of the task container includes one NPU computing task. Of course, in other scenarios, one or more NPU units in the edge device can process two or more NPU computing tasks, so that the task capacity of the task container can be more than one NPU computing task.

[0082] In a possible implementation, when the at least one NPU unit includes two or more NPU units, the distributing the corresponding number of NPU computing tasks to the at least one NPU unit for processing includes:

[0083] determining a task proportion of each NPU unit according to a processing bandwidth and a last processing rate of the NPU unit;

[0084] distributing the corresponding number of NPU computing tasks to each NPU unit according to the task proportion of the NPU unit.

[0085] In this implementation, when there is only one NPU unit in the edge device, the NPU management module can directly take out, from the task container, a number of NPU computing tasks that the NPU unit can process, and then directly send the corresponding number of NPU computing tasks to the NPU unit.

[0086] In this implementation, when there are two or more NPU units in the edge device, a corresponding number of NPU computing tasks can be taken out from the task container based on the processing capability and bandwidth of the two or more NPU units, and then a task proportion of each NPU unit is determined according to a last processing rate of the NPU unit; more NPU computing tasks can be distributed to an NPU unit with a faster processing rate. It should be noted that the distribution proportion is dynamically adjusted according to the last processing rate of the NPU unit each time NPU computing tasks are distributed, to achieve efficient optimization.

[0087] In a possible implementation, before the NPU computing task is placed in the seat of the task container, the method further includes:

[0088] placing, by a task holding module, the NPU computing task in a queuing container;

[0089] the placing, by the task management module, the NPU computing task in the seat of the task container when the preset task container is not full includes:

[0090] the placing, by the task management module, the NPU computing task in the seat of the task container when the preset task container is not full includes:

[0091] In this implementation, as Figure 2As shown, the edge device can also build a task holding module and a queuing container. When the edge device receives an NPU computing task, the NPU computing task can be put into the queuing container by the task holding module. The NPU computing tasks can be automatically queued according to the time of entering the queuing container. When the preset task container is not full, the task management module can put the NPU computing tasks in the queuing container into the seats of the task container in order according to the queuing order in the queuing container.

[0092] In a possible implementation, the method further includes:

[0093] The tasks in the queuing container are queued according to the preset priority of various tasks by the task management module.

[0094] In this implementation, the corresponding priority can be set for different NPU computing tasks in advance. For example, the NPU computing task that needs to be processed urgently can be set to have a higher priority. The NPU computing tasks in the queuing container can be queued according to the queuing rule that the higher the priority is, the earlier the position is arranged. The NPU computing task arranged in the earlier position can be taken away earlier, so that the NPU computing task that needs to be processed urgently can be taken away and processed as soon as possible.

[0095] This implementation can adjust the arrangement order of the NPU computing tasks in the queuing container according to the priority of the tasks, which can effectively improve the flexibility and fairness of task response.

[0096] In a possible implementation, the method further includes:

[0097] The revocation instruction for the NPU computing task initiated by the task initiator is received.

[0098] When the NPU computing task is in the queuing container, the NPU computing task is revoked.

[0099] In this implementation, the task initiator may need to revoke the NPU computing task due to accidental touch or incorrect initiation of the task, etc. when initiating the NPU computing task. At this time, the revocation instruction for the NPU computing task initiated by the task initiator can revoke the NPU computing task when the revocation instruction is received. If the NPU computing task to be revoked is in the queuing container, the NPU computing task can be taken out of the queuing container and destroyed by the task management module.

[0100] In a possible implementation, the method further includes:

[0101] When the task container has no idle seat, the entrance of the task container is closed, and when the task container has an idle seat, the entrance of the task container is kept open.

[0102] In this embodiment, the edge device can also set a task container entry switch for the task container, which can be closed by the task management module when there is no idle seat in the task container, so that the entry of the task container is closed and the NPU computing task cannot enter the task container; when there is an idle seat in the task container, the entry of the task container is kept open to ensure that the newly received NPU computing task can enter the task container from the entry.

[0103] Examples can illustrate the execution process of the NPU computing task scheduling method. It is assumed that an NPU unit is set in the edge device, which can process 30 units of tasks at a time, and four NPU computing tasks are denoted as A, B, C and D, each task being 2 units. At this time, the task container and the queuing container are empty, and the NPU unit is idling. The task execution process can include the following steps:

[0104] a. The A task is first initiated, and the task holding module 1 puts the A task into the queuing container, and the task management module takes the A task from the queuing container and puts it into the task container, and binds seat 1 for it;

[0105] b. The NPU management module finds that the task container is not empty, stops the NPU unit idling, and submits the A task to the NPU unit for processing;

[0106] c. After the NPU unit finishes processing, the NPU management module puts the processing result of the A task into seat 1 of the task container and notifies the task management module to take the task result from seat 1;

[0107] d. The task management module receives the notification and takes the processing result from seat 1 of the task container and delivers it to the task holding module 1, which sends it to the task initiator of the task A;

[0108] e. During the processing of the A task, the B, C and D tasks are also initiated and put into the queuing container by the corresponding task holding modules, and the processing is completed through steps b, c and d.

[0109] The present disclosure also provides an NPU computing task scheduling device, Figure 3 The structure block diagram of the NPU computing task scheduling device according to the embodiment of the present disclosure is shown, which can be realized as part or all of an artificial intelligence device through software, hardware or a combination of the two. As shown in the figure, Figure 3 The NPU computing task scheduling device includes:

[0110] a task receiving module 301 configured to receive an NPU computing task initiated by a task initiator;

[0111] ​​The task management module 302 is configured to place the NPU computing task into the seat of the preset task container when the task container is not full, wherein the NPU computing task is bound to the seat of the task container in one-to-one manner.

[0112] The NPU management module 303 is configured to, when the task container is not empty, take away a corresponding number of NPU computing tasks from the task container based on the processing capacity and bandwidth of at least one NPU unit, and distribute the corresponding number of NPU computing tasks to the at least one NPU unit for processing; after the corresponding number of NPU computing tasks are processed, place the processing result of the NPU computing task in the seat bound to the NPU computing task; and send a taking-away notification to the task management module, wherein the taking-away notification is used to notify the task management module to take away the processing result in the seat bound to the NPU computing task that has been processed from the task container.

[0113] The task management module 302 is further configured to, in response to receiving the taking-away notification, take away the processing result based on the seat notified by the taking-away notification, send the processing result to the task initiator, and release the seat bound to the processing result.

[0114] The task management module 302 and the NPU management module 303 lock the task container before operating the task container, and release the task container after the operation is completed.

[0115] In a possible implementation, the task capacity of the task container includes one NPU computing task or more than one NPU computing task.

[0116] In a possible implementation, when the at least one NPU unit includes two or more NPU units, the part of the NPU management module that distributes the corresponding number of NPU computing tasks to the at least one NPU unit for processing is configured to:

[0117] determine the task proportion of each NPU unit according to the processing rate of the last time of each NPU unit;

[0118] distribute the corresponding number of NPU computing tasks to each NPU unit according to the task proportion of each NPU unit.

[0119] In a possible implementation, before the task management module places the NPU computing task into the seat of the task container, the device further includes:

[0120] The task holding module is configured to place the NPU computing task into the queuing container.

[0121] The part of the task management module that puts the NPU computing task into the seat of the task container when the preset task container is not full is configured to:

[0122] When the preset task container is not full, the NPU computing tasks in the queuing container are sequentially put into the seats of the task container according to the queuing order of the NPU computing tasks in the queuing container.

[0123] In a possible implementation, the task management module is configured to queue the NPU computing tasks in the queuing container according to preset priorities of various tasks.

[0124] In a possible implementation, the device further includes:

[0125] The revocation receiving module is configured to receive a revocation instruction initiated by a task initiator for the NPU computing task.

[0126] The task holding module is further configured to revoke the NPU computing task when the NPU computing task is in the queuing container.

[0127] In a possible implementation, the device further includes:

[0128] The task container entrance switch module is configured to automatically close the entrance of the task container when the task container has no idle seat, and keep the entrance of the task container open when the task container has an idle seat.

[0129] The technical terms and technical features mentioned in the device embodiments are the same as or similar to those mentioned in the above method embodiments, and the explanation and description of the technical terms and technical features involved in the device can refer to the explanation and description of the above method embodiments, which will not be repeated here.

[0130] The present disclosure also discloses an artificial intelligence device, Figure 4 A structural block diagram of an artificial intelligence device according to an embodiment of the present disclosure is shown.

[0131] As Figure 4 shown, the artificial intelligence device 400 includes a memory 401 and a processor 402, wherein the memory 401 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 402 to implement the method according to the embodiment of the present disclosure.

[0132] Figure 5 A structural schematic diagram of a computer system suitable for implementing the method of the embodiment of the present disclosure is shown.

[0133] As Figure 5As shown, the computer system 500 includes a processing unit 501 which can perform various processing in the above-described embodiments according to programs stored in a read only memory (ROM) 502 or loaded into a random access memory (RAM) 503 from a storage section 508. In the RAM 503, various programs and data required for the operation of the computer system 500 are also stored. The processing unit 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0134] Connected to the I / O interface 505 are an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a display device such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as necessary. A removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 510 as necessary, so that a computer program read therefrom is installed into the storage section 508 as necessary. The processing unit 501 can be implemented as a CPU, a GPU, a TPU, a FPGA, a NPU, etc.

[0135] In particular, according to embodiments of the present disclosure, the above-described methods can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising computer instructions which, when executed by a processor, implement the steps of the above-described methods. In such embodiments, the computer program product can be downloaded and installed from a network via the communication section 509, and / or installed from the removable recording medium 511.

[0136] The flow and block diagrams in the drawings represent possible architectural, functional, and operational architectures of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0137] The units or modules described in the embodiments of the present disclosure can be implemented by software, or by programmable hardware. The described units or modules can also be arranged in a processor, and the names of the units or modules do not constitute a limitation on the units or modules themselves in some cases.

[0138] As another aspect, the present disclosure also provides a computer readable storage medium, which can be the computer readable storage medium included in the artificial intelligence device or the computer system in the above embodiments, or can exist separately and not be assembled into the device. The computer readable storage medium stores one or more programs, which are used by one or more processors to execute the method described in the present disclosure.

[0139] The above description is merely preferred embodiments of the present disclosure and a description of principles of applied technologies. It should be understood by those skilled in the art that the scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also includes other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the inventive concept. For example, the above technical features are replaced with the technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

Claims

1. A scheduling method for NPU computing tasks, characterized in that, include: Receive neural processor (NPU) computing tasks initiated by the task initiator; When the preset task container is not full, the task management module places the NPU computing task into a seat in the task container. The NPU computing task is bound one-to-one with a seat in the task container, and one seat is used to place one NPU computing task. When the task container is not empty, the NPU management module, based on the processing power and bandwidth of at least one NPU unit, retrieves a corresponding number of NPU computing tasks from the task container and distributes these tasks to the at least one NPU unit for processing. Where the at least one NPU unit consists of two or more NPU units, distributing the corresponding number of NPU computing tasks to these units includes: determining the task ratio of each NPU unit based on its most recent processing rate; and distributing the corresponding number of NPU computing tasks to each NPU unit according to its task ratio. After the corresponding number of NPU computing tasks have been processed, the NPU management module places the processing results of the NPU computing tasks in the slots bound to the NPU computing tasks. The NPU management module sends a retrieval notification to the task management module, which is used to notify the task management module to retrieve the processing results from the seat bound to the processed NPU computing task in the task container; In response to receiving the take-away notification, the task management module takes away the processing result based on the seat notified by the take-away notification, sends the processing result to the task initiator, and releases the seat bound to the processing result. Specifically, the task management module and the NPU management module lock the task container before operating it, and release the task container after the operation is completed.

2. The method according to claim 1, characterized in that, The task capacity of the task container includes one NPU computing task or more than one NPU computing task.

3. The method according to claim 1, characterized in that, Before placing the NPU computing task into the seat of the task container, the method further includes: The task holding module places the NPU computing task into the queuing container; The step of the task management module placing the NPU computing task into a slot in the pre-defined task container when the container is not full includes: When the preset task container is not full, the task management module places the NPU computing tasks in the queue container into the seats of the task container in sequence according to the queuing order of the NPU computing tasks in the queue container.

4. The method according to claim 3, characterized in that, The method further includes: The task management module queues the NPU computing tasks in the queuing container according to the preset priorities of various tasks.

5. The method according to claim 3, characterized in that, The method further includes: Receive a cancellation command for the NPU computing task initiated by the task initiator; The NPU computing task is cancelled when it is located in the queuing container.

6. The method according to claim 1, characterized in that, The method further includes: When there are no available seats in the task container, close the entrance of the task container; when there are available seats in the task container, keep the entrance of the task container open.

7. A scheduling device for NPU computing tasks, characterized in that, include: The task receiving module is configured to receive neural processor (NPU) computing tasks initiated by the task initiator. The task management module is configured to place the NPU computing task into a seat in the task container when the preset task container is not full, wherein the NPU computing task is bound one-to-one with a seat in the task container, and one seat is used to place one NPU computing task. The NPU management module is configured to, when the task container is not empty, retrieve a corresponding number of NPU computing tasks from the task container based on the processing capacity and bandwidth of at least one NPU unit, and distribute the corresponding number of NPU computing tasks to the at least one NPU unit for processing; after the corresponding number of NPU computing tasks are processed, place the processing results of the NPU computing tasks in the seat bound to the NPU computing task; and send a retrieval notification to the task management module, the retrieval notification being used to notify the task management module to retrieve the processing results from the seat bound to the processed NPU computing tasks in the task container; when the at least one NPU unit includes two or more NPU units, the part of the NPU management module that distributes the corresponding number of NPU computing tasks to the at least one NPU unit for processing is configured to: determine the task ratio of each NPU unit based on the most recent processing rate of each NPU unit; and distribute the corresponding number of NPU computing tasks to each NPU unit according to the task ratio of each NPU unit. The task management module is also configured to, in response to receiving the take-away notification, take the processing result based on the seat notified by the take-away notification, send the processing result to the task initiator, and release the seat bound to the processing result; Specifically, the task management module and the NPU management module lock the task container before operating it, and release the task container after the operation is completed.

8. The apparatus according to claim 7, characterized in that, The task capacity of the task container includes one NPU computing task or more than one NPU computing task.

9. The apparatus according to claim 7, characterized in that, Before the task management module places the NPU computing task into the seat of the task container, the device further includes: The task holding module is configured to place the NPU computing task into a queuing container; In the task management module, the part where the NPU computing task is placed into a seat in the pre-set task container when the container is not full is configured as follows: When the preset task container is not full, the NPU computing tasks in the queuing container are placed into the seats of the task container in sequence according to the queuing order of the NPU computing tasks in the queuing container.

10. The apparatus according to claim 9, characterized in that, The task management module is configured to queue the NPU computing tasks in the queuing container according to the preset priorities of various tasks.

11. The apparatus according to claim 9, characterized in that, The device further includes: The cancellation receiving module is configured to receive cancellation instructions for the NPU computing task initiated by the task initiator; The task holding module is also configured to cancel the NPU computing task when the NPU computing task is located in the queuing container.

12. The apparatus according to claim 7, characterized in that, The device further includes: The task container entrance switch module is configured to automatically close the entrance of the task container when there are no empty seats in the task container, and keep the entrance of the task container open when there are empty seats in the task container.

13. An artificial intelligence device, characterized in that, The method includes a memory and a processor, the memory being used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any one of claims 1 to 6.

14. A readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the method described in any one of claims 1 to 6.

Images