Task scheduling method and related apparatus

Abstract

Disclosed in the present disclosure are a task scheduling method and a related apparatus. According to the technical solution disclosed in the present disclosure, a task and a scheduling policy are generated on a host side; the host side can directly issue the task and the task scheduling policy therefor to a device side; and the device side controls, on the basis of the scheduling policy, the task to be issued to a processor core, such that the processor core executes the task. In this way, a task scheduling process is completed with low delay overheads by means of collaboration between the host side and the device side. Fig. 1

Description

Task scheduling methods and related devices Cross-references to related applications

[0001] This application claims priority to Chinese patent application filed on December 10, 2024, with application number 202411823098.0 and entitled “Task Scheduling Method and Related Apparatus”. Technical Field

[0002] This disclosure generally relates to the field of artificial intelligence technology. More specifically, this disclosure relates to a task scheduling method and related apparatus. Background Technology

[0003] Heterogeneous computing systems integrate different types of processors into a single computing system, allowing each processor to leverage its unique computing strengths and achieve higher computing performance. In heterogeneous computing systems, scheduling strategies directly impact the utilization of hardware resources on the device side and task waiting times, thus affecting program execution speed.

[0004] In existing technologies, generating a better scheduling strategy often requires more time overhead. How to speed up the task scheduling process so as to make use of a better scheduling algorithm has become a technical problem. In view of this, there is an urgent need to provide a task scheduling scheme so as to achieve task scheduling with lower latency overhead. Summary of the Invention

[0005] In order to at least solve one or more technical problems described in the background section above, this disclosure proposes the following technical solutions and several embodiments thereof.

[0006] In a first aspect, this disclosure discloses a task scheduling method applied to the device side of a heterogeneous computing device, comprising: receiving a task and a scheduling policy associated with the task from a host side, wherein the scheduling policy is generated on the host side; and distributing the task to the processor core on the device side according to the scheduling policy for task execution.

[0007] In a second aspect, this disclosure also discloses a task scheduling method executed by a heterogeneous computing device, the heterogeneous computing device including a host side and a device side; the method includes: on the host side: generating and sending a task and a scheduling policy associated with the task; on the device side: receiving the task and the scheduling policy; and distributing the task to the processor core on the device side according to the scheduling policy to facilitate task execution.

[0008] In a third aspect, this disclosure further discloses a heterogeneous computing device, including a device side and a host side; wherein the host side is configured to: generate and send tasks and scheduling policies associated with the tasks; the device side is configured to: receive tasks and scheduling policies; and distribute tasks to the processor cores of the device side according to the scheduling policies for task execution.

[0009] The task scheduling method and heterogeneous computing device disclosed herein allow the host side to directly send tasks to the device side after task generation. The host side then generates and sends out a scheduling policy. The device side sends tasks to the processor cores according to the scheduling policy so that the processor cores can execute the tasks. In this way, the scheduling policy is generated on the host side, while task interception is executed on the device side. Through the cooperation between the host side and the device side, the task scheduling process is completed with low latency overhead. Attached Figure Description

[0010] The above and other objects, features, and advantages of exemplary embodiments of this disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0011] Figure 1 shows an exemplary structural diagram of a board in some embodiments of this disclosure.

[0012] Figure 2 shows an exemplary structural schematic diagram of the combined processing apparatus in some embodiments of this disclosure.

[0013] Figure 3 shows an exemplary structural schematic diagram of a computing device in some embodiments of this disclosure.

[0014] Figure 4 shows an exemplary structural diagram of a processor core in some embodiments of this disclosure.

[0015] Figure 5 illustrates an exemplary schematic diagram of a processor core in some embodiments of this disclosure intending to write data to a processor core in another cluster.

[0016] Figure 6 shows an exemplary structural diagram of a heterogeneous computing system in some embodiments of this disclosure.

[0017] Figure 7 shows an exemplary schematic diagram of the software stack in some embodiments of this disclosure.

[0018] Figure 8 shows an exemplary schematic diagram of a reinforcement learning scheduling algorithm in some embodiments of this disclosure.

[0019] Figure 9 shows an exemplary flowchart of a task scheduling method in some embodiments of this disclosure.

[0020] Figure 10 shows an exemplary schematic diagram of a task scheduling method in some embodiments of this disclosure.

[0021] Figure 11 shows an exemplary schematic diagram of a device-side queue in some embodiments of this disclosure.

[0022] Figure 12 shows an exemplary schematic diagram of task scheduling using the PG algorithm in some embodiments of this disclosure.

[0023] Figure 13 shows an exemplary schematic diagram of the Event mechanism in some embodiments of this disclosure.

[0024] Figure 14 shows an exemplary schematic diagram of host-side queues and device-side queues in some embodiments of this disclosure. Detailed Implementation

[0025] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0026] It should be understood that the terms “comprising” and “including” used in this disclosure and claims indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0027] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0028] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0029] The specific embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. To clearly explain the technical solutions disclosed herein, the relevant hardware implementations and related technical terms of the embodiments of this disclosure will be introduced first.

[0030] Figure 1 illustrates an exemplary structural diagram of board 10 in some embodiments of this disclosure. As shown in Figure 1, board 10 includes chip 101, which is a system-on-a-chip (SoC) integrating one or more combined processing devices. The combined processing device is an artificial intelligence computing unit used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in cloud intelligence, and a significant characteristic of cloud intelligence applications is the large amount of input data, placing high demands on the platform's storage and computing capabilities. Board 10 in this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and powerful computing capabilities.

[0031] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. The calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface forms, such as a PCIe interface.

[0032] The board 10 also includes a storage device 104 for storing data, which includes one or more memory cells 105. The storage device 104 is connected to and transmits data with the controller 106 and the chip 101 via a bus. The controller 106 in the board 10 is configured to regulate the state of the chip 101. Therefore, in one application scenario, the controller 106 may include a microcontroller (MCU).

[0033] Figure 2 shows an exemplary structural schematic diagram of a combined processing apparatus in some embodiments of this disclosure. As shown in Figure 2, the combined processing apparatus 20 includes a computing device 201, an interface device 202, a processing device 203, and a storage device 204.

[0034] The computing device 201 is configured to perform user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0035] Interface device 202 is used to transmit data and control commands between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to on-chip storage device of computing device 201. Further, computing device 201 can obtain control commands from processing device 203 via interface device 202 and write them to on-chip control cache of computing device 201. Alternatively or optionally, interface device 202 can also read data from storage device of computing device 201 and transmit it to processing device 203.

[0036] Processing device 203, as a general-purpose processing device, performs basic control including but not limited to data transfer, and starting and / or stopping computing device 201. Depending on the implementation, processing device 203 may be one or more types of processors, including but not limited to digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, computing device 201 disclosed herein can be considered as having a single-core structure or a homogeneous multi-core structure. However, when computing device 201 and processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0037] The memory 204 is used to store data to be processed. It may be DRAM or DDR memory, typically 16GB or larger in size, and is used to store data of the computing device 201 and / or the processing device 203.

[0038] Figure 3 illustrates an exemplary structural diagram of a computing device 201 in some embodiments of this disclosure, in which the computing device 201 is implemented as a multi-core intelligent processor. The computing device 201 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 201 in Figure 3 adopts a multi-core hierarchical structure design. As a system-on-a-chip, the computing device 201 includes multiple clusters, and each cluster includes multiple processor cores. In other words, the computing device 201 is constructed in a hierarchical structure of system-on-a-chip, cluster, and processor core.

[0039] From the perspective of the system-on-a-chip hierarchy, as shown in Figure 3, the computing device 201 includes an external storage controller 301, a peripheral communication module 302, an on-chip interconnect module 303, a synchronization module 304, and multiple clusters 305.

[0040] There can be multiple external storage controllers 301; two are exemplarily shown in Figure 3. These controllers respond to access requests from the processor core to access external storage devices, such as DRAM 204 in Figure 2, thereby reading data from or writing data to external storage. The peripheral communication module 302 receives control signals from the processing device 203 via the interface device 202, initiating the computing device 201 to execute tasks. The on-chip interconnect module 303 connects the external storage controllers 301, the peripheral communication module 302, and multiple clusters 305 to transmit data and control signals between the modules. The synchronization module 304 is a global barrier controller (GBC) used to coordinate the operating speed of each cluster and ensure information synchronization. The multiple clusters 305 are the computing core of the computing device 201; four are exemplarily shown in Figure 3. With hardware development, the computing device 201 disclosed herein may also include eight, sixteen, sixty-four, or even more clusters 305. The clusters 305 are used to efficiently execute deep learning algorithms.

[0041] In terms of cluster hierarchy, as shown in Figure 3, each cluster 305 includes multiple processor cores (IPU cores) 306 and one memory core (MEM core) 307.

[0042] Four processor cores 306 are shown in the figure as an example, and this disclosure does not limit the number of processor cores 306.

[0043] Figure 4 shows an exemplary structural diagram of a processor core in some embodiments of this disclosure. Each processor core 306 includes three main modules: a control module 41, a processing module 42, and a storage module 43.

[0044] The control module 41 coordinates and controls the operation of the computation module 42 and the storage module 43 to complete the deep learning task. It includes an instruction fetch unit (IFU) 411 and an instruction decode unit (IDU) 412. The instruction fetch unit 411 fetches instructions from the processing device 203, and the instruction decode unit 412 decodes the fetched instructions and sends the decoding result as control information to the computation module 42 and the storage module 43.

[0045] The computation module 42 includes a vector operation unit 421 and a matrix operation unit 422. The vector operation unit 421 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit 422 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution.

[0046] Storage module 43 is used to store or move related data, including neuron RAM (NRAM) 431, weight RAM (WRAM) 432, input / output direct memory access (IODMA) 433, and move direct memory access (MVDMA) 434. NRAM 431 is used to store feature maps for processor core 306 to calculate and intermediate results after calculation; WRAM 432 is used to store the weights of the deep learning network; IODMA 433 controls the memory access of NRAM 431 / WRAM 432 and DRAM 204 through broadcast bus 309; MVDMA 434 controls the memory access of NRAM 431 / WRAM 432 and SRAM 308.

[0047] Returning to Figure 3, storage core 307 is primarily used for storage and communication, namely storing shared data or intermediate results among processor cores 306, and performing communication between cluster 305 and DRAM 204, communication between clusters 305, and communication between processor cores 306. In other embodiments, storage core 307 has scalar operation capabilities for performing scalar operations.

[0048] Storage core 307 includes a shared memory unit (SRAM) 308, a broadcast bus 309, a cluster direct memory access (CDMA) module 310, and a global direct memory access (GDMA) module 311. SRAM 308 acts as a high-performance data relay station. Data multiplexed between different processor cores 306 within the same cluster 305 does not need to be obtained from DRAM 204 by each processor core 306 individually. Instead, it is relayed between processor cores 306 via SRAM 308. Storage core 307 only needs to quickly distribute the multiplexed data from SRAM 308 to multiple processor cores 306, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip I / O access.

[0049] Broadcast bus 309, CDMA 310, and GDMA 311 are used to perform communication between processors 306, communication between clusters 305, and data transfer between cluster 305 and DRAM 204, respectively. These will be explained below.

[0050] The broadcast bus 309 is used to complete high-speed communication between the processor cores 306 within the cluster 305. In this embodiment, the broadcast bus 309 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point (i.e., single processor core to single processor core) data transmission. Multicast is a communication method that transmits a piece of data from SRAM 308 to several specific processor cores 306. Broadcast is a communication method that transmits a piece of data from SRAM 308 to all processor cores 306, and is a special case of multicast.

[0051] CDMA 310 is used to control SRAM 308 accesses between different clusters 305 within the same computing device 201. Figure 5 shows an exemplary schematic diagram of a processor core intending to write data to a processor core of another cluster in some embodiments of this disclosure, to illustrate the working principle of CDMA 310. In this application scenario, the same computing device includes multiple clusters. For ease of explanation, only cluster 0 and cluster 1 are shown in the figure. Cluster 0 and cluster 1 each include multiple processor cores. Similarly, for ease of explanation, only processor core 0 is shown in cluster 0, and only processor core 1 is shown in cluster 1. Processor core 0 intends to write data to processor core 1.

[0052] First, processor core 0 sends a unicast write request to write data into its local SRAM 0. CDMA 0 acts as the master and CDMA 1 acts as the slave. The master pushes the write request to the slave, that is, the master sends the write address AW and the write data W to transmit the data to SRAM 1 of cluster 1. Then, the slave sends a write response B as a response. Finally, processor core 1 of cluster 1 sends a unicast read request to read the data from SRAM 1.

[0053] Returning to Figure 3, GDMA 311 works in conjunction with external memory controller 301 to control memory access from SRAM 308 to DRAM 204 in cluster 305, or to read data from DRAM 204 into SRAM 308. As previously mentioned, communication between DRAM 204 and NRAM 431 or WRAM 432 can be achieved through two channels. The first channel is direct communication between DRAM 204 and NRAM 431 or WRAM 432 via IODMA 433; the second channel involves first transferring data between DRAM 204 and SRAM 308 via GDMA 311, and then transferring data between SRAM 308 and NRAM 431 or WRAM 432 via MVDMA 434. Although the second channel appears to require more components and has a longer data flow, in some embodiments, the bandwidth of the second channel is actually much greater than that of the first channel. Therefore, communication between DRAM 204 and NRAM 431 or WRAM 432 may be more efficient via the second channel. The embodiments disclosed herein can select the data transmission channel based on their hardware capabilities.

[0054] In other embodiments, the functions of GDMA 311 and IODMA 433 can be integrated into the same component. For ease of description, this disclosure treats GDMA 311 and IODMA 433 as different components. For those skilled in the art, any component whose implemented functions and achieved technical effects are similar to those disclosed herein falls within the scope of protection of this disclosure. Furthermore, the functions of GDMA 311, IODMA 433, CDMA 310, and MVDMA 434 can also be implemented by the same component. Similarly, any component whose implemented functions and achieved technical effects are similar to those disclosed herein falls within the scope of protection of this disclosure.

[0055] Heterogeneous computing systems

[0056] Heterogeneous computing systems integrate different types of processors into a single computing system, allowing each processor to leverage its unique computational strengths and achieve higher computing performance. Generally, a heterogeneous computing system includes a host side and a device side, with processors on both sides having different instruction set architectures. The device side features multiple processor cores, providing powerful parallel computing capabilities. The host side is primarily responsible for running the operating system and various applications, and coordinating the workflow and synchronization between the device-side processor cores. The host side can utilize a Central Processing Unit (CPU), a Field-Programmable Gate Array (FPGA), etc., while the device side can employ Application-Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Neural-Network Processing Units (NPUs), Machine Learning Units (MLUs), Tensor Processing Units (TPUs), and Graphics Processing Units (GPUs). In some embodiments, a heterogeneous computing system can be obtained by combining a CPU and a board 10. In these embodiments, the CPU is connected to the board 10 as an external device 103 through an external interface device 102.

[0057] Figure 6 illustrates an exemplary structural diagram of a heterogeneous computing system in some embodiments of this disclosure. As shown in Figure 6, the heterogeneous computing system includes a host side and a device side, which establish a communication connection through a PCIe interface. This allows the host side to send tasks to be executed and data to be processed to the device side, and the device side to return processing results to the host side. Further observation of the internal structure of the device side reveals that it includes multiple processor cores with computing capabilities, which can process computing tasks in parallel. For example, taking board 10 as an example, the multiple processor cores 306 in board 10 can process computing tasks in parallel. In the heterogeneous computing system shown in Figure 6, only the components relevant to this embodiment are shown. It will be apparent to those skilled in the art that the heterogeneous computing system may also include common components different from those shown in the figure, including but not limited to the host-side controller and the Arithmetic Logic Unit (ALU).

[0058] Software stack

[0059] The numerous processor cores on the device side provide powerful parallel computing capabilities. To effectively utilize these capabilities, a multi-layered software stack is formed to support application development. Figure 7 illustrates an exemplary schematic diagram of the software stack in some embodiments of this disclosure. As shown in Figure 7, the software stack may include a framework layer, an acceleration library layer, a runtime layer, a driver layer, and an Application Programming Interface (API) layer provided by the framework layer, acceleration library layer, runtime layer, and driver layer. It is understood that the framework layer, acceleration library layer, runtime layer, and driver layer provide concrete implementations for their respective API layers, which in turn provide calling interfaces to higher layers.

[0060] In the software stack shown in Figure 7, the Driver layer is responsible for low-level interactions between the host and device sides (including establishing hardware connections between the host and device sides, initializing device-side hardware, and directly accessing device-side hardware). Acting as a bridge between the host and device sides, the Driver layer consists of a host-side Driver layer and a device-side Driver layer. Above the Driver layer is the Driver API layer. The runtime layer, built on top of the Driver API layer, shields the complex details of the underlying device hardware and implements methods for device management, memory handling, and thread management. Above the runtime layer is the Runtime API layer. The acceleration library layer, built on top of the Runtime API layer, further shields the details of the underlying device hardware and provides implementations for specific algorithms (such as Fast Fourier Transform). Above the acceleration library layer is the Acceleration Library API layer. The framework layer shields the underlying technical details such as gradient calculation to help developers design more complex applications. Above the framework layer is the Framework API layer; common deep learning frameworks include TensorFlow, PyTorch, Keras, Caffe, MXNet, and Theano.

[0061] Scheduling strategies and scheduling algorithms

[0062] In a heterogeneous computing system consisting of a host side and a device side, the host side excels at executing control-intensive programs such as searching, parsing, and sorting, while the device side is better suited for data-intensive programs in scenarios involving inference / training of artificial intelligence (AI) models or other scenarios with a large number of matrix operations. When running a program on the device side of a heterogeneous computing system, the host side first needs to break the program down into multiple tasks and then send the tasks to the device side. The device side can then distribute the tasks received from the host side to its processor cores for processing. Specifically, the device side can be equipped with a task scheduler, which can distribute the tasks sent by the host side to the processor cores on the device side according to a scheduling policy. For example, taking board 10 as an example, the host side transmits the tasks to be executed and the scheduling policy to board 10 through the external interface device 102. The controller 106 of board 10 is equipped with a task scheduler, which controls the processor core 306 in chip 101 to execute the tasks according to the scheduling policy.

[0063] Generally, scheduling a large number of tasks is an iterative process, completing the scheduling of all tasks through multiple scheduling rounds. In some embodiments, in each scheduling round, the scheduling policy is used to determine the number of tasks to be dispatched in the current scheduling round and which processor cores to dispatch these tasks for execution, in order to reduce task waiting time and efficiently utilize the parallel computing capabilities of the device. In other embodiments, multiple task queues are used to store tasks to be executed. In these embodiments, the scheduling policy is used to determine which task queue to dispatch tasks in the current scheduling round, and how many tasks to dispatch to the selected task queue. Furthermore, the scheduling policy can be generated by the host side or the device side based on the deployed scheduling algorithm. It is understood that the scheduling algorithm directly affects the waiting time and execution time of tasks, thereby affecting the execution speed of the program.

[0064] Currently, common scheduling algorithms are divided into two types: heuristic scheduling algorithms and reinforcement learning scheduling algorithms. Heuristic scheduling algorithms typically utilize experience and heuristic rules to find local optima that satisfy specific objectives, and are suitable for workloads with uniformly distributed task durations and a single workload.

[0065] Figure 8 illustrates exemplary schematic diagrams of reinforcement learning scheduling algorithms in some embodiments of this disclosure. It should be noted that the agent shown in Figure 8 can be a parameterized policy network, which can be a neural network model containing an input layer, an output layer, and two hidden layers. Those skilled in the art can design policy networks with different structures according to actual needs, and this disclosure does not impose any limitations on this. As shown in Figure 8, the reinforcement learning scheduling algorithm generates scheduling policies through continuous interaction between the agent and the environment. The agent takes the observed environmental state as input, calculates the action probability distribution in the action space, and then selects an action according to the probability to obtain the output. In the task scheduling scenario: the environment can be regarded as the execution process of the task on the device side, and the environmental state can include the execution state of the task and the usage state of the device; actions correspond to scheduling policies; the action space corresponds to the scheduling policy space, which is a set of scheduling policies.

[0066] At a time step in the execution of the reinforcement learning scheduling algorithm, the agent first takes the environment state as input to select the scheduling policy for the current time step, and then inputs the scheduling policy into the environment. The device can schedule tasks according to the output scheduling policy. After the tasks are sent to the processor core according to the scheduling policy, the agent can observe the environment state and the reward for the scheduling policy at the current time step. The state and reward are fed back to the agent to adjust the method for generating subsequent scheduling policies.

[0067] It's understandable that reinforcement learning scheduling algorithms can adjust scheduling strategies promptly based on workload changes. When facing dynamic and diverse workloads, they can learn the optimal scheduling strategy online, maximizing resource utilization on the device side. Generally, the quality of scheduling strategies generated by reinforcement learning algorithms is superior to that of heuristic scheduling algorithms. However, reinforcement learning scheduling algorithms, especially deep reinforcement learning algorithms, require extensive training of the policy network, resulting in slow convergence. Furthermore, the inference speed of reinforcement learning scheduling algorithms is slower than the policy generation speed of heuristic scheduling algorithms, leading to increased overall latency.

[0068] Therefore, how to accelerate the task scheduling process to utilize higher-performance scheduling algorithms has become a technical problem. In view of this, this disclosure proposes a task scheduling method to achieve task scheduling with lower latency overhead through collaborative cooperation between the host side and the device side. Figure 9 shows an exemplary flowchart of the task scheduling method in some embodiments of this disclosure. This method is applied to the device side of a heterogeneous computing device. As shown in Figure 9, the method includes: step 901, receiving a task and a scheduling policy associated with the task from the host side, wherein the scheduling policy is generated on the host side; step 902, distributing the task to the processor core on the device side according to the scheduling policy for task execution.

[0069] Figure 10 illustrates an exemplary schematic diagram of a task scheduling method in some embodiments of this disclosure. As shown in Figure 10, the host side can split the code program to be executed (shown as Procedure in the figure) into multiple tasks, and generate a scheduling policy for the tasks according to a scheduling algorithm. In some embodiments, the code program refers to a program for training a deep learning model; in other embodiments, the code program refers to a program for inference using a deep learning model. Through the communication connection between the host side and the device side, the host side first sends the tasks to the device side, and then sends the scheduling policy to the device side. The device side receives the tasks and the scheduling policy, and then distributes the tasks to the processor cores for processing according to the scheduling policy.

[0070] It is understood that in the task scheduling method disclosed herein, tasks and scheduling policies are generated on the host side; the host side can directly send tasks to the device side without intercepting or processing them; after receiving the scheduling policy, the device side then controls the task to be sent to the processor core for processing according to the scheduling policy. In this way, through the coordinated cooperation between the host side and the device side, the task scheduling process is completed with low latency overhead.

[0071] Optionally, in this embodiment, the host side can directly send tasks to the device side without intercepting the tasks. In these embodiments, the host side and the device side see the same tasks almost simultaneously. The difference is that the device side intercepts the tasks and schedules them according to the scheduling policy. It should be noted that for learning-based scheduling algorithms such as reinforcement learning scheduling algorithms, because the inference of the model has a large time overhead, intercepting tasks on the host side, generating scheduling policies based on the scheduling algorithm, and then sending tasks will inevitably increase the latency of task sending. However, according to the task scheduling method disclosed in this disclosure, complex scheduling algorithms can be deployed on the host side of a heterogeneous computing system. The host side can quickly send tasks to the device side and generate scheduling policies based on the scheduling algorithm. After receiving the tasks and scheduling policies, the device side sends the tasks according to the scheduling policies. In this way, by adjusting the task scheduling process, a pipeline parallel execution is formed between "sending tasks from the host side to the device side" and "generating scheduling policies on the host side" in the whole process, thereby accelerating the task scheduling process and adapting to the introduction of complex reinforcement learning scheduling algorithms into task scheduling scenarios.

[0072] Furthermore, in cross-stream scenarios, the Event mechanism in the device-side Driver layer achieves task ordering through placeEvents and waitEvents. If task distribution is controlled on the host side according to a scheduling policy, firstly, the logic code implementing the Event needs to be moved from the device-side Driver layer to the host side, which undoubtedly involves a significant amount of engineering work; secondly, the host side needs to distribute the placeEvent task first, and then the waitEvent task, leading to an increase in program execution time. However, according to the method disclosed in this paper, the host side can directly send the task to the device side after generating it, and task ordering can be achieved based on the Event mechanism in the device-side Driver layer, thus avoiding the problems mentioned above.

[0073] Furthermore, in some embodiments, the host side splits the code program to be executed into multiple tasks. After recording the basic information of the tasks, the tasks are sent to the device side. After receiving the tasks and scheduling policy, the device side sends the tasks to the processor cores for processing according to the scheduling policy. In these embodiments, the host side can generate a scheduling policy based on the recorded basic task information. The basic task information includes, but is not limited to: the execution duration of the task, the number of processor cores required to execute the task, the index of the task, and other attributes that can identify the task.

[0074] As shown in Figure 7, when developing applications, programmers can write applications based on the framework layer API, or on the acceleration layer API, runtime API, or even the driver API. For tasks issued by applications written in different ways to be scheduled by the scheduling algorithm, the scheduling algorithm must be deployed on a software path traversed by all four methods, such as the host-side driver API layer, the host-side driver layer, and the device-side driver layer. Since the driver layer runs in kernel mode and is highly sensitive to software overhead, and implementing a complete set of library functions for the scheduling algorithm in kernel mode is a time-consuming process, in some embodiments, the scheduling strategy is generated at the host-side driver application programming interface layer.

[0075] It is understandable that the host-side driver application programming interface layer is the Driver API layer mentioned earlier. Regardless of whether the program is written based on the framework layer API, acceleration layer API, runtime API, or the Driver API, the Driver API layer is an essential software path. Therefore, after deploying the scheduling algorithm at the Driver API layer, regardless of the approach used, after the program is broken down into tasks, the tasks can be distributed to the device side for execution according to the deployed scheduling algorithm. In other words, the scheduling algorithm deployed at the Driver API layer can uniformly support programs written using all approaches, achieving optimized integration of scheduling for programs written using all approaches, thereby introducing the scheduling algorithm into heterogeneous computing systems with high development efficiency.

[0076] In some embodiments, the device intercepts the task before it is sent to the processor core on the device side.

[0077] Understandably, when there are a large number of tasks, directly assigning all tasks to the processor core would cause multiple tasks to attempt to occupy the same core, resulting in insufficient computing resources for each task to run efficiently, significantly slowing down task execution. Therefore, in such cases, the device needs to intercept tasks. Interception means not assigning tasks further, but waiting for other tasks to finish before assigning the intercepted task. In some embodiments, the device pauses the assignment of intercepted tasks to the processor core; in other embodiments, task interception is achieved by placing the tasks in a waiting queue, where tasks in the queue wait for other tasks to finish before being assigned.

[0078] In some embodiments, intercepting tasks on the device side includes: establishing at least one device-side queue for recording tasks; storing tasks received from the host side in at least one device-side queue to facilitate the distribution of tasks from the device side to the processor core on the device side.

[0079] Figure 11 illustrates an exemplary schematic diagram of device-side queues in some embodiments of this disclosure. As shown in Figure 11, in some embodiments, multiple device-side queues for recording tasks are maintained on the device side, and intercepted tasks can be stored in these queues. Tasks in each device-side queue can employ a First-In-First-Out (FIFO) strategy, ensuring that the first task to enter the queue is placed at the front of the queue, with subsequent tasks arranged sequentially at the back. When a task in the device-side queue needs to be sent to the processor core, tasks are selected sequentially from the front of the queue for sending.

[0080] In some embodiments, the device determines whether to intercept the task based on a scheduling policy before sending the task to the processor core on the device side.

[0081] Understandably, the host side can use a scheduling algorithm to generate a scheduling policy, thereby pre-distributing tasks to the device side, and then distributing the scheduling policy to the device side. In this case, the scheduling policy can be used to determine which tasks can be distributed to the processor cores on the device side for execution and which tasks need to be intercepted.

[0082] In some embodiments, determining whether to intercept a task based on a scheduling policy includes: intercepting the task when the resource utilization rate of the device side is higher than a first predetermined threshold; and sending the task to the processor core of the device side when the resource utilization rate of the device side is not higher than the first predetermined threshold.

[0083] It is understood that the device side is configured with hardware resources including computing devices and storage devices, and resource utilization refers to the degree to which these hardware devices are effectively utilized. Resource utilization can be calculated as the ratio of the amount of hardware resources actually used to the total amount of hardware resources owned by the device. In some embodiments, device-side resource utilization refers to the proportion of processor cores on the device side that are being used to execute tasks out of all processor cores. In some embodiments, a first predetermined threshold is set for device-side resource utilization. When the device-side resource utilization is not higher than the first predetermined threshold, it means that there are currently many idle processor cores on the device side, and the computing resources are not being used efficiently. Therefore, tasks can be directly sent to the processor cores on the device side without interception. When the device-side resource utilization is higher than the first predetermined threshold, it means that there is currently a high operating load on the device side, and tasks can be intercepted.

[0084] In some embodiments, when the resource utilization rate on the device side is higher than a second predetermined threshold, the scheduling strategy is generated by the host side based on a reinforcement learning scheduling algorithm; when the resource utilization rate on the device side is not higher than the second predetermined threshold, the scheduling strategy is generated by the host side based on a heuristic scheduling algorithm.

[0085] It is understandable that reinforcement learning scheduling algorithms and heuristic scheduling algorithms have their own advantages and disadvantages. Heuristic scheduling algorithms generate scheduling strategies based on simple rules and can quickly obtain near-optimal solutions with limited computing resources. However, when faced with multiple mixed tasks, heuristic scheduling algorithms cannot adjust scheduling strategies or parameters in a timely manner according to changes in workload, resulting in workload latency fluctuations and decreased device resource utilization. Reinforcement learning scheduling algorithms can handle more complex workloads, but their inference speed is relatively slow, leading to increased task delivery latency. Therefore, this disclosure proposes a scheduling method that combines reinforcement learning scheduling algorithms and heuristic scheduling algorithms, using different scheduling algorithms for different workloads. When there are many idle processor cores on the device side (i.e., low device resource utilization), a heuristic scheduling algorithm is used to generate a scheduling strategy to deliver tasks as quickly as possible. When there are few idle processor cores on the device side (i.e., high device resource utilization), the device side is not sensitive to the speed of task delivery. In this case, the host side can use a reinforcement learning scheduling algorithm to generate a scheduling strategy. In this scenario, although the inference speed of the reinforcement learning scheduling algorithm is relatively slow, resulting in a delay in the scheduling policy, this delay is acceptable for the device side. Furthermore, the scheduling policy generated based on the reinforcement learning scheduling algorithm is superior to that of the heuristic scheduling algorithm and is more suitable for the current scenario. In some embodiments, after multiple rounds of scheduling using the reinforcement learning scheduling algorithm, if the resource utilization rate on the device side does not exceed a second predetermined threshold, the host side can switch the method for generating the scheduling policy from the reinforcement learning scheduling algorithm to a heuristic scheduling algorithm.

[0086] Specifically, when the resource utilization rate on the device side is not higher than a second predetermined threshold, the reinforcement learning scheduling algorithm can obtain basic information about the tasks in the host-side queue. This basic information includes, but is not limited to, the task's execution duration, the number of processor cores required to execute the task, the task's index, and other attributes that can identify the task. Subsequently, the reinforcement learning scheduling algorithm can perform inference calculations based on this basic task information to obtain a scheduling strategy. This reinforcement learning scheduling algorithm can be obtained through training.

[0087] Figure 12 illustrates an exemplary schematic diagram of task scheduling using the Policy Gradient (PG) algorithm in some embodiments of this disclosure. These embodiments can train reinforcement learning scheduling algorithms based on the PG algorithm. As shown in Figure 12, the device side uses n+1 device-side queues to record tasks. During the entire task scheduling process, all tasks are scheduled through m+1 loops. Each loop iterates through all queues and schedules the tasks in each queue. It is understood that as tasks are issued, the number of tasks in each queue gradually decreases; therefore, each loop only schedules tasks in queues containing remaining tasks. Figure 12 uses obj... i,j This represents the process of scheduling the (j+1)th queue in the (i+1)th cycle, which is one scheduling round; further, the scheduling unit (Obj Cell) in the right half of Figure 12 shows obj i,j The specific process.

[0088] It is understandable that the Obj Cell in Figure 12 corresponds to a time step in the PG algorithm, and a trajectory in the PG algorithm can include multiple Obj Cells. As shown in the figure, at a time step, the PG algorithm: first, observes the task state (State, S); using S as input, it uses the policy network to calculate the scheduling policy probability distribution (p) in the scheduling policy space and selects a scheduling policy (a) according to the probability; it executes the scheduling policy, and after distributing the task to the processor core according to the scheduling policy, it obtains the reward (r) for the scheduling policy (a) at the current time step; the t in the dashed box (meaning it may exist) represents the trajectory termination identifier. If the trajectory termination identifier is observed at the current time step, the policy network can be updated based on the reward recorded in the current trajectory and the scheduling policy probability distribution, and the updated policy network is used for task scheduling of the next trajectory. This process is repeated until the policy network converges, obtaining a trained policy network. When the resource utilization on the device side is higher than a second predetermined threshold, the trained policy network can infer the scheduling policy.

[0089] In some embodiments, generating a scheduling policy using a heuristic scheduling algorithm includes: first, determining a feasible scheduling policy space, i.e., the set of all feasible scheduling policies; and then randomly selecting a scheduling policy from the scheduling policy space. In some embodiments, generating a scheduling policy using a heuristic scheduling algorithm includes: setting the number of tasks to be dispatched based on the number of idle processor cores and the size of the tasks; and dispatching tasks to idle processor cores based on the number of tasks to be dispatched. In these embodiments, the size of the task can be measured based on the number of processor cores required to execute the task. After determining the number of processor cores (a) required to execute each task, the number of idle processor cores (b) is divided by that number of processor cores and rounded down, i.e., by... ( The number of tasks to be issued is calculated by rounding down to the nearest integer. Tasks are randomly assigned to idle processor cores. Alternatively, a heuristic scheduling algorithm might set the number of tasks to be assigned based on empirical values, and then assign tasks from a device-side queue to processor cores according to that number.

[0090] In some embodiments, the second predetermined threshold is set empirically, for example, the second predetermined threshold is set to a value between 60% and 70%. In other embodiments, a set of test cases is prepared in advance, and the test cases are run under both heuristic scheduling algorithms and reinforcement learning scheduling algorithms. The performance index data under the two scheduling algorithms is recorded, and then a suitable second predetermined threshold is determined based on the performance index data. The performance index data includes, but is not limited to: device-side resource utilization, service quality indicators reflecting fairness, and program execution time.

[0091] In some embodiments, when the resource utilization rate on the device side is higher than a second predetermined threshold, the method for scheduling tasks further includes: the device side feeding back resource contention information to the host side so that the host side can generate a scheduling policy based on a reinforcement learning scheduling algorithm; further, after the host side generates a scheduling policy based on the reinforcement learning scheduling algorithm, the device side can receive the scheduling policy so that the task can be sent to the processor core on the device side according to the scheduling policy.

[0092] If the resource utilization rate on the device side exceeds the second predetermined threshold, it indicates that the device side is experiencing resource shortages and resource contention. In this case, the device side can send a resource contention signal to the host side to provide feedback on the resource contention information. The host side can then use this resource contention information to initiate a reinforcement learning scheduling algorithm to generate a scheduling strategy.

[0093] In some embodiments, the process of the device side sending tasks to the processor core on the device side according to a scheduling policy includes: identifying the dependencies between tasks; and sending the tasks to the processor core on the device side according to the scheduling policy and the dependencies.

[0094] It is understandable that the dependencies between tasks include data dependencies, meaning that one task can only begin execution after another task has completed. The `placeEvent` and `waitEvent` timeouts in the Event mechanism of the device-side Driver layer can effectively manage these dependencies. Figure 13 shows an exemplary schematic diagram of the Event mechanism in some embodiments of this disclosure. As shown in Figure 13, there are three tasks issued by the host side in device-side queue i, ordered i1, i2, and i3 in execution order. There are also three other tasks issued by the host side in device-side queue j, ordered j1, j2, and j3 in execution order. Among these six tasks, there is a dependency between i1 and j1; that is, j1 needs to be executed after i1 has finished. According to the Event mechanism, a placeEvent task is added between i1 and i2 to indicate whether i1 has finished executing; a waitEvent task is added before j1 to indicate whether j1 can be sent; furthermore, j1 cannot be sent while the waitEvent task is still in the device-side queue, but needs to wait for the task i1 and the placeEvent task in queue i to be sent before the waitEvent task can be sent, and only after that can j1 be sent.

[0095] Therefore, in some embodiments, the device can identify the dependencies between tasks, and then add placeEvent tasks and waitEvent tasks to the task queue according to the dependencies between tasks. When a task is sent to a queue according to the scheduling policy, if there is a placeEvent task among the tasks to be sent according to the scheduling policy, and the waitEvent task associated with the placeEvent task has not yet been sent, the sending of tasks in the queue needs to be paused, and the sending of tasks in the queue needs to be restarted after the waitEvent task associated with the placeEvent task has been sent.

[0096] In some embodiments, after the device sends the task to the processor core on the device side, it can also generate a synchronization update signal based on the sent task. The synchronization update signal is used to synchronously update the device-side queue and the host-side queue set on the host side.

[0097] Figure 14 illustrates exemplary schematic diagrams of host-side queues and device-side queues in some embodiments of this disclosure. As shown in Figure 14, after the host side splits the program into multiple tasks, it generates multiple host-side queues to record the tasks. The host side generates scheduling policies for the tasks in the host-side queues, so that the host-side queues and scheduling policies can be sent to the device side together; or, the host side can send the host-side queues to the device side first, and then send the scheduling policies to the device side after the scheduling policies are generated.

[0098] As mentioned earlier, the task scheduling process requires multiple scheduling rounds. After the device sends out tasks according to the current scheduling policy, the host generates the next round's scheduling policy. Therefore, during task execution, it is necessary to maintain consistency between the device-side queue and the host-side queue. Thus, it is necessary to synchronize and update the device-side queue and the host-side queue after the device sends out tasks. In some embodiments, after the device sends a portion of tasks to the processor core for processing, it sends a synchronization update signal to the host. The host deletes the corresponding tasks from its queue based on the synchronization update signal, and then generates a new scheduling policy based on the updated host-side queue, thereby ensuring that the task execution process proceeds in an orderly manner. In some embodiments, the synchronization update signal includes information about the tasks sent, such as the task index or other attributes that can uniquely identify the task.

[0099] This disclosure also discloses a task scheduling method executed by a heterogeneous computing device, which includes a host side and a device side; the method includes: on the host side: generating and sending a task and a scheduling policy associated with the task; on the device side: receiving the task and the scheduling policy; and distributing the task to the processor core on the device side according to the scheduling policy to facilitate task execution.

[0100] Corresponding to the task scheduling method disclosed in this disclosure as described above, this disclosure provides the following embodiments of a task scheduling method executed by a heterogeneous computing device:

[0101] In some embodiments, the scheduling policy is generated at the driver application programming interface layer on the host side.

[0102] In some embodiments, before the task is sent to the processor core on the device side, the task is intercepted on the device side.

[0103] In some embodiments, before the task is sent to the processor core on the device side, the following steps are included: on the device side: determining whether to intercept the task based on a scheduling policy.

[0104] In some embodiments, determining whether to intercept a task according to a scheduling policy includes, on the device side: intercepting the task when the device-side resource utilization is higher than a first predetermined threshold; and sending the task to the processor core on the device side when the device-side resource utilization is not higher than the first predetermined threshold.

[0105] In some embodiments, intercepting a task includes, on the device side: establishing a device-side queue for recording tasks; storing the received tasks in the device-side queue so as to distribute the tasks on the device side to the processor core on the device side.

[0106] In some embodiments, the method further includes: when the resource utilization rate on the device side is higher than a second predetermined threshold, generating a scheduling policy on the host side based on a reinforcement learning scheduling algorithm; and when the resource utilization rate on the device side is not higher than the second predetermined threshold, generating a scheduling policy on the host side based on a heuristic scheduling algorithm.

[0107] In some embodiments, when the resource utilization rate on the device side is higher than a second predetermined threshold, the method further includes: the device side sending resource contention information; and the host side receiving the resource contention information to facilitate the generation of a scheduling policy based on a reinforcement learning scheduling algorithm.

[0108] In some embodiments, the process of distributing tasks to the processor core on the device side according to a scheduling policy includes, on the device side: identifying dependencies between tasks; and distributing tasks to the processor core on the device side according to the scheduling policy and the dependencies.

[0109] In some embodiments, the method further includes: establishing a host-side queue for recording tasks on the host side; and after the tasks in the device side are sent to the processor core on the device side, the method includes: generating a synchronization update signal on the device side according to the sent tasks, for synchronously updating the device-side queue and the host-side queue.

[0110] In some embodiments, generating and sending tasks on the host side includes: recording task information using a host-side queue; and sending the task to the device side. It is understood that the host side only records basic task information and does not intercept the task; that is, the task is quickly sent to the device side according to the normal process while being recorded on the host side. It can be considered that the host side and the device side see the same task passing by almost simultaneously and record it sequentially. The difference is that the device side intercepts the task and schedules it according to a scheduling policy. Thus, through the cooperation of the host side and the device side, the task scheduling process is accelerated. Furthermore, this disclosure discloses a heterogeneous computing device, including a device side and a host side; wherein the host side is configured to: generate and send tasks and task-related scheduling policies; the device side is configured to: receive tasks and scheduling policies; and send tasks to the processor cores of the device side according to the scheduling policy for task execution.

[0111] In summary, the specific functions implemented by the heterogeneous computing device and the task scheduling method executed by the heterogeneous computing device provided in the embodiments of this specification can be explained in comparison with the foregoing embodiments in this specification, and can achieve the technical effects of the foregoing embodiments. Therefore, they will not be repeated here.

[0112] It should be noted that the task scheduling method proposed in this disclosure deploys a reinforcement learning model at the Driver API layer on the host side. It only records, but does not intercept, all passing tasks, while interception only occurs on the device side. The host side outputs scheduling decisions based on the reinforcement learning model and sends the decision information to the device side, which then distributes the tasks according to the scheduling decisions. In this way, through collaboration between the user-space-based host side and the kernel-space-based device side, a reinforcement learning scheduling algorithm can be introduced with only minor modifications to existing implementations, while keeping latency overhead within a small range.

[0113] It is important to note that in this disclosed embodiment, the scheduled task can be a computational task derived from decomposing applications across various domains, and the data involved can be of various types. For example, relevant domains can include machine translation, sentiment analysis, text summarization, text classification, and named entity recognition in Natural Language Processing (NLP), and the corresponding data types can include text data, word embeddings, sentence structures, etc.; relevant domains can include facial recognition, image classification, object detection, and image segmentation in Computer Vision (CV), and the corresponding data types can include pixel data and image features; relevant domains can include intelligent assistants, automatic caption generation, and speech-to-text conversion in Automatic Speech Recognition (ASR), and the corresponding data types can include audio signals and spectrograms; relevant domains can also include generative AI applications. Generative AI refers to artificial intelligence technologies that use complex algorithms, models, and rules to learn from large-scale datasets to create new original content, such as, but not limited to, content of various types such as text, images, sound, video, and code; the corresponding data types can include image data, audio data, video data, speech data, text data, document data, etc.

[0114] It should be noted that, for the sake of brevity, this disclosure describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solutions disclosed herein are not limited by the order of the described actions. Therefore, based on the disclosure or teachings of this document, those skilled in the art will understand that some steps can be performed in a different order or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this disclosure can be considered optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions disclosed herein. In addition, depending on the solution, the description of some embodiments in this disclosure may have different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this disclosure can also be referred to the relevant descriptions of other embodiments.

Claims

1. A task scheduling method applied to the device side of a heterogeneous computing device, comprising: Receive tasks and scheduling policies associated with the tasks from the host side, wherein the scheduling policies are generated on the host side; and The task is dispatched to the processor core on the device side according to the scheduling policy so that the task can be executed.

2. The method according to claim 1, wherein, The scheduling strategy is generated at the driver application programming interface layer on the host side.

3. The method according to claim 1, further comprising: The task is intercepted before it is sent to the processor core on the device side.

4. The method according to claim 1, further comprising: Before the task is sent to the processor core on the device side, the scheduling policy determines whether the task should be intercepted.

5. The method according to claim 4, wherein, Determining whether to intercept the task based on the scheduling strategy includes: When the device-side resource utilization rate exceeds a first predetermined threshold, the task is intercepted; and When the resource utilization rate on the device side is not higher than a first predetermined threshold, the task is sent to the processor core on the device side.

6. The method according to any one of claims 3-5, wherein, Intercepting the task includes: Establish a device-side queue for recording the tasks described; The received tasks are stored in the device-side queue so that the tasks in the device side can be sent to the processor core of the device side.

7. The method according to any one of claims 1-6, further comprising: When the resource utilization rate on the device side is higher than the second predetermined threshold, the scheduling strategy is generated by the host side based on the reinforcement learning scheduling algorithm. as well as When the resource utilization rate on the device side is not higher than the second predetermined threshold, the scheduling strategy is generated by the host side based on a heuristic scheduling algorithm.

8. The method according to claim 7, wherein, When the resource utilization rate on the device side exceeds a second predetermined threshold, the method further includes: Resource contention information is fed back to the host side so that the host side can generate the scheduling strategy based on the reinforcement learning scheduling algorithm.

9. The method according to claim 1, wherein, The processor cores that distribute the task to the device side according to the scheduling policy include: Identify the dependencies between the tasks; Based on the scheduling strategy and the dependencies, the task is distributed to the processor core on the device side.

10. The method according to claim 6, wherein, After the task in the device side is distributed to the processor core in the device side, the following steps are included: Based on the issued task, a synchronization update signal is generated to synchronously update the device-side queue and the host-side queue set on the host side.

11. A task scheduling method executed by a heterogeneous computing device, the heterogeneous computing device comprising a host side and a device side; the method comprising: On the host side: generate and send tasks and scheduling policies associated with the tasks; On the equipment side: Receive the task and the scheduling policy; The task is dispatched to the processor core on the device side according to the scheduling policy so that the task can be executed.

12. The method according to claim 11, wherein, The scheduling strategy is generated at the driver application programming interface layer on the host side.

13. The method according to claim 11, wherein, Before the task is sent to the processor core on the device side, the following is included on the device side: The task is intercepted.

14. The method according to claim 11, wherein, Before the task is sent to the processor core on the device side, the following is included on the device side: The scheduling strategy determines whether the task should be intercepted.

15. The method according to claim 14, wherein, Determining whether to intercept the task based on the scheduling policy includes, on the device side: When the resource utilization rate on the device side exceeds a first predetermined threshold, the task is intercepted; as well as When the resource utilization rate on the device side is not higher than a first predetermined threshold, the task is sent to the processor core on the device side.

16. The method according to any one of claims 13-15, wherein, Intercepting the task includes, on the device side: Establish a device-side queue for recording the tasks described; The received tasks are stored in the device-side queue so that the tasks in the device side can be sent to the processor core of the device side.

17. The method according to any one of claims 11-16, further comprising: When the resource utilization rate on the device side is higher than the second predetermined threshold, the host side generates the scheduling strategy based on the reinforcement learning scheduling algorithm. as well as When the resource utilization rate on the device side is not higher than the second predetermined threshold, the host side generates the scheduling strategy based on the heuristic scheduling algorithm.

18. The method according to claim 17, wherein, When the resource utilization rate on the device side exceeds a second predetermined threshold, the method further includes: The device sends resource contention information; The host receives the resource contention information to generate the scheduling strategy based on the reinforcement learning scheduling algorithm.

19. The method according to claim 11, wherein, The task is distributed to the processor core on the device side according to the scheduling policy, including, on the device side: Identify the dependencies between the tasks; Based on the scheduling strategy and the dependencies, the task is distributed to the processor core on the device side.

20. The method of claim 16, further comprising: A host-side queue is established to record the tasks. After the task on the device side is sent to the processor core on the device side, the process includes: on the device side, generating a synchronization update signal based on the sent task, for synchronously updating the device-side queue and the host-side queue.

21. The method according to claim 20, wherein, Generating and sending tasks on the host side includes, on the host side: The host-side queue is used to record the information of the task; and The task is sent to the device side.

22. A heterogeneous computing device, comprising a device side and a host side; wherein, The host side is configured to generate and send tasks and scheduling policies associated with the tasks; The device side is configured as follows: Receive the task and the scheduling policy; and The task is dispatched to the processor core on the device side according to the scheduling policy so that the task can be executed.

Images