A task scheduling method, device and related equipment

Abstract

The present application provides a task scheduling method, which is applied to a heterogeneous system including multiple nodes. The tasks to be executed include at least one heterogeneous API call. The method includes: determining scheduling reference information of the heterogeneous API call according to the parameter scale value of the heterogeneous API call. The scheduling reference information is used to describe the execution efficiency of multiple processing engines in the heterogeneous system. Then, according to the scheduling reference information, a target processing engine whose execution efficiency meets a preset condition is determined from the available processing engines in the heterogeneous system, and the heterogeneous API call is scheduled to the corresponding target processing engine. This method supports selecting a suitable processing engine according to the parameter scale value of the heterogeneous API call, scheduling the heterogeneous API call to the above-mentioned suitable processing engine for execution, improving the execution efficiency of the heterogeneous API call, and further improving the execution efficiency of the task.

Description

Technical Field

[0001] This application relates to the field of computing technologies, and in particular, to a task scheduling method, apparatus, device, and computer-readable storage medium. Background Art

[0002] With the in-depth popularization of intelligent devices such as computers in different application fields, more and more domain specific architectures (DSAs) are emerging. Based on these domain specific architectures, new processing engines have emerged. Among them, a processing engine refers to a processor for processing data.

[0003] For example, in the field of image processing, processing engines based on domain specific architectures include graphical processing units (GPUs), image processors (IPs), etc. In the field of digital signal processing, processing engines based on domain specific architectures include digital signal processors (DSPs). In the field of artificial intelligence, processing engines based on domain specific architectures include neural-network processing units (NPUs), etc.

[0004] Among them, a data processing system including two or more processing engines is called a heterogeneous system. An application developed for this heterogeneous system is a heterogeneous application. A heterogeneous application usually includes at least one heterogeneous application programming interface (API) call. Among them, a heterogeneous API is an API with a unified interface but different optimized implementations on different processing engines.

[0005] Currently, the efficiency of a heterogeneous system in executing a task including a heterogeneous API call is low. The industry urgently needs to provide an efficient task scheduling method. Summary of the Invention

[0006] This application provides a task scheduling method, which supports selecting a suitable processing engine according to the parameter scale value of a heterogeneous API call in a task, and scheduling the heterogeneous API call to the suitable processing engine, thereby improving the execution efficiency of the heterogeneous API call, and further improving the execution efficiency of the heterogeneous application. This application also provides a corresponding apparatus, device, computer-readable storage medium, and computer program product for the above method.

[0007] In a first aspect, the present application provides a task scheduling method. This method is applied to a heterogeneous system including multiple nodes. Among them, a heterogeneous system refers to a data processing system including multiple processing engines. A processing engine is a unit capable of performing data processing operations. The processing engine includes a central processing unit (CPU) and a processing unit based on a domain specific architecture (DSA). Among them, the processing unit based on the DSA architecture includes a graphical processing unit (GPU), an image processor (IP), a digital signal processor (DSP), a neural-network processing unit (NPU), a field programmable gate array (FPGA), and so on.

[0008] An application developed for a heterogeneous system is a heterogeneous application. A heterogeneous application usually includes at least one heterogeneous application programming interface (API) call. Among them, a heterogeneous API is specifically an API with a unified interface but different optimized implementations on different processing engines. A task submitted to a heterogeneous system includes at least one heterogeneous API call.

[0009] When a heterogeneous system executes a heterogeneous API call, it determines scheduling reference information for the heterogeneous API call according to the parameter scale value of the heterogeneous API call. Among them, the scheduling reference information is used to describe the execution efficiency of multiple processing engines in the heterogeneous system. The heterogeneous system determines a target processing engine whose execution efficiency meets a preset condition from the available processing engines in the heterogeneous system, and schedules the heterogeneous API call to the corresponding target processing engine.

[0010] This method supports selecting a suitable processing engine according to the parameter scale value of a heterogeneous API call, and scheduling the heterogeneous API call to the above-mentioned suitable processing engine for execution, improving the execution efficiency of the heterogeneous API call, and thus improving the execution efficiency of the task.

[0011] In some possible implementation manners, the target processing engine that meets the preset condition includes the processing engine with the highest execution efficiency, so that the execution efficiency of heterogeneous API calls can be maximized. In some embodiments, the target processing engine that meets the preset condition includes the processing engine with an execution efficiency higher than the preset efficiency. Specifically, when the processing engine with the highest execution efficiency corresponding to multiple heterogeneous API calls is the same processing engine, in order to avoid some heterogeneous API calls from being in a waiting state, one of the multiple heterogeneous API calls can be scheduled to the processing engine with the highest execution efficiency, and the other heterogeneous API calls can be scheduled to other processing engines with an execution efficiency higher than the preset efficiency, such as the processing engine with the second highest execution efficiency. In this way, the overall execution efficiency of the task can be improved.

[0012] In some possible implementation manners, the heterogeneous system includes a heterogeneous cluster, and the heterogeneous cluster includes a master node and multiple slave nodes. The task is scheduled by the master node to a first slave node among the multiple slave nodes. The heterogeneous system (specifically, the first slave node in the heterogeneous system) can determine a target processing engine whose execution efficiency meets the preset condition from the processing engines available to the first slave node according to the scheduling reference information. In this way, the scheduling of heterogeneous API calls within the node is realized, the execution efficiency of heterogeneous API calls is improved, and further the execution efficiency of the task is improved.

[0013] In some possible implementation manners, the heterogeneous system includes a heterogeneous cluster. The heterogeneous cluster includes a master node and multiple slave nodes. The task is scheduled by the master node to a first slave node among the multiple slave nodes. When the processing engine whose execution efficiency meets the preset condition in the first slave node is unavailable, the heterogeneous system (specifically, the first slave node in the heterogeneous system) can determine a second slave node from the heterogeneous cluster according to the scheduling reference information, and then determine a target processing engine whose execution efficiency meets the preset condition from the processing engines available to the second slave node. In this way, the cross-node scheduling of heterogeneous API calls is realized, the execution efficiency of heterogeneous API calls is improved, and the overall execution efficiency of the task is improved.

[0014] In some possible implementation manners, considering that when performing cross-node scheduling, actual parameters need to be transmitted and results need to be written back, which will generate additional scheduling overhead. The heterogeneous system (specifically, the first slave node of the first system) can also consider the scheduling overhead when determining the second slave node. The heterogeneous system can determine, according to the scheduling reference information, that the node whose available processing engines meet the preset condition and whose scheduling overhead is less than the preset overhead in the heterogeneous cluster is the second slave node.

[0015] Specifically, the heterogeneous system can determine multiple slave nodes whose available processing engines meet the preset condition from the heterogeneous cluster according to the scheduling reference information, respectively determine the scheduling overhead from the first slave node to the multiple slave nodes, and determine the node whose scheduling overhead is less than the preset overhead as the second slave node.

[0016] When this method schedules heterogeneous API calls across nodes, it not only improves the execution efficiency of heterogeneous API calls but also avoids a significant increase in scheduling overhead.

[0017] In some possible implementation manners, the heterogeneous system can determine scheduling reference information by looking up a table. Specifically, the heterogeneous system (specifically, the first slave node in the heterogeneous system) looks up a relationship table according to the parameter scale value of the heterogeneous API call. This relationship table includes the corresponding relationship between the parameter scale value and the execution efficiency of the processing engine. In this way, the scheduling reference information of the heterogeneous API call can be obtained. Based on this scheduling reference information, the heterogeneous API call can be scheduled to a suitable processing engine, improving the execution efficiency of the heterogeneous API call.

[0018] In some possible implementation manners, the multiple slave nodes are homogeneous single-node heterogeneous systems or heterogeneous single-node heterogeneous systems. Among them, a single-node heterogeneous system refers to a heterogeneous system formed by a single node including different types of processing engines.

[0019] A homogeneous single-node heterogeneous system refers to a single-node heterogeneous system with the same structure. For example, if slave nodes 1 to N are all nodes including processing engines 1, 2,..., N, then slave nodes 1 to N are homogeneous single-node heterogeneous systems.

[0020] A heterogeneous single-node heterogeneous system refers to a single-node heterogeneity with different structures. For example, if slave node 1 is a node including processing engines 1 and 2, and slave node 2 is a node including processing engines 3 and 4, then nodes 1 and 2 are heterogeneous single-node heterogeneous systems.

[0021] In this way, it is possible to implement scheduling of heterogeneous API calls within a node or across nodes, thereby improving the execution efficiency of heterogeneous API calls and further improving the overall execution efficiency of tasks.

[0022] In some possible implementation manners, for a multi-node heterogeneous system, each of the multiple slave nodes can be a homogeneous system. That is, one slave node includes one type of processing engine. And at least two slave nodes have different processing engine architectures. For example, slave node 1 includes one type of processing engine, i.e., CPU, and slave node 2 includes one type of processing engine, i.e., GPU. In this way, it is possible to schedule heterogeneous API calls across nodes and improve the execution efficiency of the heterogeneous API call.

[0023] In some possible implementations, multiple slave nodes are nodes in a cloud environment, an edge environment, or a terminal environment. Among them, the cloud environment is specifically a computing cluster including at least one cloud computing device (such as a central server). The edge environment is specifically a computing cluster including at least one edge computing device (such as an edge server). The terminal environment includes at least one terminal computing device. The terminal computing device can be abbreviated as a terminal or a terminal device, including but not limited to a desktop computer, a laptop computer, and a smart phone, etc.

[0024] In a second aspect, the present application provides a task scheduling device. The device is applied to a heterogeneous system including multiple nodes, and the tasks to be executed include at least one heterogeneous API call. The device includes:

[0025] A determination module, configured to determine scheduling reference information for the heterogeneous API call according to the parameter scale value of the heterogeneous API call, where the scheduling reference information is used to describe the execution efficiency of multiple processing engines in the heterogeneous system;

[0026] The determination module is further configured to determine a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the heterogeneous system according to the scheduling reference information;

[0027] A scheduling module, configured to schedule the heterogeneous API call to the corresponding target processing engine.

[0028] In some possible implementations, the target processing engine that meets the preset condition includes: the processing engine with the highest execution efficiency or the processing engine whose execution efficiency is higher than the preset efficiency.

[0029] In some possible implementations, the heterogeneous system includes a heterogeneous cluster, the heterogeneous cluster includes a master node and multiple slave nodes, and the task is scheduled by the master node to a first slave node among the multiple slave nodes;

[0030] The determination module is specifically configured to:

[0031] Determine a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the first slave node according to the scheduling reference information.

[0032] In some possible implementations, the heterogeneous system includes a heterogeneous cluster, the heterogeneous cluster includes a master node and multiple slave nodes, and the task is scheduled by the master node to a first slave node among the multiple slave nodes;

[0033] The determination module is specifically configured to:

[0034] When the processing engine whose execution efficiency meets the preset condition in the first slave node is unavailable, determine a second slave node from the heterogeneous cluster according to the scheduling reference information;

[0035] Determine a target processing engine from the processing engines available to the second slave node, where the execution efficiency of the target processing engine meets a preset condition.

[0036] In some possible implementation manners, the determining module is specifically configured to:

[0037] According to the scheduling reference information, determine that a node in the heterogeneous cluster, for which the available processing engine meets the preset condition and the scheduling overhead is less than the preset overhead, is the second slave node.

[0038] In some possible implementation manners, the determining module is specifically configured to:

[0039] Look up a relationship table according to the parameter scale value of the heterogeneous API call to obtain the scheduling reference information of the heterogeneous API call. The relationship table includes the corresponding relationship between the parameter scale value and the execution efficiency of the processing engine.

[0040] In some possible implementation manners, the multiple slave nodes are a homogeneous single-node heterogeneous system or a heterogeneous single-node heterogeneous system.

[0041] In some possible implementation manners, each of the multiple slave nodes is a homogeneous system, and the architectures of the processing engines of at least two slave nodes are different.

[0042] In some possible implementation manners, the multiple slave nodes are nodes in a cloud environment, an edge environment, or a terminal environment.

[0043] In a third aspect, the present application provides a device, which includes a processor and a memory. The processor and the memory communicate with each other. The processor is configured to execute instructions stored in the memory so that the device executes the task scheduling method in the first aspect or any implementation manner of the first aspect.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored, and the instructions direct a device to execute the task scheduling method in the first aspect or any implementation manner of the first aspect.

[0045] In a fifth aspect, the present application provides a computer program product including instructions, which, when running on a device, cause the device to execute the task scheduling method in the first aspect or any implementation manner of the first aspect.

[0046] Based on the implementation manners provided in the above aspects, the present application can be further combined to provide more implementation manners. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] To more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings required for the embodiments will be briefly introduced below.

[0048] Figure 1A System architecture diagram of a task scheduling method provided by an embodiment of the present application;

[0049] Figure 1B System architecture diagram of a task scheduling method provided by an embodiment of the present application;

[0050] Figure 1C System architecture diagram of a task scheduling method provided by an embodiment of the present application;

[0051] Figure 2 Schematic diagram of edge-cloud collaboration provided by an embodiment of the present application;

[0052] Figure 3 Flowchart of a task scheduling method provided by an embodiment of the present application;

[0053] Figure 4 Flowchart of a task scheduling method provided by an embodiment of the present application;

[0054] Figure 5 Flowchart of a task scheduling method provided by an embodiment of the present application;

[0055] Figure 6 Structural schematic diagram of a task scheduling device provided by an embodiment of the present application;

[0056] Figure 7 Structural schematic diagram of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0057] The terms "first" and "second" in the embodiments of the present application are only used for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0058] First, some technical terms involved in the embodiments of the present application are introduced.

[0059] The processing engine, also known as the hardware engine, is specifically a unit capable of performing data processing operations. The present application does not limit the specific type and form of the processing engine, and any unit capable of performing data processing operations can be used as the processing engine. In some examples, the processing engine may include a central processing unit (CPU).

[0060] Furthermore, the processing engine may also include a processing unit based on a domain specific architecture (DSA). For example, the processing engine may also include any one or more of a graphical processing unit (GPU), an image processor (IP), a digital signal processor (DSP), a neural-network processing unit (NPU), and a field programmable gate array (FPGA).

[0061] A heterogeneous system is a data processing system that includes multiple different types of processing engines. Specifically, the multiple different types of processing engines may include at least one CPU, and the remaining processing engines may be processing engines of a type different from the CPU. For example, the remaining processing engines may include some or all of the following: GPU, IP, DSP, NPU, or FPGA. Of course, in some embodiments, the remaining processing engines may also include a CPU.

[0062] The heterogeneous system may specifically be a multi-node heterogeneous system. A multi-node heterogeneous system refers to a heterogeneous system deployed on multiple computing nodes. In some embodiments, the heterogeneous system may be distributedly deployed on multiple computing nodes. Further, the multiple computing nodes may form a heterogeneous cluster. The heterogeneous cluster includes a master node and multiple slave nodes.

[0063] An application developed for the above heterogeneous system is a heterogeneous application. A heterogeneous application typically includes at least one call to a heterogeneous application programming interface (API). Among them, the heterogeneous API is specifically an API with a unified interface but different optimized implementations on different processing engines.

[0064] One of the multiple processing engines may serve as a scheduler for data interaction with the remaining processing engines and assisting the remaining processing engines in data processing. In the embodiments of the present application, taking the scheduler as the CPU as an example, for the convenience of description, the CPU serving as the scheduler in the embodiments of the present application is referred to as the scheduling CPU, or the host CPU.

[0065] The runtime refers to the component that provides the program running environment. This component may include a virtual machine and a standard function library. Taking Java as an example, the Java runtime (JRE) provides the running environment for Java programs. The JRE includes a Java virtual machine (JVM) and a standard Java function library.

[0066] The runtime running on a heterogeneous system (specifically, the host CPU of the heterogeneous system) is called a heterogeneous runtime (HRT). Among them, the heterogeneous runtime is used to postpone the decision-making for the heterogeneous system from compile time and link time to runtime execution. The decision-making for the heterogeneous system may refer to selecting a processing engine from the heterogeneous system for executing heterogeneous API calls.

[0067] The definition of a heterogeneous API includes the formal parameters (referred to as formal parameters for short) of the heterogeneous API. When calling a heterogeneous API, actual parameters (actual parameters) corresponding to the above formal parameters also need to be provided. The parameter scale of the same heterogeneous API may be different when it is called at different call points. The parameter scale can describe the size of the parameters required to call the heterogeneous API. It should be noted that in the embodiments of the present application, the size of the parameter does not refer to the numerical size of the parameter, but refers to the number of bytes occupied by the parameter in memory or in the processing engine. Based on this, the value of the parameter scale (i.e., the parameter scale value) can take any integer in the interval (0, +∞).

[0068] The parameter scale value can usually be determined according to the expression for calculating the parameter scale. For example, the following statement can be added to the prototype declaration file of the heterogeneous API:

[0069] #pragma HAPI_PARA_SIZE(hapi_para_size_expr)

[0070] Among them, #pragma HAPI_PARA_SIZE indicates that the parameter scale of the heterogeneous API is declared here, and hapi_para_size_expr is the expression for the parameter scale.

[0071] For example, the expression for the parameter scale can be max(max(A.Size(), B.Size()), C.Size())). Here, max(max(A.Size(), B.Size()), C.Size()) means taking the largest value of Size among the three parameters A, B, and C as the parameter scale.

[0072] The embodiments of the present application do not limit the specific content of the expression of the parameter scale. Generally, the operands appearing in the expression can be positive integer constants, positive integer parameters in the corresponding heterogeneous API prototype declaration file, positive integer member variables of the parameters, or member functions with positive integer return values.

[0073] When the heterogeneous API is called, the parameter scale values are different, and its execution efficiency on different processing engines included in the heterogeneous system can be different. In view of the problem that the efficiency of the heterogeneous system in executing tasks including heterogeneous API calls is low, the embodiments of the present application provide a task scheduling method. This method can be executed by the heterogeneous runtime running on the host CPU.

[0074] Specifically, the task includes at least one heterogeneous API call. For any heterogeneous API call, the heterogeneous runtime determines the scheduling reference information of the heterogeneous API call according to the parameter scale value of the heterogeneous API call. This scheduling reference information is used to describe the execution efficiency of various processing engines in the multi-node heterogeneous system. Then, the heterogeneous runtime determines the target processing engine whose execution efficiency meets the preset conditions from the available processing engines of the heterogeneous system, and then schedules the heterogeneous API call to the corresponding target processing engine.

[0075] This method supports selecting a suitable processing engine according to the parameter scale value of the heterogeneous API call, scheduling the heterogeneous API call to the above-mentioned suitable processing engine for execution, improving the execution efficiency of the heterogeneous API call, and further improving the execution efficiency of the task.

[0076] Among them, the heterogeneous system includes multiple nodes. In some embodiments, the heterogeneous system can be a heterogeneous cluster, which includes a master node and multiple slave nodes. When a task is scheduled to a slave node, the heterogeneous runtime on the slave node can select a suitable processing engine from the available processing engines of the slave node and schedule the heterogeneous API call to the processing engine for execution. Further, when the processing engine with high execution efficiency on the current slave node is unavailable, the heterogeneous runtime on the slave node can select a suitable processing engine from other slave nodes and schedule the heterogeneous API call to the processing engine for execution, so as to further improve the execution efficiency of the heterogeneous API call and the execution efficiency of the task.

[0077] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the task scheduling method provided by the embodiments of the present application will be introduced below with reference to the accompanying drawings.

[0078] See Figure 1AThe system architecture diagram shown, the heterogeneous system 100 includes a master node 102 and multiple slave nodes 104. Among them, the multiple slave nodes 104 are homogeneous single-node heterogeneous systems, that is, the multiple slave nodes 104 are single-node heterogeneous systems with the same structure (nodes including multiple processing engines). As Figure 1A shown, the slave node 104 includes processing engine 1, processing engine 2,..., processing engine N (N is a positive integer). Among them, processing engine 1 is a scheduling processing engine. In some embodiments, processing engine 1 is a host CPU. The heterogeneous runtime is included in processing engine 1.

[0079] When a task is scheduled by the master node 102 to a slave node 104, the heterogeneous runtime running on the processing engine 1 of the slave node 104 can determine the scheduling reference information of the heterogeneous API call according to the parameter scale value included in the task. The heterogeneous runtime can determine a target processing engine whose execution efficiency meets the preset conditions from the available processing engines of the heterogeneous system 100, and schedule the heterogeneous API call to the corresponding target processing engine.

[0080] In some possible implementation manners, the multiple slave nodes 104 can also be heterogeneous single-node heterogeneous systems. That is, the multiple slave nodes 104 can be single-node heterogeneous systems with different structures. As Figure 1B shown, the first slave node 104 includes processing engine 11, processing engine 12,..., processing engine 1N, the second slave node 104 includes processing engine 21, processing engine 22,..., processing engine 2N, and so on. The Mth slave node 104 includes processing engine M1, processing engine M2,..., processing engine MN. Among them, N and M are positive integers. It should be noted that Figure 1B the example is described with the number of types of processing engines included in each slave node 104 being N. In some embodiments, the number of types of processing engines included in different slave nodes 104 can also be different. For example, one slave node 104 can include N types of processing engines, and another slave node 104 can include K types of processing engines, where K is a positive integer. Processing engines 11, 21,..., M1 are the scheduling processing engines of each slave node 104. The heterogeneous runtime is included in the scheduling processing engine.

[0081] When a task is scheduled by the master node 102 to a slave node 104, for example, the first slave node 104, the heterogeneous runtime running on the processing engine 11 of the slave node 104 can determine the scheduling reference information of the heterogeneous API call according to the parameter scale value included in the task. The heterogeneous runtime determines a target processing engine whose execution efficiency meets the preset conditions from the available processing engines of the heterogeneous system 100, and schedules the heterogeneous API call to the corresponding target processing engine.

[0082] In Figure 1A , Figure 1B the embodiment shown, the heterogeneous runtime determines a target processing engine from the processing engines available in the heterogeneous system 100, which may be to determine a target processing engine with execution efficiency meeting a preset condition from the processing engines available within a node. In some other possible implementation manners, the heterogeneous runtime may also determine a target processing engine from the processing engines available in other slave nodes 104 to implement cross-node scheduling of heterogeneous API calls.

[0083] In some possible implementation manners, each of the multiple slave nodes 104 is a homogeneous system. And, the architectures of the processing engines of at least two slave nodes 104 are different. As Figure 1C shown, the processing engines of the first slave node 104 are all processing engine 1, the processing engines of the second slave node 104 are all processing engine 2, and so on, the processing engines of the Nth slave node 104 are all processing engine N. Each slave node 104 includes a scheduling processing engine, and the scheduling processing engine includes a heterogeneous runtime.

[0084] When a task is scheduled by the master node 102 to a slave node 104, the heterogeneous runtime running on the processing engine of this slave node 104 determines scheduling reference information for the heterogeneous API call according to the parameter scale value of the heterogeneous API call included in the task. The heterogeneous runtime determines a target processing engine with execution efficiency meeting a preset condition from the processing engines available in the heterogeneous system 100 (including each slave node 104) according to this scheduling reference information, and schedules the heterogeneous API call to the corresponding target processing engine.

[0085] It should be noted that the above multiple slave nodes 104 may be nodes in a cloud environment, an edge environment or a terminal environment as Figure 2 shown. Among them, the cloud environment is specifically a computing cluster including at least one cloud computing device (such as a central server). The edge environment is specifically a computing cluster including at least one edge computing device (such as an edge server). The terminal environment includes at least one terminal computing device. The terminal computing device may be simply referred to as a terminal or a terminal device, including but not limited to a desktop computer, a laptop computer, and a smart phone, etc.

[0086] The system architecture of the task scheduling method is described in detail above. Next, the task scheduling method provided by the embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0087] Refer to Figure 3 the flowchart of the task scheduling method shown. This method includes:

[0088] S302: The heterogeneous runtime determines scheduling reference information for the heterogeneous API call according to the parameter scale value of the heterogeneous API call.

[0089] The scheduling reference information is used to describe the execution efficiency of multiple processing engines in a heterogeneous system. The heterogeneous runtime can preferentially schedule heterogeneous API calls to the processing engine with high execution efficiency. Therefore, the execution efficiency can characterize the scheduling priority of the processing engine.

[0090] The execution efficiency refers to the amount of work for executing heterogeneous API calls per unit time. The execution efficiency can be determined according to the execution time of the heterogeneous API. Assuming that the amount of work for executing a heterogeneous API call is 1, the execution efficiency is the reciprocal of the execution time. In some embodiments, the scheduling reference information may include the execution efficiencies of multiple processing engines. In other embodiments, the scheduling reference information may include the execution times of multiple processing engines, and the execution times can be used to describe the execution efficiency.

[0091] Of course, the scheduling reference information may also be a rank obtained by sorting the execution efficiency or the execution time, and this rank can describe the level of the execution efficiency. Specifically, the scheduling reference information may be a rank obtained by sorting the execution efficiency or the execution time in descending or ascending order. This rank can be used as the scheduling priority of the heterogeneous API call. Taking the rank obtained by sorting the execution time in ascending order as an example, the higher the rank, the higher the scheduling priority, and the lower the rank, the lower the scheduling priority.

[0092] Among them, the format of the scheduling reference information is diverse. For example, the scheduling reference information can be in vector format, or in key-value pair format, or in other formats, which will not be listed one by one in the embodiments of this application.

[0093] In some embodiments, the scheduling reference information may be a scheduling priority vector. The element value of each element of the scheduling priority vector is respectively used to characterize the scheduling priority of a processing engine. In other embodiments, the scheduling reference information may also be a key-value pair, which includes the processing engine and its corresponding scheduling priority.

[0094] Specifically, the heterogeneous runtime can determine the scheduling reference information of the heterogeneous API call according to the parameter scale value of the heterogeneous API call and in combination with the pre-constructed relationship between the parameter scale and the processing engine. Among them, the relationship between the parameter scale and the processing engine describes the scheduling reference information corresponding to different parameter scale values of the heterogeneous API. The heterogeneous runtime can determine the corresponding scheduling reference information according to the identifier of the heterogeneous API and the parameter scale value of the heterogeneous API call.

[0095] The relationship between the parameter scale and the processing engine can be presented by one or more of a relationship table, a relationship text, or a relationship diagram. For ease of description, the following uses the relationship table to present the relationship between the parameter scale and the processing engine as an example. Among them, the relationship table is mainly used to help select a suitable processing engine for heterogeneous runtimes. Therefore, the above relationship table can also be called a processing engine selection table. For ease of understanding, the embodiments of the present application also provide an example of the relationship table as follows:

[0096] Table 1 Relationship Table between Parameter Scale and Processing Engine

[0097]

[0098]

[0099] As can be seen from the above table, the heterogeneous runtime can look up the relationship table according to the identifier of the heterogeneous API to obtain the scheduling priority vector corresponding to the heterogeneous API call under different parameter scale values. Then, the heterogeneous runtime can look up the relationship table according to the parameter scale value to obtain the scheduling priority vector corresponding to this parameter scale value.

[0100] S304: The heterogeneous runtime determines a target processing engine whose execution efficiency meets the preset conditions from the processing engines available in the heterogeneous system according to the scheduling reference information.

[0101] The available processing engine refers to a processing engine with idle resources. In some embodiments, the available processing engine can be a processing engine with a load rate less than a preset threshold. The processing engines available in the heterogeneous system can be the processing engines available within the current node or the processing engines available for the entire heterogeneous system (including other nodes, such as other slave nodes).

[0102] The heterogeneous runtime can determine a target processing engine whose execution efficiency meets the preset conditions from the processing engines available within the current node (such as the first slave node) according to the scheduling reference information. Among them, the target processing engine that meets the preset conditions can be the processing engine with the highest execution efficiency or a processing engine with an execution efficiency higher than the preset efficiency. When the scheduling reference information is the ranking of the execution efficiencies of multiple processing engines, that is, the scheduling priorities of multiple processing engines, the processing engine with the highest execution efficiency can be the processing engine ranked at the top, and the processing engines with an execution efficiency higher than the preset efficiency can be several processing engines ranked at the top.

[0103] In some possible implementations, a task is scheduled by the master node to the first slave node among multiple slave nodes 104. When the processing engine whose execution efficiency meets the preset condition in the first slave node is unavailable, the heterogeneous runtime can determine a second slave node from the heterogeneous cluster according to the scheduling reference information, where the second slave node is a slave node whose execution efficiency of the available processing engine meets the preset condition. Then, the heterogeneous runtime can determine a target processing engine whose execution efficiency meets the preset condition from the available processing engines of the second slave node described above.

[0104] Furthermore, additional overheads are required for cross-node scheduling of heterogeneous API calls, such as the overhead of transmitting the actual parameters of the heterogeneous API, the overhead of writing back the results, etc. For this reason, the heterogeneous runtime can also consider the overhead when determining the second slave node to determine a suitable second slave node.

[0105] Specifically, the heterogeneous runtime can determine, according to the scheduling reference information, that a node in the heterogeneous cluster whose available processing engine meets the preset condition and whose scheduling overhead is less than the preset overhead is the second slave node. In some embodiments, the heterogeneous runtime can determine multiple slave nodes in the heterogeneous cluster whose available processing engines meet the preset condition, and then respectively determine the scheduling overheads for scheduling the heterogeneous API call from the first slave node to the multiple slave nodes described above. The heterogeneous runtime determines that the node with a scheduling overhead less than the preset overhead is the second slave node.

[0106] S306: The heterogeneous runtime schedules the heterogeneous API call to the corresponding target processing engine.

[0107] When the target processing engine is a processing engine within a node, the heterogeneous runtime directly schedules the heterogeneous API call to the corresponding target processing engine. When the target processing engine includes a processing engine on other nodes, the heterogeneous runtime can also schedule the heterogeneous API call across nodes to the corresponding target processing engine.

[0108] In some embodiments, the master node schedules multiple tasks to a slave node. If at least two tasks among the multiple tasks include heterogeneous API calls whose corresponding target processing engines include the same processing engine, such as the processing engine with the highest scheduling priority is the same processing engine, and the remaining resources of this processing engine are insufficient to support the execution of multiple tasks, then the heterogeneous runtime running on this slave node can also schedule some tasks to the processing engine with the second highest scheduling priority (the scheduling priority of this processing engine is higher than the preset level).

[0109] Based on the above description, an embodiment of the present application provides a task scheduling method. In this method, the heterogeneous runtime determines scheduling reference information for a heterogeneous API call according to the parameter scale value of the heterogeneous API call. Then, the heterogeneous runtime determines a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the heterogeneous system according to the scheduling reference information, and then schedules the heterogeneous API call to the corresponding target processing engine.

[0110] This method supports selecting a suitable processing engine according to the parameter scale value of the heterogeneous API call, scheduling the heterogeneous API call to the above-mentioned suitable processing engine for execution, improving the execution efficiency of the heterogeneous API call, and thus improving the execution efficiency of the task. Moreover, when the execution efficiency of the processing engines available within a node does not meet the preset condition, this method can also support selecting a processing engine whose execution efficiency meets the preset condition from the processing engines available in other nodes as the target processing engine, and scheduling the heterogeneous API call across nodes to the above-mentioned target processing engine to improve the execution efficiency of the heterogeneous API call, and thus improve the execution efficiency of the task. In addition, selecting a suitable scheduling engine from the entire heterogeneous system to execute the heterogeneous API call is beneficial to improving the overall resource utilization rate of the heterogeneous system.

[0111] Next, the specific implementations of in-node scheduling and cross-node scheduling will be described in detail with reference to the accompanying drawings.

[0112] See Figure 4 , the master node and the slave nodes each maintain a heterogeneous application task queue. The heterogeneous application task queue is used to store heterogeneous application tasks. The master node is mainly responsible for scheduling heterogeneous application tasks to the slave nodes and is responsible for the load balancing of the heterogeneous application tasks on the slave nodes. The slave nodes are responsible for executing the heterogeneous application tasks scheduled to this node. Among them, when executing a heterogeneous application task, a slave node can use only the processing engines within this node to execute the heterogeneous API calls included in the heterogeneous application task.

[0113] Specifically, the master node schedules the heterogeneous application tasks to the slave nodes through the following processing logic:

[0114] The master node detects the heterogeneous application task queue of the master node. When the heterogeneous application task queue is not empty, the master node finds the slave node n with the relatively empty (e.g., the emptiest) heterogeneous application task queue among all the slave nodes, and then removes the first heterogeneous application task (specifically, task t) from the heterogeneous application task queue of the master node and adds task t to the heterogeneous application task queue of the slave node n.

[0115] Furthermore, the master node can also balance the load of the slave nodes through the following processing logic:

[0116] When the master node detects that the heterogeneous application task queue of slave node m is empty, it finds a slave node n with a relatively full heterogeneous application task queue (e.g., the fullest) among other slave nodes, removes the first heterogeneous application task (specifically task t) from the heterogeneous application task queue of slave node n, and adds task t to the heterogeneous application task queue of slave node m.

[0117] Slave node n executes the task mainly through the following processing logic:

[0118] When the heterogeneous application task queue of slave node n is not empty, slave node n removes the first heterogeneous application task (specifically task t) from the heterogeneous application task queue of this slave node n, reports the status change of task t to the master node, specifically from the waiting state to the execution state, and slave node n executes task t on this single-node heterogeneous system. Among them, slave node n determines the scheduling reference information of the heterogeneous API call according to the parameter scale value of the heterogeneous API call in task t, determines the target processing engine whose execution efficiency meets the preset conditions according to this scheduling reference information, and schedules the heterogeneous API call to the target processing engine for execution. After the execution is completed, slave node n can also report the status change of task t to the master node, specifically from the execution state to the completion state.

[0119] Figure 4 This mainly elaborates on the processing logics of the master node and slave nodes during the in-node scheduling process. Next, the processing logics of the master node and slave nodes during the cross-node scheduling process will be elaborated in detail.

[0120] See Figure 5 , the master node and slave nodes each maintain a heterogeneous application task queue. The heterogeneous application task queue is used to store heterogeneous application tasks. The master node is mainly responsible for scheduling heterogeneous application tasks to slave nodes and for load balancing of heterogeneous application tasks on slave nodes. The slave node is responsible for executing the heterogeneous application tasks scheduled to this node.

[0121] During the execution of heterogeneous application tasks by the slave node, specifically, it is implemented through the heterogeneous runtime. The heterogeneous runtime obtains the scheduling priority vector by retrieving the processing engine selection table and determines the processing engine Eh whose priority meets the preset conditions (execution efficiency meets the preset conditions). When this processing engine is unavailable, such as being occupied, the heterogeneous runtime on the current slave node n can check whether Eh on other slave nodes is available. If Eh on a certain slave node m is available and the overhead of executing this heterogeneous API call on m is less than the preset overhead, then it can request slave node m to execute this heterogeneous API call.

[0122] When each node in the heterogeneous cluster shares a global memory pool, the overhead of passing actual parameters and writing back results between nodes can be greatly reduced. Slave node n can schedule a heterogeneous API call to slave node m, and execute the heterogeneous API call on the processing engine Eh of slave node m.

[0123] It should be noted that the heterogeneous runtime on slave node n checks whether the Eh on other slave nodes is available. Specifically, it is obtained by querying the master node. The processing logic of the master node can be referred to Figure 4 In the illustrated embodiment, the processing logic of the slave node is as follows:

[0124] Slave node n (specifically, the heterogeneous runtime of slave node n) looks up the processing engine selection table through the API ID and the parameter scale value, and obtains the corresponding scheduling priority vector v. For the processing engine Eh in this slave node n whose scheduling priority meets the preset conditions. If this Eh is available, mark the processing engine Eh of this slave node n as unavailable, execute the current heterogeneous API call on the Eh of this slave node n, and when the execution is completed, mark the processing engine Eh of this slave node n as available.

[0125] If the Eh of this slave node n is unavailable, send a query request to the master node, requesting to query a slave node whose Eh is available and the overhead of executing the current heterogeneous API call is less than the preset overhead. Slave node n receives the query response returned by the master node. The query response carries the identifier of slave node m queried by the master node. Slave node n sends a scheduling request to slave node m according to the identifier of slave node m, requesting to schedule the heterogeneous API call to slave node m.

[0126] As described above in conjunction with FIGS. 1 to Figure 5 The task scheduling method provided by the embodiments of the present application has been introduced in detail. Next, the task scheduling device and equipment provided by the embodiments of the present application will be introduced in conjunction with the accompanying drawings.

[0127] See Figure 6 In the structural schematic diagram of the task scheduling device shown, the device 600 is applied to a heterogeneous system including multiple nodes. The task includes at least one heterogeneous application programming interface API call. The device 600 includes:

[0128] A determination module 602, configured to determine scheduling reference information for the heterogeneous API call according to the parameter scale value of the heterogeneous API call. The scheduling reference information is used to describe the execution efficiency of multiple processing engines in the heterogeneous system;

[0129] The determination module 602 is further configured to determine a target processing engine whose execution efficiency meets the preset conditions from the available processing engines in the heterogeneous system according to the scheduling reference information;

[0130] A scheduling module 604, configured to schedule heterogeneous API calls to corresponding target processing engines.

[0131] In some possible implementation manners, the target processing engines that meet the preset conditions include: the processing engine with the highest execution efficiency or the processing engine whose execution efficiency is higher than the preset efficiency.

[0132] In some possible implementation manners, the heterogeneous system includes a heterogeneous cluster, which includes a master node and multiple slave nodes, and tasks are scheduled by the master node to a first slave node among the multiple slave nodes;

[0133] The determining module 602 is specifically configured to:

[0134] Determine a target processing engine whose execution efficiency meets the preset conditions from the processing engines available to the first slave node according to the scheduling reference information.

[0135] In some possible implementation manners, the heterogeneous system includes a heterogeneous cluster, which includes a master node and multiple slave nodes, and tasks are scheduled by the master node to a first slave node among the multiple slave nodes;

[0136] The determining module 602 is specifically configured to:

[0137] When the processing engines in the first slave node whose execution efficiency meets the preset conditions are unavailable, determine a second slave node from the heterogeneous cluster according to the scheduling reference information;

[0138] Determine a target processing engine whose execution efficiency meets the preset conditions from the processing engines available to the second slave node.

[0139] In some possible implementation manners, the determining module 602 is specifically configured to:

[0140] Determine, according to the scheduling reference information, that a node in the heterogeneous cluster whose available processing engines meet the preset conditions and whose scheduling overhead is less than the preset overhead is the second slave node.

[0141] In some possible implementation manners, the determining module 602 is specifically configured to:

[0142] Search a relationship table according to the parameter scale value of the heterogeneous API call to obtain the scheduling reference information of the heterogeneous API call. The relationship table includes the corresponding relationship between the parameter scale value and the execution efficiency of the processing engine.

[0143] In some possible implementation manners, the multiple slave nodes are homogeneous single-node heterogeneous systems or heterogeneous single-node heterogeneous systems.

[0144] In some possible implementation manners, each of the multiple slave nodes is a homogeneous system, and the architectures of the processing engines of at least two slave nodes are different.

[0145] In some possible implementations, multiple slave nodes are nodes in a cloud environment, an edge environment, or a terminal environment.

[0146] The task scheduling device 600 according to the embodiments of the present application may correspond to executing the methods described in the embodiments of the present application, and the above and other operations and / or functions of each module / unit of the task scheduling device 600 are respectively for implementing Figure 3 the corresponding processes of the respective methods in the illustrated embodiments. For the sake of brevity, they will not be described herein again.

[0147] The embodiments of the present application further provide a computing device 700. The computing device 700 may be an end-side device such as a laptop computer or a desktop computer, or a cloud computing device in a cloud environment, such as a central server in a central cloud or an edge server in an edge cloud. The computing device 700 is specifically used to implement Figure 6 the functions of the task scheduling device 600 in the illustrated embodiments.

[0148] Figure 7 A structural schematic diagram of a computing device 700 is provided. As Figure 7 shown, the computing device 700 includes a bus 701, a processor 702, a communication interface 703, and a memory 704. The processor 702, the memory 704, and the communication interface 703 communicate with each other through the bus 701.

[0149] The bus 701 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 7 only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0150] The processor 702 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0151] The communication interface 703 is used for external communication. For example, obtaining the parameter scale value of a heterogeneous API call, obtaining the identifier of a slave node whose available processing engine meets a preset condition, and so on.

[0152] The memory 704 may include volatile memory, such as random access memory (RAM). The memory 704 may also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0153] The executable code is stored in the memory 704, and the processor 702 executes the executable code to perform the foregoing task scheduling method. Specifically, in the case of implementing Figure 6 the illustrated embodiment, and Figure 6 when each module of the task scheduling device 600 described in the embodiment is implemented by software, the software or program code required to execute the functions of each module in Figure 6 is stored in the memory 704. The processor 702 executes the program code stored in the memory 704 to execute the task scheduling method in Figure 3 the illustrated embodiment.

[0154] An embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that a computing device can store or a data storage device such as a data center that includes one or more available media. The available media may be magnetic media (e.g., floppy disk, hard disk, magnetic tape), optical media (e.g., DVD), or semiconductor media (e.g., solid state drive), etc. The computer-readable storage medium includes instructions that direct the computing device to execute the above-described task scheduling method applied to the task scheduling device 600.

[0155] An embodiment of the present application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, they wholly or partially generate the processes or functions described in the embodiments of the present application.

[0156] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center by wire (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (e.g., infrared, wireless, microwave, etc.).

[0157] The computer program product may be a software installation package. In the case of needing to use any of the foregoing task scheduling methods, the computer program product can be downloaded and executed on a computing device.

[0158] The descriptions of the processes or structures corresponding to the foregoing various drawings each have their own focuses. For parts not detailed in a certain process or structure, reference may be made to the relevant descriptions of other processes or structures.

Claims

1. A task scheduling method, characterized in that, Applied to a heterogeneous system including multiple nodes, where the task includes at least one heterogeneous application programming interface (API) call, the method includes: Determine scheduling reference information for the heterogeneous API call according to the parameter scale value of the heterogeneous API call, where the parameter scale value is used to describe the size of the parameters required for calling the heterogeneous API, and the size of the parameters refers to the number of bytes occupied by the parameters in memory or in the processing engine, and the scheduling reference information is used to describe the execution efficiency of various processing engines in the heterogeneous system; Determine a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the heterogeneous system according to the scheduling reference information, where the target processing engine that meets the preset condition includes: the processing engine with the highest execution efficiency or the processing engine whose execution efficiency is higher than the preset efficiency; Schedule the heterogeneous API call to the target processing engine.

2. The method according to claim 1, wherein The heterogeneous system includes a heterogeneous cluster, the heterogeneous cluster includes a master node and multiple slave nodes, and the task is scheduled by the master node to a first slave node among the multiple slave nodes; The step of determining a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the heterogeneous system according to the scheduling reference information includes: Determine a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the first slave node according to the scheduling reference information.

3. The method according to claim 1, wherein The heterogeneous system includes a heterogeneous cluster, the heterogeneous cluster includes a master node and multiple slave nodes, and the task is scheduled by the master node to a first slave node among the multiple slave nodes; The step of determining a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the heterogeneous system according to the scheduling reference information includes: When the processing engine in the first slave node whose execution efficiency meets the preset condition is unavailable, determine a second slave node from the heterogeneous cluster according to the scheduling reference information; Determine a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the second slave node.

4. The method according to claim 3, characterized in that, The step of determining a second slave node from the heterogeneous cluster according to the scheduling reference information includes: Determine, according to the scheduling reference information, that a node in the heterogeneous cluster whose available processing engines meet the preset condition and whose scheduling overhead is less than the preset overhead is the second slave node.

5. The method according to any one of claims 1 to 4, characterized in that, The step of determining the scheduling reference information for the heterogeneous API call according to the parameter scale value of the heterogeneous API call includes: Look up a relationship table according to the parameter scale value of the heterogeneous API call to obtain the scheduling reference information for the heterogeneous API call, where the relationship table includes the correspondence between the parameter scale value and the execution efficiency of the processing engine.

6. The method according to any one of claims 2 to 4, characterized in that The multiple slave nodes are homogeneous single-node heterogeneous systems or heterogeneous single-node heterogeneous systems.

7. The method according to claim 3 or 4, characterized in that, Each of the multiple slave nodes is a homogeneous system, and the architectures of the processing engines of at least two slave nodes are different.

8. The method according to any one of claims 2 to 4, characterized in that The multiple slave nodes are nodes in a cloud environment, an edge environment, or a terminal environment.

9. A task scheduling device, characterized in that, Applied to a heterogeneous system including multiple nodes, where the task includes at least one heterogeneous application programming interface (API) call, the apparatus includes: A determination module, configured to determine scheduling reference information for the heterogeneous API call according to a parameter scale value of the heterogeneous API call, where the parameter scale value is used to describe the size of the parameters required for calling the heterogeneous API, and the size of the parameters refers to the number of bytes occupied by the parameters in memory or in the processing engine, and the scheduling reference information is used to describe the execution efficiency of multiple processing engines in the heterogeneous system; The determination module is further configured to determine, according to the scheduling reference information, a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the heterogeneous system, where the target processing engine meeting the preset condition includes: the processing engine with the highest execution efficiency or the processing engine whose execution efficiency is higher than the preset efficiency; A scheduling module, configured to schedule the heterogeneous API call to the target processing engine.

10. The device according to claim 9, characterized in that, The heterogeneous system includes a heterogeneous cluster, the heterogeneous cluster includes a master node and multiple slave nodes, and the task is scheduled by the master node to a first slave node among the multiple slave nodes; The determination module is specifically configured to: Determine, according to the scheduling reference information, a target processing engine whose execution efficiency meets a preset condition from the processing engines available in the first slave node.

11. The device according to claim 9, characterized in that, The heterogeneous system includes a heterogeneous cluster, the heterogeneous cluster includes a master node and multiple slave nodes, and the task is scheduled by the master node to a first slave node among the multiple slave nodes; The determination module is specifically configured to: When the processing engine whose execution efficiency meets the preset condition in the first slave node is unavailable, determine a second slave node from the heterogeneous cluster according to the scheduling reference information; Determine, from the processing engines available in the second slave node, a target processing engine whose execution efficiency meets the preset condition.

12. The device according to claim 11, wherein The determination module is specifically configured to: Determine, according to the scheduling reference information, that a node in the heterogeneous cluster whose available processing engines meet the preset condition and whose scheduling overhead is less than the preset overhead is the second slave node.

13. The device according to any one of claims 9 to 12, characterized in that, The determination module is specifically configured to: Look up a relationship table according to the parameter scale value of the heterogeneous API call to obtain the scheduling reference information for the heterogeneous API call, where the relationship table includes the corresponding relationship between the parameter scale value and the execution efficiency of the processing engine.

14. The device according to any one of claims 10 to 12, characterized in that, The multiple slave nodes are homogeneous single-node heterogeneous systems or heterogeneous single-node heterogeneous systems.

15. The device according to claim 11 or 12, characterized in that, Each of the multiple slave nodes is a homogeneous system, and the architectures of the processing engines of at least two slave nodes are different.

16. The device according to any one of claims 10 to 12, characterized in that, The multiple slave nodes are nodes in a cloud environment, an edge environment, or a terminal environment.

17. A computing device, characterized in that, The computing device includes a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the computing device executes the method according to any one of claims 1 to 8.

Images