Heterogeneous compute allocation method and apparatus, and related device

Abstract

The present application provides a heterogeneous compute allocation method and apparatus, and a related device. The method comprises: a management device acquires a computation graph corresponding to a computing task, and partitions the computation graph into multiple subgraphs, wherein the computation graph comprises multiple nodes, each of the multiple nodes represents an operator, and each of the multiple subgraphs comprises at least one node; then, the management device deploys each subgraph onto different types of computing devices for execution, and acquires performance data for each subgraph on different types of computing devices; and finally, the management device determines a target deployment scheme from among multiple deployment schemes which are determined on the basis of the performance data for each subgraph on different types of computing devices, the number of different types of computing devices, and a dependency relationship between the subgraphs. Different operators can be deployed onto computing devices suitable for executing the operators, so that compute allocation results are more accurate, and heterogeneous computing efficiency is improved.

Description

A method, apparatus and related equipment for heterogeneous computing power partitioning

[0001] This application claims priority to Chinese Patent Application No. 202510132961.6, filed on February 6, 2025, entitled "A Method, Apparatus and Related Equipment for Heterogeneous Computing Power Sharing", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application belongs to the field of computer technology and relates to a method, apparatus and related equipment for heterogeneous computing power partitioning. Background Technology

[0003] Heterogeneous computing refers to computing through the joint use of different types of hardware resources. These different types of hardware resources include central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), field programmable gate arrays (FPGAs), etc. Heterogeneous computing can leverage the advantages of different types of hardware resources to improve overall computing performance.

[0004] Computational power partitioning refers to dividing a computing task into multiple smaller subtasks and then allocating them to different types of hardware resources for processing. Computational power partitioning has important application value in large-scale computing. For example, in the training or inference process of artificial intelligence models, the efficiency of the system can be improved by deploying the operators included in the model to different computing devices for parallel execution. However, current computational power partitioning algorithms cannot effectively evaluate the performance of different operators on different computing devices, resulting in inaccurate computational power partitioning results. Summary of the Invention

[0005] This application provides a method, apparatus, and related equipment for heterogeneous computing power partitioning, which can improve the accuracy of computing power partitioning and the efficiency of heterogeneous computing.

[0006] Firstly, this application provides a heterogeneous computing power partitioning method. A management device acquires a computing graph corresponding to a computing task and divides the computing graph into multiple subgraphs. Each computing graph includes multiple nodes, and each node represents an operator. Each subgraph includes at least one node. The management device then deploys each subgraph to different types of computing devices and executes it, acquiring performance data for each subgraph on different types of computing devices. This performance data includes the execution time of each subgraph on different types of computing devices. Finally, the management device determines a target deployment scheme from multiple deployment schemes determined based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph. These multiple deployment schemes are various ways to deploy multiple subgraphs to different types of computing devices, and the target deployment scheme is the deployment scheme with the shortest computation time among the multiple deployment schemes.

[0007] By implementing the above scheme, the computation graph corresponding to a computing task is divided into multiple subgraphs, and each subgraph is deployed to different types of computing devices for execution. The execution time of operators corresponding to the multiple nodes of each subgraph on different computing devices is obtained, which determines the affinity between different subgraphs and different computing devices. Affinity refers to whether a subgraph is suitable for execution on a certain type of computing device. The shorter the execution time of a subgraph on a certain type of device, the more suitable it is for execution on that type of computing device. Since there are multiple subgraphs and multiple types of computing devices, different deployment schemes can be obtained. For a deployment scheme, the management device can determine the computing devices deployed for each subgraph in that scheme. Based on the performance data mentioned above, the execution time of the computing task corresponding to that deployment scheme can be obtained. Then, based on the execution time of different deployment schemes, a target deployment scheme is determined. According to the target deployment scheme, the operators corresponding to each subgraph are deployed to different computing devices, ensuring that different operators are deployed to computing devices suitable for executing those operators. This makes the result of computing power allocation more accurate and improves the efficiency of heterogeneous computing.

[0008] In one possible implementation, determining the target deployment scheme from multiple deployment schemes determined based on performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph includes: the management device using a dynamic programming algorithm to determine the target deployment scheme from the multiple deployment schemes based on performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph.

[0009] Dynamic programming is a method that decomposes a complex problem into multiple smaller subproblems, defines the state of each subproblem, and solves the optimal solution by recursively applying state transition equations to the multiple subproblems. In this application, a computation graph is divided into multiple subgraphs, and multiple different types of computing devices are included. Deploying operators in multiple subgraphs onto multiple different types of computing devices includes multiple deployment schemes. Dynamic programming can quickly and accurately find an optimal deployment scheme, thereby improving the efficiency of computing power allocation.

[0010] In one possible implementation, the different types of computing devices include central processing units (CPUs) and heterogeneous accelerators. The management device uses a dynamic programming algorithm to determine a target deployment scheme from the multiple deployment schemes based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph. This includes: the management device obtaining the state transition equation dp[I][m][n], where dp[I][m][n] represents the execution time of deploying prefix I using m heterogeneous accelerators and n CPUs, where prefix I includes some or all of the subgraphs in the multiple subgraphs; obtaining device configuration information, which includes the number of heterogeneous accelerators M and the number of CPUs N; where 0 ≤ m ≤ M; and the management device determining the target deployment scheme from the multiple deployment schemes based on the state transition equation, the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph. The multiple deployment schemes include deploying prefix I to m heterogeneous accelerators and deploying prefix I to n CPUs.

[0011] In one possible implementation, the above-mentioned division of the computation graph into multiple subgraphs includes: a management device obtaining operator configuration information, which includes at least one target operator used to divide the computation graph; and then the management device dividing the computation graph into the multiple subgraphs according to the operator configuration information; wherein each of the multiple subgraphs includes a node corresponding to at least one target operator.

[0012] The aforementioned operator configuration information can be configured by the user according to the actual computing task. For different computing tasks, users can configure different target operators, thus making the computing power allocation method applicable to different scenarios. For example, different models are used in different scenarios such as image recognition, natural language processing, and recommendation systems. Different models include different operators. Users can configure the corresponding target operators according to their actual needs by configuring the operator configuration information. Furthermore, by dividing the computation graph into multiple subgraphs and deploying them on different types of computing devices at the subgraph level to obtain the affinity between different operators and different types of computing devices, compared to deploying operators on different types of computing devices to obtain affinity, the computational load can be reduced, and the efficiency of heterogeneous computing power allocation can be improved.

[0013] In one possible implementation, the above-mentioned division of the computation graph into multiple subgraphs based on operator configuration information includes: the management device traversing the computation graph starting from the starting node of the computation graph, and when it is determined that the operator represented by the first node belongs to the target operator, dividing the nodes between the first node and the second node, as well as the first node, into a subgraph; wherein, the operator corresponding to the second node belongs to the target operator, and the second node is located before the first node in the computation graph.

[0014] In one possible implementation, the operator configuration information also includes an in-degree threshold; when the management device determines that the operator represented by the first node belongs to the target operator, it divides the nodes between the first node and the second node and the first node into a subgraph, including: when it is determined that the operator represented by the first node belongs to the target operator and the in-degree of the first node is greater than or equal to the in-degree threshold, it divides the nodes between the first node and the second node and the first node into a subgraph.

[0015] If the in-degree of a node is small, it means that the operator corresponding to that node processes a small amount of data, and therefore the computation is small. Only when the in-degree of a node is greater than or equal to the threshold is that node used as a node to partition a subgraph. Subsequently, the operators corresponding to the nodes in each subgraph are deployed to the same computing device on a per-subgraph basis, so as to make full use of the computing power of the computing device.

[0016] In one possible implementation, obtaining the performance data of each subgraph on different types of computing devices includes: a management device obtaining device configuration information, which includes the type of computing device used to deploy the computation graph; then the computing device deploys each subgraph to the different types of computing devices included in the device configuration information and executes it, obtaining the performance data of each subgraph on the different types of computing devices; wherein, the different types of computing devices include at least two of the following: central processing unit (CPU), graphics processing unit (GPU), tensor processing unit (TPU), neural network processing unit (NPU), and field programmable gate array (FPGA).

[0017] In one possible implementation, the computation task is the recommendation task of a recommendation system, and the target operators include the embedding lookup operator and the concat operator.

[0018] Secondly, this application provides a heterogeneous computing power partitioning device, including an acquisition module, a subgraph partitioning module, a testing module, and a processing module. The acquisition module is used to acquire a computation graph corresponding to a computing task, the computation graph including multiple nodes, each node representing an operator. The subgraph partitioning module is used to partition the computation graph into multiple subgraphs, each subgraph including at least one node. The testing module is used to deploy each subgraph to different types of computing devices and execute it, acquiring performance data of each subgraph on different types of computing devices, the performance data including the execution time of each subgraph on different types of computing devices. The processing module is used to determine a target deployment scheme from multiple deployment schemes determined based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph. The multiple deployment schemes are various ways to deploy multiple subgraphs to different types of computing devices, and the target deployment scheme is the deployment scheme with the shortest computation time among the multiple deployment schemes.

[0019] In one possible implementation, the above processing module is specifically used to: use a dynamic programming algorithm to determine the target deployment scheme from multiple deployment schemes based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph.

[0020] In one possible implementation, the acquisition module is further configured to acquire operator configuration information, which includes at least one target operator used to partition the computation graph; the subgraph partitioning module is specifically configured to partition the computation graph into multiple subgraphs according to the operator configuration information; wherein each of the multiple subgraphs includes at least one node corresponding to a target operator.

[0021] In one possible implementation, the subgraph partitioning module is specifically used to: traverse the computation graph starting from the starting node of the computation graph, and when it is determined that the operator represented by the first node belongs to the target operator, partition the nodes between the first node and the second node, as well as the first node, into a subgraph; wherein the operator corresponding to the second node belongs to the target operator, and the second node is located before the first node in the computation graph.

[0022] In one possible implementation, the operator configuration information also includes an in-degree threshold; the subgraph partitioning module is specifically used to: when it is determined that the operator represented by the first node belongs to the target operator, and the in-degree of the first node is greater than or equal to the in-degree threshold, partition the nodes between the first node and the second node, as well as the first node, into a subgraph.

[0023] In one possible implementation, the computation task is the recommendation task of a recommendation system, and the target operators include the embedding lookup operator and the concat operator.

[0024] Thirdly, this application provides a computing device including a processor and a memory, wherein the processor of each computing device is used to execute instructions stored in the memory to cause a cluster of computing devices to implement the operational steps of the method as described in the first aspect and any possible implementation thereof.

[0025] Fourthly, this application provides a computer-readable storage medium storing computer program instructions that, when executed by a computing device or a cluster of computing devices, implement the operational steps of the method described in the first aspect or any possible implementation thereof.

[0026] Fifthly, this application provides a computer program product comprising a computer program that, when executed by a computing device or a cluster of computing devices, implements the operational steps of the method described in the first aspect or any possible implementation thereof.

[0027] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0028] Figure 1 is a schematic diagram of a calculation graph provided in this application;

[0029] Figure 2 is a flowchart illustrating a heterogeneous computing power partitioning method provided in this application;

[0030] Figure 3 is a schematic diagram of another calculation graph provided in this application;

[0031] Figure 4 is a schematic diagram of a subgraph partitioning process provided in this application;

[0032] Figure 5 is a schematic diagram of the relationship between subgraphs provided in this application;

[0033] Figure 6 is a schematic diagram of a heterogeneous computing power splitting device provided in this application;

[0034] Figure 7 is a schematic diagram of a computing device provided in this application. Detailed Implementation

[0035] Heterogeneous computing refers to computation performed using different types of hardware resources. In heterogeneous computing, various types of hardware resources are integrated together to improve overall computing performance by leveraging the advantages of each type. These different types of hardware resources include central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), neural network processing units (NPUs), and field-programmable gate arrays (FPGAs).

[0036] Computational power partitioning refers to dividing a computational task into multiple smaller subtasks and then allocating them to different types of hardware resources for processing. Computational power partitioning has significant application value in large-scale computing. For example, in the training or inference process of artificial intelligence models, by deploying the operators included in the model to different types of hardware resources for parallel execution, the advantages of different hardware resources can be fully utilized to improve system efficiency. However, current computational power partitioning algorithms cannot effectively evaluate the performance of different operators on different computing hardware resources, resulting in inaccurate computational power partitioning results.

[0037] The following section will first introduce relevant information such as the computation graph and the in-degree of nodes.

[0038] A computation graph consists of nodes and directed edges. Nodes represent operators, indicating how data is processed, such as multiplication, addition, matrix multiplication, and data filling. Directed edges indicate the dependencies between nodes, and their direction indicates the flow of input and output data. It's important to note that a computation graph is essentially a program logic graph constructed in memory.

[0039] As exemplarily shown in Figure 1, which is a schematic diagram of a computation graph provided in this application, the computation graph includes four nodes, namely nodes 1 to 5. Nodes 1 and 2 represent multiplication operators, used to perform multiplication operations on input data; nodes 3 and 4 represent addition operators, used to perform addition operations on input data; and node 5 represents division operations, used to perform division operations on input data. The multiplication operator at node 1 performs multiplication operations on input data a and b to obtain a product f, and then outputs data f to node 3. The multiplication operator at node 2 performs multiplication operations on input data c and d to obtain a product g, and then outputs data g to nodes 3 and 4. Node 3 performs addition operations on input data f and input data ug to obtain data i, and then outputs data i to node 5. Node 4 performs counting operations on input data g and input data e to obtain data h, and then outputs data h to node 5. Node 5 performs division operations on input data i and input data h to obtain data j.

[0040] It should be understood that an operator can have one or more input data. Therefore, in a computational graph, a node can have one or more inputs, meaning that multiple directed edges can point to a single node. For example, in Figure 1, nodes 1 and 2 each have a directed edge pointing to node 3, and nodes 3 and 4 each have a directed edge pointing to node 5. After an operator processes the input data and obtains the result, it can also output the result to one or more operators. Therefore, in a computational graph, multiple directed edges can emanate from a single node pointing to different nodes. For example, in Figure 1, the result from node 2 needs to be output to nodes 3 and 4; node 2 has two directed edges pointing to nodes 3 and 4 respectively.

[0041] The in-degree of a node refers to the number of directed edges pointing to that node in the computational graph. For example, in Figure 1 above, the in-degree of nodes 3 and 5 is 2.

[0042] This application provides a computing power allocation method. The method first divides the computing graph of a computing task into multiple subgraphs. Each computing graph includes multiple nodes, each node representing an operator, and each subgraph includes at least one node. After dividing the computing graph into multiple subgraphs, each subgraph is deployed to different types of computing devices and executed. Performance data of each subgraph executed on different types of computing devices is obtained, such as the execution time of different subgraphs on different computing devices. Finally, based on the type and number of computing devices included in the heterogeneous platform where the computing task will be deployed, and combined with the performance data of each subgraph on different types of computing devices, the subgraphs deployed on each computing device in the heterogeneous platform are determined, i.e., the operators deployed on each computing device in the heterogeneous platform are determined. By executing the partitioned subgraphs on different computing devices to obtain performance data for each subgraph on different computing devices, the performance of different subgraphs on different computing devices can be effectively evaluated. Each subgraph includes at least one node, that is, each subgraph includes at least one operator, so the performance of different operators on different devices can also be evaluated. Finally, based on the performance data of different subgraphs on different devices, the subgraphs to be deployed on each computing device are determined. Deploying each subgraph to each computing device can deploy different operators on more suitable hardware resources, and more accurately and fully utilize the advantages of different hardware devices.

[0043] The heterogeneous computing power partitioning method provided in this application is described below with reference to the accompanying drawings. See Figure 2, which is a flowchart of a heterogeneous computing power partitioning method provided in this application.

[0044] S201. Manage the device to obtain the calculation graph corresponding to the calculation task.

[0045] The aforementioned computational tasks can be high-performance computing tasks, or training or inference tasks of AI models, such as training or inference of recommendation systems, training or inference of natural language models, etc. This application does not specifically limit the aforementioned computational tasks.

[0046] For a computational task, after acquiring the task, the management device can convert it into a computation graph to describe the order of operations, the flow of data, and the dependencies between different operations. For example, in the field of AI, each layer of an AI model is equivalent to performing operations on the input data and model parameters of that layer through operators to obtain the output result of that layer. Therefore, the management device can convert AI models into computation graphs using AI learning frameworks (such as TensorFlow or PyTorch).

[0047] For example, as shown in Figure 3, which is a schematic diagram of another computation graph provided in this application, Figure 3 illustrates the computation graph corresponding to a fully connected layer. The computation of a fully connected layer can be represented as Y = WX + b, where W is the weight matrix of the fully connected layer, X is the input data of the fully connected layer, b is the bias, and Y represents the output result. The output result Y is then processed by an activation function to obtain the final output of the fully connected layer. Therefore, a fully connected layer can be converted into three operators: a matrix multiplication operator, an addition operator, and an activation function operator. This can be converted into the computation graph shown in Figure 3, which includes three nodes. The input of the matrix multiplication operator includes the weights W and the input data X, and the output result is Z. The input of the node corresponding to the addition operator includes the output result Z of the matrix multiplication operator and the bias, and the output result is Y. The input of the node corresponding to the activation function operator is Y.

[0048] S202. The management device divides the calculation diagram into multiple sub-diagrams.

[0049] After obtaining the computation graph corresponding to the computation task, the management device divides the computation graph into multiple subgraphs. Each subgraph includes one or more nodes from the computation graph, meaning each subgraph includes one or more operators. For example, if the computation graph has 100 nodes, it can be divided into 10 subgraphs, each including at least one node. If a subgraph includes multiple nodes, these nodes are contiguous within the computation graph; that is, any node in the same subgraph has at least one directed edge pointing to it, or at least one directed edge pointing from it to another node.

[0050] In one possible implementation, the management device divides the computation graph into multiple subgraphs based on the target operators. After acquiring the computation graph, the management device obtains operator configuration information, which includes one or more target operators. The management device divides the computation graph into multiple subgraphs based on the nodes corresponding to the target operators, with each subgraph including at least one node corresponding to a target operator.

[0051] Optionally, the management device starts from the starting node of the computation graph and traverses the computation graph along the directed edges. When the management device identifies a node representing an operator that is one or more of the target operators mentioned above, for example, the operator represented by the second node in the computation graph belongs to the target operator, the second node and the nodes before the second node are divided into a subgraph. Then, taking the node pointed to by the second node as the new starting node, the management device continues to traverse the computation graph along the directed edges. If the management device identifies another node representing an operator that belongs to the target operator, for example, the operator represented by the first node belongs to the target operator, the first node and the nodes between the first node and the second node (excluding the second node) are divided into a subgraph. This process is repeated for each node in the computation graph until the computation graph is divided into multiple subgraphs, with each node in the computation graph belonging to one of the subgraphs.

[0052] Optionally, the above computational task is a recommendation task of the recommendation model, and the above target operator includes, but is not limited to, any one or more of the embedding lookup operator, concat operator, and matrix multiplication operator. This application does not make any specific limitation.

[0053] For example, if the operator configuration information includes three target operators, namely, an embedding search operator, a connection operator, and a matrix multiplication operator, Figure 4 is a schematic diagram of a subgraph partitioning process provided in this application. The computation graph in Figure 4 includes 13 nodes, with node 1 as the starting node. The management device traverses the computation graph along directed edges starting from node 1. Since node 1 has three directed edges pointing to nodes 2 through 4, the management device traverses along these three directed edges. If the node represented by node 2 is a target operator in the operator configuration information, such as an embedding search operator, the management device identifies that the operator represented by node 2 is a target operator, and then uses node 2 as the partitioning node for dividing the computation graph; node 2 and node 1 are partitioned into a subgraph. If the nodes represented by nodes 3 and 4 are also target operators in the operator configuration information, the management device identifies that the operators represented by nodes 3 and 4 are both target operators, and then partitions node 3 and node 1 into a subgraph, and partitions node 4 and node 1 into a subgraph. Since node 1 and node 2 are already in a subgraph, nodes 3 and 4 can be in a separate subgraph, or nodes 3, 4, 2, and 1 can be grouped into the same subgraph. For example, in recommendation systems, there are usually multiple embedding lookup operators at the front of the computation graph. These embedding lookup operators are computed in parallel and have relatively low computational cost, so they can be deployed on the same device, such as a CPU, and processed in parallel using the CPU's multiple cores.

[0054] After dividing nodes 1 to 4 into a subgraph, the nodes pointed to by nodes 2 to 4 are taken as new starting nodes. In Figure 4, the nodes pointed to by node 2 include nodes 5 and 6. Taking node 5 as the new starting node, the traversal continues along the directed edges. If neither node 8 nor node 11 belongs to the target operator, the traversal continues. If the management device identifies node 10 as belonging to the target operator, then node 10 is taken as the dividing node for partitioning the computation graph. Since the directed edges from nodes 8 and 11 both point to node 10, the management device divides node 10 and the nodes between node 10 and node 2 into a subgraph. Therefore, this subgraph includes nodes 5, 8, 11, and 10. Taking node 6 as the new starting point, node 6 points to node 9. The management device identifies node 9 as belonging to the target operator, so nodes 6 and 9 are divided into a subgraph. Node 3 points to node 7, and node 7 becomes the new starting node. Node 7 points to node 9. Since nodes 9 and 6 have already formed a subgraph, nodes 7, 6, and 9 can also be grouped into a subgraph. Node 4 points to node 5. Since nodes 5, 8, 11, and 10 have already formed a subgraph, the management device continues traversing along the directed edge from node 10. Node 10 points to node 13, and nodes 7 and 9 also point to node 13. Node 11 points to node 12, and node 12 points to node 13. Therefore, the management device groups nodes 12 and 13 into a subgraph. Finally, the 13 nodes are divided into 4 subgraphs.

[0055] In another possible implementation, the management device divides the computation graph into multiple subgraphs based on the in-degree of nodes. After acquiring the computation graph, the management device obtains operator configuration information, including an in-degree threshold. Starting from the initial node of the computation graph, the management device traverses the computation graph along its directed edges. When the in-degree of a node is greater than or equal to the in-degree threshold (e.g., the in-degree of the third node is greater than or equal to the in-degree threshold), the third node is used as the splitting node, and the third node and the nodes preceding it are grouped into a subgraph. Then, using the third node as the new initial node, the traversal continues. If another node has an in-degree greater than or equal to the in-degree threshold (e.g., the in-degree of the fourth node is greater than or equal to the in-degree threshold), the fourth node and the nodes between the third and fourth nodes are grouped into a subgraph.

[0056] In another possible implementation, the management device partitions the computation graph based on one or more target operators and the in-degree threshold of nodes. After acquiring the computation graph, the management device obtains operator configuration information, which includes the in-degree threshold and one or more target operators. Starting from the initial node of the computation graph, the management device traverses the computation graph along the directed edges. If a node represents an operator that is a target operator, and the node's in-degree is greater than or equal to the in-degree threshold, then that node is designated as a splitting node, and the splitting node partitions the computation graph. Optionally, if there are multiple target operators, the in-degree threshold can be set only for some target operators. For example, when the target operators include embedding lookup operators and join operators, the in-degree threshold can be set for join operators. When the management device determines that a node represents a join operator, if the node's in-degree is greater than or equal to the in-degree threshold, then that node is designated as a splitting node; if the node's in-degree is less than the in-degree threshold, then that node is not designated as a splitting node.

[0057] It should be understood that the above is an example of how the management device can partition the computation graph. The management device can also partition the computation graph using other methods, such as setting different target operators according to different application scenarios or different AI models. This application does not limit this.

[0058] S203. Deploy each subgraph to different types of computing devices and execute them, obtaining performance data for each subgraph on different types of computing devices.

[0059] After dividing the computation graph into multiple subgraphs, the management device obtains device configuration information, which includes the type of computing device used to deploy the aforementioned computation graph. Different types of computing devices include at least two of the following: CPU, GPU, TPU, NPU, or FPGA. Other types of computing devices may also be included, but will not be listed here.

[0060] The management device determines the target type of computing devices to be deployed on based on the device configuration information. Then, the management device deploys each subgraph to the target type of computing device for execution, obtaining performance data for each subgraph on different types of computing devices. This performance data includes the execution time of each subgraph on each type of computing device. For example, if the device configuration information indicates that the above stage graph is deployed to heterogeneous devices including CPUs and NPUs, the management device will deploy each subgraph to the CPU and NPU respectively for execution, obtaining performance data for each subgraph on the CPU and NPU. For instance, if the above computing graph is divided into four subgraphs, each subgraph will be deployed to the CPU and NPU respectively for execution, obtaining performance data for each subgraph on each type of computing device.

[0061] Optionally, the above device configuration information also includes the type and data of different types of computing devices used to deploy the above computation graph, as well as the memory size of each computing device. The above performance data also includes the memory occupied by each subgraph during execution.

[0062] S204. The management device determines the target deployment scheme from a variety of deployment schemes determined based on performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between multiple subgraphs.

[0063] It should be understood that after dividing the computation graph into multiple subgraphs, each subgraph can serve as a new node, and multiple subgraphs can form a new computation graph. Each node in the new computation graph represents a subgraph, and directed edges represent the direction of data flow. For example, as shown in Figure 5, which is a schematic diagram of the relationship between subgraphs provided in this application, if the computation graph is divided into 5 subgraphs, namely subgraph 1 to subgraph 5, these 5 subgraphs are numbered 1 to 5 respectively. The dependency relationship between the 5 subgraphs is shown in Figure 5. The input data of subgraphs 2, 3, 4, and 5 all include the output data of subgraph 1. Therefore, subgraphs 2, 3, 4, and 5 are all dependent on subgraph 1. Subgraphs 2, 3, 4, and 5 are all dependent on subgraph 1. The input data of subgraph 4 also includes subgraph 2. Therefore, subgraph 4 and subgraph 2 are dependent. Subgraph 2 is dependent on subgraph 4.

[0064] After determining the performance data for each computation graph, the management device determines a target deployment scheme from a variety of deployment schemes determined based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between multiple subgraphs; wherein, the target deployment scheme indicates the operators included in one or more subgraphs deployed on each computing device.

[0065] After determining the dependencies, when deploying the operators represented by nodes in the computation graph to different types of computing devices, operators are deployed on a subgraph-by-subgraph basis. That is, the operators represented by nodes within each subgraph are deployed to the same computing device. Since different computing devices execute operations sequentially—for example, if subgraph 1 is deployed on the first computing device, and subgraphs 2 and 3 on the second computing device—the input data for the computation task is first input to the first computing device. Only after the first computing device outputs its computation results to the second computing device can the second computing device begin executing the operators included in subgraphs 2 and 3. Therefore, when deploying multiple subgraphs, the operators included in the subgraph that the data source of a subgraph depends on must be executed first. Thus, if a subgraph is deployed on computing device A, the subgraphs it depends on must be deployed on a device that executes computations before device A, or the subgraph and its dependent subgraphs must be deployed on the same device. For example, subgraph 4 depends on subgraphs 1 and 2. Therefore, subgraphs 1 and 2 can be deployed on the same computing device, and subgraph 4 on another computing device, or subgraphs 1, 2, and 4 can all be deployed on the same computing device. If subgraphs 1 and 4 are deployed on the first computing device, and subgraph 2 on the second computing device, the input data needs to be processed by the operators included in subgraph 1. Therefore, the first computing device first executes the calculation of the operators included in subgraph 1, obtains the first calculation result, and outputs it to the second computing device. After the second computing device executes the operators included in subgraph 2 and obtains the second calculation result, it returns the second calculation result to the first computing device, which then executes the operators included in subgraph 4. This scheme involves two data transmissions between the first and second computing devices, which will increase latency.

[0066] To facilitate the description of various deployment schemes determined based on dependencies, this application defines two concepts: prefix and sub-prefix. A prefix includes at least one subgraph from the aforementioned multiple subgraphs, and each prefix includes a first subgraph, which is the subgraph containing the starting node of the computation graph, such as subgraph 1 in Figure 5. When a prefix includes two or more subgraphs, any subgraph included in the prefix other than the first subgraph, such as a second subgraph, is also included in the prefix if it depends on other subgraphs. For example, if subgraph 1 and subgraph 4 in Figure 5 are included in a prefix, then subgraph 2, which subgraph 4 depends on, is also included in the prefix. A prefix corresponds to at least one subprefix. The subgraphs included in each subprefix are proper subsets of the subgraphs included in the prefix. The subgraphs included in the subprefix must also satisfy the requirements of the prefix mentioned above. That is, a subprefix is ​​also a prefix in essence. The subprefix includes fewer subgraphs than the subgraphs included in its corresponding prefix. A prefix can be obtained by transforming a subprefix of a prefix.

[0067] The relationship between prefixes and subprefixes is explained in detail below using the five subgraphs included in Figure 5. Figure 5 includes five subgraphs, namely subgraph 1 to subgraph 5, which are numbered 1 to 5 respectively. The prefixes formed by the five subgraphs in Figure 5 and the corresponding subprefixes of each prefix are shown in Table 1 below. The five nodes in Figure 5 form a total of 12 prefixes.

[0068] Where I[i] represents the i-th prefix, I[i]:{} represents the subgraph included by the i-th prefix, i is the prefix number of the prefix, and {} records the number of the subgraph included by the i-th prefix; sub I[i] represents the subprefix of the i-th prefix; sub I[i]=[] represents the subprefix corresponding to the i-th prefix, and [] records the prefix index of the subprefix corresponding to the i-th prefix, that is, a prefix can be a subprefix of another prefix. For example, I[3]:{1,3} means that the third prefix includes subgraphs numbered 1 and 3, that is, the third prefix includes subgraph 1 and subgraph 3. I[3]:{1,3} corresponds to sub I[3]=[1], which means that the subprefix corresponding to the third prefix is ​​the first prefix, that is, I[1] is a subprefix of I[3]. For another example, I[9]:{1,2,3,4} means that the ninth prefix includes subgraphs numbered 1, 2, 3 and 4, that is, the ninth prefix includes subgraph 1, subgraph 2, subgraph 3 and subgraph 4. I[9]:{1,2,3,4} corresponds to sub I[9] = [1, 2, 3, 5, 6], indicating that the subprefixes corresponding to the 9th prefix include the 1st, 2nd, 3rd, 5th, and 6th prefixes, that is, I[1], I[2], I[3], I[5], and I[6] are subprefixes of I[9]. Optionally, I[0] can also be defined, where I[0] is an empty set and I[0] is a subprefix of any prefix.

[0069] Table 1

[0070] In this application, after obtaining the aforementioned multiple subgraphs, when deploying these multiple subgraphs to multiple computing devices, these multiple subgraphs constitute a prefix, which corresponds to multiple subprefixes. Subgraphs included in any subprefix can be deployed to the same computing device, and subgraphs other than those included in the subprefix can be deployed to other devices. Combining the number of different types of computing devices used to deploy the computing graphs, a variety of different deployment schemes can be obtained. For example, taking the five subgraphs shown in Figure 5 above as an example, if there are two computing devices, the prefix is ​​I

[0012] :{1, 2, 3, 4, 5}, and the subprefixes include I[1], I[2], I[3], I[4], I[5], I[6], I[7], I[8], I[9], I

[0010] , I

[0011] , then the deployment schemes include, but are not limited to:

[0071] (1) If subgraph 1, which is included in subprefix I[1], is deployed to a computing device, then subgraphs 2 to 5 are deployed to another device;

[0072] (2) If the subgraphs 1 and 2 included in the subprefix I[2] are deployed to one computing device, then the subgraphs 3 to 5 are deployed to another device;

[0073] (3) If subgraph 1 and subgraph 3, which are included in subprefix I[3], are deployed to one computing device, then subgraph 2, subgraph 4 and subgraph 5 are deployed to another device;

[0074] (4) If subgraph 1 and subgraph 5, which are included in subprefix I[4], are deployed to one computing device, then subgraph 2, subgraph 3 and subgraph 4 are deployed to another device;

[0075] (5) If the subgraphs 1, 2 and 3 included in the subprefix I[5] are deployed to one computing device, then the subgraphs 4 and 5 are deployed to another device;

[0076] (6) If the subgraphs 1, 2 and 4 included in the subprefix I[6] are deployed to one computing device, then the subgraphs 3 and 5 are deployed to another device.

[0077] Based on the above description of the 6 deployment schemes, there are 12 possible deployment schemes when deploying the 5 subgraphs shown in Figure 5 to 2 computing devices, which will not be listed here.

[0078] Since different operators are deployed on different types of computing devices, the execution time of the same operator varies when different types of computing devices execute it. Therefore, after determining multiple deployment schemes, for a given deployment scheme, the management device can determine the subgraph deployed on each computing device in that deployment scheme. Then, by combining the performance data of each subgraph on different types of computing devices, the management device can determine the execution time corresponding to that deployment scheme. This execution time refers to the time it takes for multiple different types of computing devices to execute the computing task when the operators included in the computing task are deployed according to that deployment scheme.

[0079] In one possible implementation, the management device can also use a dynamic programming algorithm to determine a target deployment scheme based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the various deployment schemes mentioned above. The management device needs to determine the prefixes that the multiple subgraphs can form and the subprefixes corresponding to each prefix. See Figure 5 and Table 1 above for details.

[0080] In this application, when using the dynamic programming algorithm to determine the target deployment scheme, the state transition equation system dp is defined as shown in Equation 1:

[0081] Where dp[I][m][n] represents the execution time of prefix I when it is deployed and executed on m heterogeneous accelerators and n CPUs, where m is an integer less than M, n is an integer less than N, M is the number of heterogeneous accelerators used to deploy the above computation graph, and N is the number of CPUs used to deploy the above computation graph; I represents the prefix, I′ represents any subprefix of prefix I, and num represents the number of prefixes I that the computation graph ultimately forms. If the relationship after the computation graph is divided into subgraphs is as shown in Figure 5, then the number of prefixes is 1. 2; S represents the set of subgraphs in prefix I excluding subprefix I′, that is, the subgraphs included in prefix I excluding the subprefix I′; cpu(S) represents the execution time of the operator in set S when executed on the CPU; acc(S) represents the execution time of the operator in set S when executed on a heterogeneous accelerator; optionally, since the data is usually input from the CPU side, acc(S) can also include the time for data to be transferred from the CPU to the heterogeneous accelerator, the execution time of the operator in set S when executed on the heterogeneous accelerator, and the time for data to be transferred from the heterogeneous accelerator to the CPU.

[0082] The aforementioned cpu(S) and acc(S) can both be obtained from the performance data of the subgraph on different types of computing devices. For example, the execution time of subgraph 2 on the CPU is T1, and the execution time of subgraph 3 on the CPU is T2. If the set S includes subgraph 2 and subgraph 3, then the value of cpu(S) is T1+T2.

[0083] In one possible implementation, if the performance data mentioned above includes the memory required by each subgraph during execution, and the device configuration information mentioned above includes the memory size of the computing device used to deploy the computation graph, then the state transition equation set dp can also be defined as shown in Equation 2:

[0084] Where mem(S) represents the memory required by the operators in set S during execution. This represents the memory of the nth CPU. This represents the memory of the m-th heterogeneous accelerator.

[0085] Initialize the state transition equation system. For m,n = (0,0) or I = 0, dp[I][m][n] = 0. Define the backlink array back_link[I]. Then, the above formula 1 can be expressed in pseudocode as follows:

[0086] The following uses a specific example to illustrate the above pseudocode. If the relationship between multiple subgraphs obtained by partitioning the computational graph is as shown in Figure 5, and if n = 1, m = 1, i = 9, it means that the prefix is I[9], and the prefix I[9] includes 4 subgraphs, namely Subgraph 1 to Subgraph 4. If at this time d = 3 is traversed, then I[d] includes Subgraph 1 and Subgraph 3, and S includes Subgraph 2 and Subgraph 4. At this time, acc(S) represents the execution duration of deploying Subgraph 2 and Subgraph 4 on the accelerator. If acc(S) < dp[9][1][1], then update the value of dp[9][1][1] to the larger value between dp[3][0][1] and acc(S), where dp[3][0][1] represents the execution duration of deploying Subgraph 1 and Subgraph 3 on the CPU. Finally, record the device used in this case and the deployment situation of the deployment prefix and sub-prefix. For example, at this time, it is recorded as back_link[9].device = acc, back_link[9].prev = 3, indicating that when deploying the operators included in Subgraph 1 to Subgraph 4 with 1 accelerator and 1 CPU, deploy the operators included in Subgraph 1 and Subgraph 3 included in I[3] to the CPU for execution, and deploy Subgraph 2 and Subgraph 4 to the heterogeneous accelerator for execution, which is the best solution.

[0087] Execute the process described in the above pseudocode. If there are 12 prefixes as shown in Table 1 and 1 heterogeneous accelerator and 1 CPU are used to deploy the operators included in the multiple subgraphs shown in Figure 5 above, then the execution duration represented by the final result dp

[0012] [1][1] obtained after traversal is the optimal execution duration among all solutions. With the help of the backtracking array back_link[I], obtain the device information of each node, that is, which subgraphs are deployed on each device. The backtracking logic is as follows:

[0088] If back_link

[0012] .device = acc, back_link

[0012] .device = 6; back_link[6].device = cpu, back_link[6].device = 0; Since I

[0012] includes 5 subgraphs from Subgraph 1 to Subgraph 5, I[6] includes Subgraph 1, Subgraph 2, and Subgraph 4, and I

[0012] - I[5] = {3, 5}, that is, deploying Subgraph 3 and Subgraph 5 on the heterogeneous accelerator and deploying Subgraph 1, Subgraph 2, and Subgraph 4 on the CPU is the optimal deployment solution, that is, the above target deployment solution. It should be understood that the above description of deploying subgraphs on a heterogeneous accelerator or CPU actually means deploying the operators included in the subgraphs on a heterogeneous accelerator or CPU.

[0089] For the sake of simplicity, the above method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application. Other reasonable combinations of steps that those skilled in the art can conceive of based on the above description also fall within the scope of protection of this application. Again, those skilled in the art should be familiar with the fact that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0090] The heterogeneous computing power partitioning method provided in this application has been described in detail above with reference to Figures 1 to 5. The heterogeneous computing power partitioning device, system and computing equipment provided in this application will be described below with reference to Figures 6 and 7.

[0091] Referring to Figure 6, which is a schematic diagram of a heterogeneous computing power partitioning device provided in this application, the heterogeneous computing power partitioning device includes an acquisition module 610, a subgraph partitioning module 620, a testing module 630, and a processing module 640. The acquisition module 610 is used to acquire the computation graph corresponding to the computation task; the computation graph includes multiple nodes, and each of the multiple nodes represents an operator.

[0092] The subgraph partitioning module 620 is used to divide the computation graph into multiple subgraphs, each of which includes at least one node. The testing module 630 is used to deploy each subgraph to different types of computing devices and execute it, obtaining performance data for each subgraph on different types of computing devices, including the execution time of each subgraph on different types of computing devices. The processing module 640 is used to determine a target deployment scheme from multiple deployment schemes determined based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph. These multiple deployment schemes are various ways to deploy multiple subgraphs to different types of computing devices, and the target deployment scheme is the one with the shortest computation time among these multiple deployment schemes.

[0093] In one possible implementation, the processing module 640 uses a dynamic programming algorithm to determine the target deployment scheme from multiple deployment schemes based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph.

[0094] In one possible implementation, the acquisition module 610 is further configured to acquire operator configuration information, which includes at least one target operator used to partition the computation graph; the subgraph partitioning module 620 partitions the computation graph into multiple subgraphs according to the operator configuration information; wherein each of the multiple subgraphs includes at least one node corresponding to a target operator.

[0095] In one possible implementation, the subgraph partitioning module 620 traverses the computation graph starting from the starting node. If it is determined that the operator represented by the first node belongs to the target operator, the nodes between the first node and the second node, as well as the first node, are partitioned into a subgraph. The operator corresponding to the second node belongs to the target operator, and the second node is located before the first node in the computation graph.

[0096] In one possible implementation, the operator configuration information mentioned above also includes an in-degree threshold; when the subgraph partitioning module 620 determines that the operator represented by the first node belongs to the target operator and the in-degree of the first node is greater than or equal to the in-degree threshold, it partitions the nodes between the first node and the second node, as well as the first node, into a subgraph.

[0097] Specifically, the method for implementing computing power allocation using the computing power allocation device can refer to the method for implementing computing power allocation using the management device in the method embodiments corresponding to Figures 1-5 above, and will not be repeated here.

[0098] The acquisition module 610, subgraph partitioning module 620, testing module 630, and processing module 640 described above can all be implemented in software or in hardware. For example, the implementation of processing module 640 will be described below; the implementation methods of other modules can refer to the implementation method of processing module 640.

[0099] Processing module 640, as an example of a software functional unit, includes code running on a computing instance. The computing instance includes at least one of a physical host, a virtual machine, and a container. Further, the aforementioned computing instance can be one or more. For example, processing module 640 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically proximate data centers. A region may include multiple AZs.

[0100] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0101] As an example of a hardware functional unit, the processing module 640 can be implemented using a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD can be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a neural network processing unit (NPU), a system-on-chip (SoC), an offload card, an accelerator card, or any combination thereof.

[0102] This application also provides a heterogeneous computing power partitioning system, which can be used to implement the heterogeneous computing power partitioning method provided in this application. The heterogeneous computing power partitioning system includes a client and a heterogeneous computing power partitioning device.

[0103] The communication connection between the client and the heterogeneous computing power partitioning device can be one or more clients, and this application does not make any specific limitations.

[0104] Heterogeneous computing power partitioning devices can be deployed on a single computing device or in a cluster of computing devices including multiple computing devices. The computing devices can be servers, virtual machines, or containers. Servers can be servers in cloud data centers, edge servers, or local servers in enterprise on-premises data centers; this application does not impose specific limitations. A virtual machine refers to a complete computer system simulated by software, possessing full hardware system functionality and running in a completely isolated environment. Any task that can be performed on a physical computer can also be performed in a virtual machine. When creating a virtual machine on a computing device, a portion of the physical machine's hard drive and memory capacity needs to be used as the virtual machine's hard drive and memory capacity. Each virtual machine has an independent basic input / output system, hard drive, and operating system, and can be operated like a physical machine. Containers are a lightweight virtualization technology that allows multiple isolated application instances to run within the same operating system. They can merge an application and all its dependencies into a single software package, which is not limited by the underlying host operating system. This eliminates the need to build complex environments, simplifying the application development and deployment process.

[0105] The client is deployed on the user's terminal device to enable human-computer interaction. In this application, the user can configure computing tasks through the client, such as building a recommendation model or other AI models, and configure the aforementioned operator configuration information and device configuration information through the client. This allows the heterogeneous computing power allocation device to implement the heterogeneous computing power allocation method implemented by the management device based on the computing tasks, operator configuration information, and device configuration information. The client can be software or an application running on the user's terminal device, such as a client for a personal computer (PC), a browser-based client or browser plugin, an application (APP) running on a mobile terminal, or a cloud platform console; this application does not specifically limit this. Terminal devices include personal computers, smartphones, wearable devices, handheld processing devices, tablets, laptops, smart conferencing devices, etc., and are not specifically limited here.

[0106] In one possible implementation, the client is a client of a cloud platform provided by a cloud service provider. The cloud platform is used to provide various cloud services to users, who can purchase or rent cloud services through the client. The heterogeneous computing power partitioning method provided in this application can be one of these cloud services. The aforementioned heterogeneous computing power partitioning device is deployed on computing equipment in a cloud data center to provide users with heterogeneous computing power partitioned cloud services. Users can use the heterogeneous computing power partitioned cloud services through the client.

[0107] In another possible implementation, the heterogeneous computing power partitioning device is deployed on computing equipment in the enterprise's local data center. This device, either provided by the enterprise itself or a third party, is a solution for implementing source code workloads. Clients are deployed on terminal devices used by users within the enterprise, such as desktops and laptops. Users can then access the services provided by the heterogeneous computing power partitioning device through these clients.

[0108] In another possible implementation, the heterogeneous computing power partitioning device and the client can also be deployed on a single computing device, which can be a user-used terminal device. When the heterogeneous computing power partitioning device and the client are deployed on a single computing device, they can also be integrated into a single application.

[0109] It should be understood that the above deployment method is only for illustrative purposes. Clients and heterogeneous computing power splitting devices can be flexibly deployed according to actual business needs, and this application does not impose specific limitations.

[0110] Referring to Figure 7, which is a schematic diagram of a computing device provided in this application, the computing device 700 includes a bus 702, a processor 704, a memory 706, and a communication interface 708. The processor 704, the memory 706, and the communication interface 708 communicate with each other via the bus 702. It should be understood that this application does not limit the number of processors and memories in the computing device 700. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a desktop computer, a laptop computer, or a smartphone, or other terminal device.

[0111] Bus 702 can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one line is used in Figure 7, but this does not imply that there is only one bus or one type of bus. Bus 702 can include pathways for transmitting information between various components of computing device 700 (e.g., memory 706, processor 704, communication interface 708). The Unified Bus can also be called a Lingqu Bus.

[0112] The processor 702 may be a central processing unit (CPU), or may include a CPU and other hardware chips. The aforementioned hardware chips may include one or more types, such as graphics processing unit (GPU), microprocessor (MP), digital signal processor (DSP), ASIC, FPGA, CPLD, NPU, SoC, offload card, accelerator card, etc. The computing device 700 may include one or more of the aforementioned types of hardware chips, or may include multiple types of the aforementioned hardware chips. This application embodiment does not make specific limitations.

[0113] Memory 706 may include volatile memory, such as random access memory (RAM). Memory 706 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 706 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0114] It should be noted that the same type of storage medium can be configured in the same computing device to realize the function of memory 706, or two or more types of storage media can be configured to realize the function of memory 706. This application does not limit this.

[0115] The memory 706 stores executable program code, and the processor 704 executes the executable program code to implement the methods in the method embodiments corresponding to Figures 1-5, thereby realizing the partitioning of heterogeneous computing power. That is, the memory 706 stores instructions for executing the heterogeneous computing power partitioning methods in the method embodiments corresponding to Figures 1-5.

[0116] The communication interface 708 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 700 and other devices or communication networks.

[0117] The computing device 700 provided according to this application can be used to execute the method described in this application, and other operations and / or functions of each unit in the computing device 700 are respectively for implementing the corresponding flow of the method corresponding to Figures 1-5. For the sake of brevity, they will not be described in detail here.

[0118] This application also provides a computing device cluster. The computing device cluster includes at least one computing device as shown in FIG7. The memory 706 of one or more computing devices 700 in the computing device cluster may store the same instructions for executing the methods in the embodiments corresponding to FIG1-FIG5.

[0119] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the heterogeneous computing power partitioning method. In other words, one or more computing devices 700 can jointly execute instructions for implementing the heterogeneous computing power partitioning method.

[0120] It should be noted that the memory 706 in different computing devices 700 within the computing device cluster can store different instructions, which are used to implement some functions of the heterogeneous computing power partitioning device. That is, the instructions stored in the memory 706 of different computing devices 700 can acquire the functions of one or more modules among the acquisition module 610, subgraph partitioning module 620, testing module 630, and processing module 640.

[0121] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN). For example, a heterogeneous computing power partitioning method can be implemented using two computing devices, 700A and 700B, connected via a network. Each computing device connects to the network through its communication interface. In this type of possible implementation, the memory 706 in computing device 700A stores instructions for executing the functions of the acquisition module 610 and the subgraph partitioning module 620. Simultaneously, the memory 706 in computing device 700B stores instructions for executing the functions of the testing module 630 and the processing module 640.

[0122] It should be understood that the functions of computing device 700A can also be performed by multiple computing devices 700. Similarly, the functions of computing device 700B can also be performed by multiple computing devices 700.

[0123] This application also provides a computer program product containing instructions, which may be a software or program product containing instructions capable of running on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, the at least one computing device performs the methods implemented in the embodiments corresponding to Figures 1-5 above.

[0124] This application also provides a computer-readable storage medium, which can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the methods implemented in the embodiments corresponding to Figures 1-5 above.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A method for heterogeneous computing power partitioning, characterized in that, The method includes: Obtain the computation graph corresponding to the computation task, wherein the computation graph includes multiple nodes, and each node in the multiple nodes represents an operator; The computation graph is divided into multiple subgraphs, and each subgraph includes at least one node. Each subgraph is deployed to different types of computing devices and executed, and performance data of each subgraph on different types of computing devices is obtained, including the execution time of each subgraph on different types of computing devices; A target deployment scheme is determined from a variety of deployment schemes based on performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph. The multiple deployment schemes are various ways to deploy the multiple subgraphs to the different types of computing devices. The target deployment scheme is the deployment scheme with the shortest computation time among the multiple deployment schemes.

2. The method according to claim 1, characterized in that, The process of determining the target deployment scheme from multiple deployment schemes determined based on performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph includes: The target deployment scheme is determined from the multiple deployment schemes using a dynamic programming algorithm based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph.

3. The method according to claim 1 or 2, characterized in that, The step of dividing the computation graph into multiple subgraphs includes: Obtain operator configuration information, which includes at least one target operator used to partition the computation graph; The computation graph is divided into multiple subgraphs according to the operator configuration information; wherein each subgraph in the multiple subgraphs includes at least one node corresponding to the target operator.

4. The method according to claim 3, characterized in that, The step of dividing the computation graph into multiple subgraphs according to the operator configuration information includes: Starting from the starting node of the computation graph, traverse the computation graph. If it is determined that the operator represented by the first node belongs to the target operator, divide the nodes between the first node and the second node, as well as the first node, into a subgraph. The operator corresponding to the second node belongs to the target operator, and the second node is located before the first node in the computation graph.

5. The method according to claim 4, characterized in that, The operator configuration information also includes an in-degree threshold; When it is determined that the operator represented by the first node belongs to the target operator, dividing the nodes between the first node and the second node, as well as the first node, into a subgraph includes: If it is determined that the operator represented by the first node belongs to the target operator, and the in-degree of the first node is greater than or equal to the in-degree threshold, the nodes between the first node and the second node, as well as the first node, are divided into a subgraph.

6. A heterogeneous computing power sharing device, characterized in that, include: The acquisition module is used to acquire the computation graph corresponding to the computation task. The computation graph includes multiple nodes, and each node in the multiple nodes represents an operator. A subgraph partitioning module is used to divide the computation graph into multiple subgraphs, each of the multiple subgraphs including at least one node; The testing module is used to deploy each subgraph to different types of computing devices and execute it, and to obtain performance data of each subgraph on different types of computing devices, including the execution time of each subgraph on different types of computing devices; The processing module is used to determine a target deployment scheme from multiple deployment schemes determined based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph. The multiple deployment schemes are multiple ways to deploy the multiple subgraphs to the different types of computing devices, and the target deployment scheme is the deployment scheme with the shortest computation time among the multiple deployment schemes.

7. The apparatus according to claim 6, characterized in that, The processing module is specifically used for: The target deployment scheme is determined from the multiple deployment schemes using a dynamic programming algorithm based on the performance data of each subgraph on different types of computing devices, the number of different types of computing devices, and the dependencies between each subgraph.

8. The apparatus according to claim 6 or 7, characterized in that, The acquisition module is further configured to acquire operator configuration information, the operator configuration information including at least one target operator, the at least one target operator being used to partition the computation graph; The subgraph partitioning module is specifically used to partition the computation graph into multiple subgraphs according to the operator configuration information; wherein each subgraph in the multiple subgraphs includes at least one target operator.

9. The apparatus according to claim 8, characterized in that, The subgraph partitioning module is specifically used for: Starting from the starting node of the computation graph, traverse the computation graph. If it is determined that the operator represented by the first node belongs to the target operator, divide the nodes between the first node and the second node, as well as the first node, into a subgraph. The operator corresponding to the second node belongs to the target operator, and the second node is located before the first node in the computation graph.

10. The apparatus according to claim 9, characterized in that, The operator configuration information also includes an in-degree threshold; The subgraph partitioning module is specifically used to: when it is determined that the operator represented by the first node belongs to the target operator, and the in-degree of the first node is greater than or equal to the in-degree threshold, partition the nodes between the first node and the second node, as well as the first node, into a subgraph.

11. A computing device, characterized in that, The method includes a processor and a memory, the memory storing instructions, and the processor executing the instructions stored in the memory to implement the method as claimed in any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, It includes computer program instructions that, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 5.

Images