A computing graph partitioning method, apparatus and storage medium

Abstract

Embodiments of the present application disclose a computing graph partitioning method, device and storage medium, wherein the computing graph partitioning method comprises the steps of: setting hardware priority and I / O priority for hardware devices in a current heterogeneous environment based on the computing power size and I / O performance of the hardware devices; performing preliminary partitioning on subgraphs of a computing graph based on the hardware priority and the I / O priority of the hardware devices; setting a reference threshold according to the number of operators corresponding to the subgraphs in the computing graph after the preliminary partitioning and the maximum supported operator number of the hardware devices corresponding to the subgraphs; and fusing the subgraphs and adjacent subgraphs in the computing graph after the preliminary partitioning that meet a preset condition based on the reference threshold, so as to reduce data transmission overhead.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer information processing, and in particular to a computing graph partitioning method and device and a storage medium. BACKGROUND

[0002] Artificial intelligence algorithms based on deep learning gradually dominate the mainstream in daily applications, and related products emerge in an endless stream, such as face payment, intelligent voice, and bank customer service robots, which are specific landing applications of AI algorithms. The landing application of AI technology cannot be separated from the deployment method and deployment tool on different platform devices, and the most important one is the inference framework tool. Based on the tool, developers can easily transplant deep learning algorithms to different platform devices, output efficient execution, and then produce artificial intelligence applications, bringing AI technology to user life. However, with the development of deep learning, more complex models often accompany high load and large-scale computation, which challenges the existing inference framework, and combined with the rapid development of domestic NPU chips, therefore, the deep learning heterogeneous inference framework emerges as the times require.

[0003] The heterogeneous inference framework can receive models trained from different deep learning frameworks such as Pytorch and Tensorflow, after receiving, the model is serialized into a computing graph through a model converter (Model Converter), and the computing graph is preliminarily graph-optimized through an accelerator (Graph Transformer), finally, the computing graph is partitioned into multiple sub-graphs by a computing graph partitioner (Graph Partitioner) and distributed to different hardware, achieving the purpose of accelerating model inference computation.

[0004] However, with the development of detection and Transformer models, the models are gradually complex, and are affected by the limitations of NPU memory, I / O resources, etc., and only relying on a single NPU chip for inference reaches a bottleneck. From the perspective of computing graph, this problem is to allocate the computing graph structure that NPU is not suitable for (such as the post-processing structure of the detection model) to other types of chips for calculation, and at the same time, ensuring the minimum cut edge during computing graph partitioning becomes a solution worth exploring. The traditional graph partitioning method is to take the minimum cut edge number as the optimization target, and rarely considers the influence of the heterogeneous environment on the performance of distributed graph computing, and the partitioned graph structure may increase the communication cost between graphs, and the graph structure may not meet the case of hardware heterogeneity, resulting in low computing graph performance and slow inference speed. SUMMARY

[0005] The embodiment of the present application aims to provide a calculation graph division method, device and storage medium, so as to solve the problems of increased communication cost between divided graphs, low performance of divided calculation graph and slow inference speed in the prior art.

[0006] To achieve the above-mentioned purpose, the embodiment of the present application provides a calculation graph division method, comprising the steps of: setting hardware priority and I / O priority for hardware devices in a current heterogeneous environment based on the computing power size and I / O performance of the hardware devices;

[0007] Preliminarily dividing subgraphs of a calculation graph based on the hardware priority and the I / O priority of the hardware devices;

[0008] Setting a reference threshold value according to the number of operators corresponding to the subgraphs in the calculation graph after the preliminary division and the maximum number of supported operators of the hardware devices corresponding to the subgraphs, and fusing the subgraphs and adjacent subgraphs in the calculation graph after the preliminary division that meet a preset condition based on the reference threshold value, so as to reduce data transmission overhead.

[0009] Optionally, the method further comprises:

[0010] Constructing a calculation graph division problem model to evaluate the effect of a calculation graph division scheme,

[0011] The calculation graph division problem model comprises a formula:

[0012]

[0013] Wherein, cost(G,S) is the average cost of the calculation graph division scheme, and the lower the average cost, the better the division effect, G=(V,E), V represents a vertex set of the calculation graph, and E represents a set of all edges in the calculation graph, and let the division scheme S={P1,P2,...,Pn}, each division scheme is composed of n partitions P, and satisfies match_rate is the matching rate of the calculation graph structure and the operation characteristics of the hardware devices, and band is the bandwidth between different partitions of the calculation graph.

[0014] Optionally, the method of preliminarily dividing the subgraphs of the calculation graph based on the hardware priority and the I / O priority of the hardware devices comprises:

[0015] Allocating a calculation density operator to the hardware device with high hardware priority and an I / O limited operator to the hardware device with high I / O priority, the calculation density operator comprising Conv, GEMM and / or Pooling, and the I / O limited operator comprising NMS.

[0016] Optionally, according to the number of operators corresponding to the subgraph in the calculation graph after the preliminary division and the maximum number of supported operators of the hardware device corresponding to the subgraph, the method for setting the reference threshold value comprises:

[0017] by the formula:

[0018]

[0019] to obtain the reference threshold value, wherein Score represents the reference threshold value, N represents the number of subgraphs obtained by the hardware device after the preliminary division, and Sub represents the number of operators in the i-th subgraph of the hardware device.

[0020] Optionally, it further comprises:

[0021] The hardware priority and the I / O priority of the hardware device are preset according to prior knowledge. In order to achieve the above-mentioned purpose, the present application further provides a calculation graph division device, comprising: a memory; and

[0022] a processor connected with the memory, the processor being configured to:

[0023] based on the computing power size and the I / O performance of the hardware device in the current heterogeneous environment, setting a hardware priority and an I / O priority for the hardware device respectively;

[0024] based on the hardware priority and the I / O priority of the hardware device, performing a preliminary division on the subgraph of the calculation graph;

[0025] according to the number of operators corresponding to the subgraph in the calculation graph after the preliminary division and the maximum number of supported operators of the hardware device corresponding to the subgraph, setting a reference threshold value, and based on the reference threshold value, fusing the subgraph that meets a preset condition in the calculation graph after the preliminary division with the adjacent subgraph to reduce the data transmission overhead.

[0026] Optionally, the processor is further configured to:

[0027] constructing a calculation graph division problem model to evaluate the effect of the calculation graph division scheme,

[0028] the calculation graph division problem model comprises the formula:

[0029]

[0030] Wherein, the cost (G, S) is the average cost of the calculation graph division scheme, the lower the average cost is, the better the division effect is, G=(V, E), V represents the vertex set of the calculation graph, E represents the set of all edges in the calculation graph, and it is assumed that the division scheme S={P1, P2,..., Pn} is composed of n partitions P, which satisfies The match_rate is the matching rate of the calculation graph structure and the operation characteristics of the hardware device, and the band is the bandwidth between different partitions of the calculation graph.

[0031] Optionally, the processor is further configured to:

[0032] The method for preliminarily dividing the subgraph of the calculation graph based on the hardware priority and the I / O priority of the hardware device comprises the following steps.

[0033] The calculation density operator is allocated to the hardware device with high hardware priority, and the I / O limited operator is allocated to the hardware device with high I / O priority, the calculation density operator includes Conv, GEMM and / or Pooling, and the I / O limited operator includes NMS.

[0034] Optionally, the processor is further configured to:

[0035] According to the number of operators corresponding to the subgraph in the calculation graph after the preliminary division and the maximum supported operator number of the hardware device corresponding to the subgraph, the method for setting the reference threshold value comprises the following steps.

[0036] The reference threshold value is obtained by the formula:

[0037]

[0038] Wherein, Score represents the reference threshold value, N represents the number of subgraphs obtained by the hardware device after the preliminary division, and Sub represents the number of operators in the i-th subgraph of the hardware device.

[0039] To achieve the above object, the application further provides a computer storage medium, which stores a computer program, wherein the computer program is executed by a machine to realize the steps of the method.

[0040] The embodiments of the application have the following advantages:

[0041] The embodiment of the application provides a computing graph division method, comprising the steps of: setting hardware priority and I / O priority for hardware devices in a current heterogeneous environment based on the computing power size and I / O performance of the hardware devices; performing preliminary division on subgraphs of a computing graph based on the hardware priority and the I / O priority of the hardware devices; setting a reference threshold according to the number of operators corresponding to the subgraphs in the computing graph after the preliminary division and the maximum supported operator number of the hardware devices corresponding to the subgraphs, and fusing the subgraphs meeting a preset condition in the computing graph after the preliminary division and adjacent subgraphs based on the reference threshold, so as to reduce data transmission overhead.

[0042] By the above method, the subgraphs of the computing graph are divided based on the hardware priority and the I / O priority of the hardware devices, so that the computing graph divided by the compiler can adapt to the corresponding underlying heterogeneous hardware computing logic and optimization strategy, and load balancing can be realized in the whole computing graph execution process. When different hardware backends are added in the heterogeneous framework, the efficient division of the computing graph can be completed by calling the strategy in the implementation process, without separate development and debugging. The communication cost between the divided graphs is reduced by fusing the subgraphs meeting the preset condition in the computing graph after the preliminary division and adjacent subgraphs based on the reference threshold, so that the performance and inference speed of the divided computing graph are improved, which is beneficial to the application and popularization of deep learning products and improves the competitiveness. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and those skilled in the art can obtain other implementation drawings according to the provided drawings without creative labor.

[0044] Figure 1 A flowchart of a computing graph division method provided by the embodiment of the application;

[0045] Figure 2 A module block diagram of a computing graph division device provided by the embodiment of the application. DETAILED DESCRIPTION

[0046] The following embodiments of the application are explained by way of specific examples, which are not meant to limit the scope of the application. Other objects, advantages, and novel features of the application will become apparent from the following detailed description when considered in conjunction with the annexed drawings. It is to be understood that the described embodiments are merely a part of the application, and that the application is not limited to the described embodiments.

[0047] Moreover, the technical features involved in the different embodiments of the application described below can be combined with each other as long as there is no conflict.

[0048] An embodiment of the application provides a computing graph partitioning method, which refers to Figure 1 , Figure 1 A flowchart of a computing graph partitioning method provided in an embodiment of the application is shown in FIG. 1. It should be understood that the method can further include additional blocks not shown or can omit the blocks shown, and the scope of the application is not limited in this respect.

[0049] Currently, there are mainly two implementation methods for computing graph partitioning in the deep learning inference stage:

[0050] 1) Static computing graph partitioning, which places as many associated nodes on the same partition as possible in a large graph, so as to convert most of the network communication overhead into local machine memory copy, thereby effectively improving the speed of distributed graph computing.

[0051] 2) Parallel lightweight dynamic computing graph repartitioning, which effectively adapts the decomposition to the graph dynamics by incrementally migrating vertices between partitions, and improves the mapping of application communication patterns to the underlying hardware topology, which is suitable for large-scale dynamic computing graph partitioning with uneven weights and sizes.

[0052] The two implementations have the following disadvantages:

[0053] 1) The influence of hardware platforms in a heterogeneous environment on the execution performance of the partitioned computing graph is not considered, resulting in that the full performance of the heterogeneous resources cannot be fully utilized.

[0054] 2) The fusion optimization problem of the partitioned computing subgraph at Runtime is not considered, and the subgraph operator does not match the hardware optimization strategy.

[0055] Generally, the main factors affecting the computing graph partitioning of a deep learning model include the following four aspects:

[0056] 1) Operator computation density;

[0057] 2) I / O size of a single layer of the computing graph;

[0058] 3) the computational efficiency of the operator itself on hardware;

[0059] 4) the fusion strategy of the subgraph.

[0060] The scheme in the embodiment can, according to the four elements of the above-mentioned computation graph division strategy, through hierarchical division in a heterogeneous environment, make hardware of different architectures obtain computation subgraphs that conform to their own optimization strategies in the process of deep learning inference, and achieve the purposes of computation load balancing and efficient data transmission.

[0061] At step 101, based on the computing power size and I / O performance of the hardware devices in the current heterogeneous environment, hardware priority and I / O priority are respectively set for the hardware devices.

[0062] Specifically, the computation priority is set. First, the hardware priority is set for hardware devices of different types according to the computing power of the hardware, and then the priority is fine-tuned according to the hardware I / O performance to set the I / O priority. In some embodiments, the priority can also be set by prior knowledge, and the matching rate of the computation graph structure and the hardware operation characteristics is increased as much as possible to reduce the computation cost. In the heterogeneous environment, the computing power priority and the I / O priority are configured for the hardware according to the architecture type, which is used as a reference when executing the division strategy, and the priority is saved by using HashMap to map the address of the hardware.

[0063] At step 102, based on the hardware priority and the I / O priority of the hardware devices, the subgraph of the computation graph is preliminarily divided.

[0064] In some embodiments, the method of preliminarily dividing the subgraph of the computation graph based on the hardware priority and the I / O priority of the hardware devices includes:

[0065] allocating a computation density operator to the hardware device with high hardware priority, and allocating an I / O limited operator to the hardware device with high I / O priority, the computation density operator including Conv, GEMM and / or Pooling, and the I / O limited operator including NMS.

[0066] Specifically, the subgraph is preliminarily divided according to the set hardware priority and I / O priority. In some embodiments, high-computing-density operators such as Conv, GEMM, Pooling, etc. are allocated to hardware with high computation priority, and I / O limited operators such as NMS are allocated to hardware with high I / O priority, to achieve the first-level division of the computation graph, and to reserve the intermediate results. In the execution process of the inference framework, the computation graph is preliminarily divided according to the computing power priority and the I / O priority, and the division result is reserved in the memory.

[0067] At step 103, a reference threshold is set according to the number of operators corresponding to the subgraph in the calculation graph after the preliminary division and the maximum number of supported operators of the hardware device corresponding to the subgraph, and the subgraph in the calculation graph after the preliminary division that meets a preset condition is fused with adjacent subgraphs based on the reference threshold to reduce data transmission overhead.

[0068] In some embodiments, the method of setting the reference threshold according to the number of operators corresponding to the subgraph in the calculation graph after the preliminary division and the maximum number of supported operators of the hardware device corresponding to the subgraph includes:

[0069] The reference threshold is obtained by the formula:

[0070]

[0071] wherein Score represents the reference threshold, N represents the number of subgraphs obtained by the hardware after the preliminary division, and Sub represents the number of operators in the i-th subgraph of the hardware.

[0072] Specifically, in a strong computing hardware, according to the primary subgraph division result, as much as possible, the larger scale calculation subgraph is retained, and the subgraph composed of isolated operators or a small number of operators is merged into the subgraph where the original parent node is located, thereby reducing the overhead of data transmission in different architecture hardware. The division method is as follows:

[0073]

[0074] As shown in formula (1), it is a weighted average value of the scale of the hardware subgraph. Wherein N represents the number of subgraphs obtained by the hardware after the preliminary division of the calculation graph, Sub represents the number of operators in the i-th subgraph of the hardware, and the weight value is set to the number of operators in the i-th subgraph divided by the total number of operators supported by the current hardware device. Finally, the weighted score of the hardware device is accumulated. Taking the calculated score as a reference threshold, the calculation subgraph with small scale or composed of isolated operators in the hardware is filtered out and merged into the adjacent subgraph, thereby reducing the data transmission overhead and ensuring that the underlying hardware obtains a relatively complete calculation subgraph, which better applies the graph optimization strategy of the hardware itself. Before the inference framework assigns the calculation graph to the hardware, the weighted average value of the current subgraph is used to perform secondary division on the preliminary division result, to retain a relatively complete calculation subgraph for the hardware, for the underlying optimization of the hardware. According to the above determined execution plan, the calculation graph is re-allocated according to the second layer algorithm score to generate a final high-performance calculation subgraph.

[0075] In some embodiments, it further includes:

[0076] A computation graph partition problem model is constructed to evaluate the effect of a computation graph partition scheme,

[0077] The computation graph partition problem model includes a formula:

[0078]

[0079] wherein cost(G, S) is the average cost of the computation graph partition scheme, the lower the average cost, the better the partition effect, G=(V, E), V represents a vertex set of the computation graph, E represents a set of all edges in the computation graph, and it is assumed that a partition scheme S={P1, P2,..., Pn} is composed of n partitions P, which satisfy match_rate is a matching rate of the computation graph structure and the operation characteristics of the hardware device, and band is a bandwidth between different partitions of the computation graph.

[0080] Specifically, in a computation graph G=(V, E), V represents a vertex set of the computation graph, and E represents a set of all edges in the graph. It is assumed that a partition scheme S={P1, P2,..., Pn} is composed of n partitions P, which satisfy After partitioning, it is ensured that each partition has no repeated nodes. After graph partitioning, the computation speed and I / O performance of the computation graph after partitioning need to be quantified to ensure the reliability and computation efficiency of the partitioned graph, to avoid the problems that the partitioned subgraph does not conform to the hardware computation logic or the I / O communication volume increases and the computation consumption is huge after partitioning. The effect of the partition scheme is evaluated by the following formula:

[0081]

[0082] The average cost of the partition scheme S is determined by the matching rate (match_rate) of the computation graph structure and the operation characteristics of the hardware and the bandwidth (band) between different partitions. The higher the matching rate of the graph structure and the hardware characteristics, the lower the computation cost. If the bandwidth between device i and device j is higher, the partition computation cost is also lower. Therefore, in the partitioning process, the computation cost of the computation graph partition scheme is reduced by improving the graph structure matching rate and reducing the communication overhead as much as possible.

[0083] By the above method, based on the hardware priority and the I / O priority of the hardware device, the subgraphs of the computation graph are divided, so that the computation graph divided by the compiler can adapt to the corresponding underlying heterogeneous hardware computing logic and optimization strategy, and at the same time, load balancing can be realized in the whole computation graph execution process. When different hardware backends are added in the heterogeneous framework, the efficient division of the computation graph can be completed by calling this strategy in the implementation process, without separate development and debugging. The reference threshold is used to fuse the subgraphs that meet the preset conditions and adjacent subgraphs in the computation graph after the preliminary division, thereby reducing the communication cost between the divided graphs, improving the performance and inference speed of the divided computation graph, and being beneficial to the application and popularization of deep learning products and improving the competitiveness.

[0084] The method can perceive hardware priority and I / O priority through a reasonable algorithm in a heterogeneous environment, thereby realizing more reasonable and efficient computation graph division, effectively improving the computing power utilization rate of the chip, reducing the communication overhead of the heterogeneous device, and accelerating the inference speed. This method has the following characteristics:

[0085] 1) Improving the computing power utilization rate, which can maximize the utilization rate of the underlying chip's computing power according to the model operator type while minimizing the data transmission overhead.

[0086] 2) High scalability, which can fully consider the heterogeneity of the underlying computing infrastructure and the I / O transmission scale, incrementally adjust the computation graph division, and support different types of models and different hardware devices.

[0087] Figure 2 A module block diagram of a computation graph division device provided by an embodiment of the present application is provided. The device includes:

[0088] The memory 201; and the processor 202 connected with the memory 201, wherein the processor 202 is configured to set hardware priority and I / O priority for hardware devices in a current heterogeneous environment based on the computing power size and I / O performance of the hardware devices.

[0089] Preliminarily divide subgraphs of a computation graph based on the hardware priority and the I / O priority of the hardware devices.

[0090] Set a reference threshold based on the number of operators corresponding to the subgraphs in the computation graph after the preliminary division and the maximum number of supported operators of the hardware devices corresponding to the subgraphs, and fuse the subgraphs that meet the preset conditions and adjacent subgraphs in the computation graph after the preliminary division based on the reference threshold, so as to reduce the data transmission overhead.

[0091] In some embodiments, the processor 202 is further configured to further include:

[0092] constructing a computation graph partition problem model to evaluate the effect of a computation graph partition scheme,

[0093] The computation graph partition problem model includes a formula:

[0094]

[0095] wherein cost(G, S) is the average cost of the computation graph partition scheme, the lower the average cost, the better the partition effect, G = (V, E), V represents a vertex set of the computation graph, E represents a set of all edges in the computation graph, and let a partition scheme S = {P1, P2,..., Pn} be composed of n partitions P, satisfying match_rate is a matching rate of the computation graph structure and the operation characteristics of the hardware device, and band is a bandwidth between different partitions of the computation graph.

[0096] In some embodiments, the processor 202 is further configured to, based on the hardware priority and the I / O priority of the hardware device, the method of preliminarily partitioning the subgraph of the computation graph includes:

[0097] allocating a computation density operator to the hardware device with high hardware priority, and allocating an I / O limited operator to the hardware device with high I / O priority, the computation density operator including Conv, GEMM and / or Pooling, and the I / O limited operator including NMS.

[0098] In some embodiments, the processor 202 is further configured to, according to the number of operators in the subgraph of the computation graph and the maximum number of supported operators of the hardware device corresponding to the subgraph after the preliminary partition is performed, the method of setting the reference threshold includes:

[0099] obtaining the reference threshold by a formula:

[0100]

[0101] wherein Score represents the reference threshold, N represents the number of subgraphs obtained by the hardware device after the preliminary partition, and Sub represents the number of operators in the i-th subgraph of the hardware device.

[0102] In some embodiments, the processor 202 is further configured to further include:

[0103] presetting the hardware priority and the I / O priority of the hardware device according to prior knowledge.

[0104] The method is implemented as described above with reference to the method embodiments. The method is implemented by means of a computer program product, comprising computer readable instructions, which when executed by a processor, cause the processor to carry out the steps of the method.

[0105] The present application can be a method, an apparatus, a system, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for carrying out aspects of the present application.

[0106] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a magnetically encoded device such as magnetic strip

[0107] The computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0108] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0109] Various aspects of the present application can be described in the general context of methods and apparatuses (systems) and computer program products, respectively. It should be appreciated that the various aspects of the present application can be implemented as one or more computer programs, each of which runs in conjunction with an operating system and / or application program and / or other computer programs. The computer program can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer program can be fixed in at least one computer readable medium, which can be any data storage device or computer readable media which can be non-transitory. The computer readable medium can include, but is not limited to, semiconductor memory devices, solid state memory, magnetic tape, self-clocking signal, packet, optical medium, hard wired circuitry, floppy diskettes, compact diskettes, holographic data storage, RAM, ROM, EEPROM, flash memory cards, DVD, BLU-RAY, and the like. The computer readable medium can also include, but is not limited to, carrier waves of signals modulated as described herein.

[0110] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can be non-transitory. The computer readable storage medium can include a computer data signal, computer readable memory, computer readable storage, or any combination thereof. The computer readable storage medium can also be used to store data that is manipulated by the computer readable program instructions.

[0111] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0112] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0113] Note that all of the disclosed features described above can be replaced by alternative features serving the same, equivalent or similar purposes, unless explicitly stated otherwise. Thus, no individual feature is a prerequisite. Where used, further, more further, even more further and even more preferably are simple precursors for yet another embodiment to be set out below based on the preceding embodiment, the combination of the preceding embodiment and what is stated after further, more further, even more further or even more preferably forms a further embodiment. Any combination of several further, more further, even more further or even more preferably settings after the same embodiment forms yet another embodiment.

[0114] While the present application has been described in detail with respect to particular embodiments thereof, it will be apparent to those skilled in the art that various modifications or changes in form and details can be made therein without departing from the spirit and scope of the present application. Therefore, the scope of the present application should not be limited to the embodiments described herein but should be defined by the appended claims and their equivalents.

Claims

1. A computational graph partitioning method, characterized in that, Includes the following steps: Based on the computing power and I / O performance of the hardware devices in the current heterogeneous environment, set hardware priorities and I / O priorities for the hardware devices respectively; Based on the hardware priority and I / O priority of the hardware device, the subgraph of the computation graph is initially divided; Based on the number of operators corresponding to the subgraphs in the computation graph after the initial partitioning and the maximum number of operators supported by the hardware device corresponding to the subgraph, a reference threshold is set. Subgraphs in the computation graph that meet preset conditions after the initial partitioning are merged with adjacent subgraphs based on the reference threshold to reduce data transmission overhead. The method for initially partitioning the subgraphs of the computation graph based on the hardware priority and I / O priority of the hardware device includes: The computation density operator is assigned to the hardware device with the highest hardware priority, and the I / O-restricted operator is assigned to the hardware device with the highest I / O priority. The computation density operator includes Conv, GEMM and / or Pooling, and the I / O-restricted operator includes NMS.

2. The computational graph partitioning method according to claim 1, characterized in that, Also includes: A computational graph partitioning problem model is constructed to evaluate the effectiveness of different partitioning schemes. The computational graph partitioning problem model includes the following formulas: Where cost(G,S) is the average cost of the computation graph partitioning scheme. The lower the average cost, the better the partitioning effect. G=(V,E), where V represents the set of vertices in the computation graph, and E represents the set of all edges in the computation graph. Let the partitioning scheme S={P1,P2,...,Pn}, and each partitioning scheme consists of n partitions P, satisfying... , match_rate is the matching rate between the computation graph structure and the operational characteristics of the hardware device, and band is the bandwidth between different partitions of the computation graph.

3. The computational graph partitioning method according to claim 1, characterized in that, The method for setting the reference threshold based on the number of operators corresponding to the subgraphs in the computational graph after the initial partitioning and the maximum number of operators supported by the hardware device corresponding to the subgraph includes: Through the formula: The reference threshold is obtained, where Score represents the reference threshold, N represents the number of subgraphs obtained by the hardware device after the initial partitioning, and Sub represents the number of operators in the i-th subgraph of the current hardware device.

4. The computational graph partitioning method according to claim 1, characterized in that, Also includes: The hardware priority and I / O priority of the hardware device are preset based on prior knowledge.

5. A computational graph partitioning device, characterized in that, include: Memory; as well as A processor connected to the memory, the processor being configured to: Based on the computing power and I / O performance of the hardware devices in the current heterogeneous environment, set hardware priorities and I / O priorities for the hardware devices respectively; Based on the hardware priority and I / O priority of the hardware device, the subgraph of the computation graph is initially divided; Based on the number of operators corresponding to the subgraphs in the computation graph after the initial partitioning and the maximum number of operators supported by the hardware device corresponding to the subgraph, a reference threshold is set. Subgraphs in the computation graph that meet preset conditions after the initial partitioning are merged with adjacent subgraphs based on the reference threshold to reduce data transmission overhead. The method for initially partitioning the subgraphs of the computation graph based on the hardware priority and I / O priority of the hardware device includes: The computation density operator is assigned to the hardware device with the highest hardware priority, and the I / O-restricted operator is assigned to the hardware device with the highest I / O priority. The computation density operator includes Conv, GEMM and / or Pooling, and the I / O-restricted operator includes NMS.

6. The computational graph partitioning device according to claim 5, characterized in that, The processor is also configured to: A computational graph partitioning problem model is constructed to evaluate the effectiveness of different partitioning schemes. The computational graph partitioning problem model includes the following formulas: Where cost(G,S) is the average cost of the computation graph partitioning scheme. The lower the average cost, the better the partitioning effect. G=(V,E), where V represents the set of vertices in the computation graph, and E represents the set of all edges in the computation graph. Let the partitioning scheme S={P1,P2,...,Pn}, and each partitioning scheme consists of n partitions P, satisfying... , match_rate is the matching rate between the computation graph structure and the operational characteristics of the hardware device, and band is the bandwidth between different partitions of the computation graph.

7. The computational graph partitioning device according to claim 6, characterized in that, The processor is also configured to: The method for setting the reference threshold based on the number of operators corresponding to the subgraphs in the computational graph after the initial partitioning and the maximum number of operators supported by the hardware device corresponding to the subgraph includes: Through the formula: The reference threshold is obtained, where Score represents the reference threshold, N represents the number of subgraphs obtained by the hardware device after the initial partitioning, and Sub represents the number of operators in the i-th subgraph of the current hardware device.

8. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a machine, it implements the steps of the method as described in any one of claims 1 to 5.

Images