Method and device for acquiring optimal parallel computing strategy of deep network
By constructing the overall dependency tree of the computing graph and the target dependency tree of the deep learning model, the minimum summary and redistribution cost is calculated to determine the parallel computing strategy, the calculation cost problem caused by the parallelization of tensor operations in deep networks is solved, and efficiency improvement is achieved.
Patent Information
- Application Number
- CN202310602286.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-25
- Publication Date
- 2024-12-03
AI Technical Summary
In deep network models, the parallelization method of tensor operations is not arbitrary, and different parallelization strategies will lead to the summary and re-split of operation results, increasing the cost of calculation, how to determine the optimal parallelization method from a global perspective to minimize Summary/re-split cost between operators.
By constructing the overall dependency tree of the computing graph of the deep learning model, the target dependency tree composed of the target root node and its subtree nodes is obtained, the minimum summary and redistribution cost of the target dependency tree is calculated, and the corresponding tensor splitting strategy is determined as Parallel computing strategy.
The summary/re-split cost between operators during the calculation process is effectively minimized and the calculation efficiency is improved.
Smart Images

Figure CN119067188A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and device for obtaining an optimal parallel computing strategy for a deep network. Background Art
[0002] Deep network models involve a large number of tensor operations. When multiple computing units are available, splitting the tensor operations into individual computing units for parallel computing can greatly speed up the calculation of the entire network and shorten the calculation time.
[0003] In the prior art, the parallelization method for tensor operations in the deep network model is not arbitrary, but is determined by the computational logic of the operation itself; and different parallelization methods require that the input of the tensor operation be specifically split and distributed to each computing unit, and the calculation result of each computing unit is a local result of the split of the overall operation result; if the parallelization strategy of the subsequent tensor operation has the same input splitting requirement as the result splitting method of the previous operation, the result of the previous operation can be reused directly, otherwise, the result of the previous operation needs to be aggregated, and then the aggregated result is split and distributed according to the parallelization strategy of the subsequent operation; the process of aggregating / re-splitting the operation results requires the introduction of additional memory transfer, which brings additional computational cost; each operation in the deep network model may have multiple parallel strategies, and different aggregation / re-splitting costs will be generated between operations according to the selection of their respective parallel strategies.
[0004] In summary, how to determine the optimal parallelization method for each operator in the deep network model from a global perspective, so as to minimize the aggregation / re-splitting cost between operators during the calculation process is a problem that needs to be solved at present. Summary of the invention
[0005] In view of this, an embodiment of the present invention provides a method and apparatus for obtaining an optimal parallel computing strategy for a deep network, which can minimize the aggregation / re-splitting costs between operators during the operation process.
[0006] In a first aspect, an embodiment of the present invention provides a method for obtaining an optimal parallel computing strategy for a deep network, the method comprising:
[0007] Constructing an overall dependency tree according to the deep learning model calculation graph, wherein any tree node in the overall dependency tree corresponds to at least one operation node of the deep learning model calculation graph;
[0008] Obtain any tree node in the overall dependency tree as a target root node, wherein the target root node and its subtree nodes constitute a target dependency tree;
[0009] Calculate the minimum aggregated redistribution cost of the target dependency tree, wherein the aggregated redistribution cost represents the aggregated redistribution cost of the tensor splitting strategy of at least one computing node corresponding to the target root node;
[0010] The tensor splitting strategy corresponding to the minimum summary redistribution cost is determined as the parallel computing strategy of at least one computing node corresponding to the target root node.
[0011] Optionally, the calculating the minimum aggregate redistribution cost of the target dependency tree specifically includes:
[0012] The sum of the internal aggregated redistribution costs of the target root node and the accumulated value is determined as the minimum aggregated redistribution cost of the target dependency tree, wherein the accumulated value is the accumulated value of the minimum value of the sum of the aggregated redistribution costs between the target root node and any of its subtree nodes in the target dependency tree and the minimum sum of the aggregated redistribution costs of the dependency tree when any of its subtree nodes in the target dependency tree is the root.
[0013] Optionally, the sum of the internal aggregated redistribution costs of the target root node specifically includes:
[0014] When the tensor splitting strategy of at least one computing node corresponding to the target root node is the first tensor splitting strategy, the sum of the internal costs of at least one computing node corresponding to the target root node, wherein the first tensor splitting strategy is the optimal tensor splitting strategy of at least one computing node corresponding to the target root node.
[0015] Optionally, the redistribution cost between the target root node and any of its subtree nodes in the target dependency tree is summarized, specifically including:
[0016] Determine the first tensor splitting strategy of at least one computing node corresponding to the target root node in the target dependency tree, and the second tensor splitting strategy combination of at least one computing node corresponding to any subtree node in the target dependency tree, wherein the second tensor splitting strategy is the optimal tensor splitting strategy of at least one computing node corresponding to any subtree node;
[0017] When it is determined that the target root node is the first tensor splitting strategy and any of its subtree nodes is the second tensor splitting strategy, the aggregate redistribution cost between any of its subtree nodes and the target root node in the target dependency tree.
[0018] Optionally, the method further includes:
[0019] Determine a first tensor splitting strategy for at least one computing node corresponding to the target root node.
[0020] Optionally, the determining of a first tensor splitting strategy of at least one computing node corresponding to the target root node specifically includes:
[0021] Calculating multiple cost upper bounds of multiple candidate tensor splitting strategies for at least one computing node corresponding to the target root node;
[0022] Determining a minimum upper cost bound value among the plurality of upper cost bounds;
[0023] The candidate tensor splitting strategy corresponding to the minimum cost upper bound value is determined as the first tensor splitting strategy.
[0024] Optionally, the upper bound of the cost is the sum of a ratio and the number of bytes of the input or output tensor, wherein the ratio is the ratio of the number of bytes of the input or output tensor to the total number of computing nodes used by the distribution method corresponding to the candidate tensor splitting strategy.
[0025] In a second aspect, an embodiment of the present invention provides a device for obtaining an optimal parallel computing strategy for a deep network, the device comprising:
[0026] A construction unit, configured to construct an overall dependency tree according to a deep learning model calculation graph, wherein any tree node in the overall dependency tree corresponds to at least one operation node of the deep learning model calculation graph;
[0027] An acquisition unit, used for acquiring any tree node in the overall dependency tree as a target root node, wherein the target root node and its subtree nodes constitute a target dependency tree;
[0028] A computing unit, configured to compute a minimum aggregated redistribution cost of the target dependency tree, wherein the aggregated redistribution cost represents an aggregated redistribution cost of a tensor splitting strategy of at least one computing node corresponding to the target root node;
[0029] A determination unit is used to determine the tensor splitting strategy corresponding to the minimum summary redistribution cost as the parallel computing strategy of at least one computing node corresponding to the target root node.
[0030] Optionally, the calculation unit is specifically used to: determine the sum of the internal aggregated redistribution costs of the target root node and the sum of the accumulated values as the minimum aggregated redistribution cost of the target dependency tree, wherein the accumulated value is the accumulated value of the minimum value of the sum of the aggregated redistribution costs between the target root node and any of its subtree nodes in the target dependency tree and the minimum sum of the minimum aggregated redistribution costs of the dependency tree when any of its subtree nodes in the target dependency tree is the root.
[0031] Optionally, the internal summary redistribution cost sum of the target root node specifically includes: when the tensor splitting strategy of at least one computing node corresponding to the target root node is the first tensor splitting strategy, the internal cost sum of at least one computing node corresponding to the target root node, wherein the first tensor splitting strategy is the optimal tensor splitting strategy of at least one computing node corresponding to the target root node.
[0032] Optionally, the redistribution cost between the target root node and any of its subtree nodes in the target dependency tree is summarized, specifically including:
[0033] Determine the first tensor splitting strategy of at least one computing node corresponding to the target root node in the target dependency tree, and the second tensor splitting strategy combination of at least one computing node corresponding to any subtree node in the target dependency tree, wherein the second tensor splitting strategy is the optimal tensor splitting strategy of at least one computing node corresponding to any subtree node;
[0034] When it is determined that the target root node is the first tensor splitting strategy and any of its subtree nodes is the second tensor splitting strategy, the aggregate redistribution cost between any of its subtree nodes and the target root node in the target dependency tree.
[0035] Optionally, the determining unit is further configured to:
[0036] Determine a first tensor splitting strategy for at least one computing node corresponding to the target root node.
[0037] Optionally, the determining unit is specifically configured to:
[0038] Calculating multiple cost upper bounds of multiple candidate tensor splitting strategies for at least one computing node corresponding to the target root node;
[0039] Determining a minimum upper cost bound value among the plurality of upper cost bounds;
[0040] The candidate tensor splitting strategy corresponding to the minimum cost upper bound value is determined as the first tensor splitting strategy.
[0041] Optionally, the upper bound of the cost is the sum of a ratio and the number of bytes of the input or output tensor, wherein the ratio is the ratio of the number of bytes of the input or output tensor to the total number of computing nodes used by the distribution method corresponding to the candidate tensor splitting strategy.
[0042] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the method as described in the first aspect and any possible embodiment of the first aspect.
[0043] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement a method as described in the first aspect and any possible embodiment of the first aspect.
[0044] In a fifth aspect, an embodiment of the present invention provides a chip comprising a memory and a processing core, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processing core to implement a method as described in the first aspect or any possible embodiment of the first aspect.
[0045] In a sixth aspect, an embodiment of the present invention provides a board card, which includes the chip according to the fifth aspect.
[0046] In a seventh aspect, an embodiment of the present invention provides a server, wherein the server comprises the board card according to the sixth aspect.
[0047] The embodiment of the present invention constructs an overall dependency tree according to a deep learning model calculation graph, wherein any tree node in the overall dependency tree corresponds to at least one computing node of the deep learning model calculation graph; obtains any tree node in the overall dependency tree as a target root node, wherein the target root node and its subtree nodes constitute a target dependency tree; calculates the minimum aggregate redistribution cost of the target dependency tree, wherein the aggregate redistribution cost represents the aggregate redistribution cost of the tensor splitting strategy of at least one computing node corresponding to the target root node; and determines the tensor splitting strategy corresponding to the minimum aggregate redistribution cost as the parallel computing strategy of at least one computing node corresponding to the target root node. Through the above method, the aggregate / re-splitting cost between operators during the operation process can be minimized. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0049] Figure 1 is a flow chart of a method for obtaining an optimal parallel computing strategy for a deep network in an embodiment of the present invention;
[0050] Figure 2 It is a schematic diagram of a deep learning model calculation graph in an embodiment of the present invention;
[0051] Figure 3 is a schematic diagram of the structure of an overall dependency tree in an embodiment of the present invention;
[0052] Figure 4 is a schematic diagram of the structure of a target dependency tree in an embodiment of the present invention;
[0053] Figure 5 is a flow chart of another method for obtaining an optimal parallel computing strategy for a deep network in an embodiment of the present invention;
[0054] Figure 6 It is another schematic diagram of a deep learning model calculation graph in an embodiment of the present invention;
[0055] Figure 7 It is a schematic diagram of a device for obtaining an optimal parallel computing strategy for a deep network in an embodiment of the present invention. DETAILED DESCRIPTION
[0056] The present invention is described below based on the embodiments, but the present invention is not limited to these embodiments. In the detailed description of the present invention below, some specific details are described in detail. It is possible for a person skilled in the art to fully understand the present invention without the description of these details. In order to avoid confusing the essence of the present invention, known methods, processes, procedures, components and circuits are not described in detail.
[0057] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.
[0058] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".
[0059] In the description of the present invention, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more.
[0060] In the prior art, the parallelization method of tensor operations in the deep network model is not arbitrary, but is determined by the computational logic of the operation itself; for example, when calculating the activation function ReLU of a given input tensor X on a device with 8 computing units, the input tensor X can be split into 8 sub-tensors, and each computing unit only calculates one of the sub-tensors, which can shorten the calculation time of the input tensor X to one eighth; assuming that the matrix multiplication operation: matmul(A, B), there are two feasible splitting methods: method 1, splitting matrix A into several sub-matrices by row, matrix B is not split, and each computing unit calculates the matrix multiplication of a sub-matrix of matrix A and matrix B; method 2, splitting matrix B into several sub-matrices by column, matrix A is not split, and each computing unit calculates the matrix multiplication of matrix A and a sub-matrix of matrix B. Different parallelization methods require that the input of tensor operations be specifically split and distributed to various computing units, and the calculation result of each computing unit is a local result of the split of the overall operation result; if the parallelization strategy of subsequent tensor operations requires the splitting of the input to be exactly the same as the result splitting method of the previous operation, the result of the previous operation can be reused directly, otherwise, the results of the previous operation need to be summarized, and then the summarized results are split and distributed according to the parallelization strategy of subsequent operations; the process of summarizing / re-splitting the operation results requires the introduction of additional memory movement, which brings additional computational costs; each operation in the deep network model may have multiple parallel strategies, and different aggregation / re-splitting costs will be generated between operations according to the selection of their respective parallel strategies.
[0061] Therefore, how to determine the optimal parallelization method for each operator in the deep network model from a global perspective, so as to minimize the aggregation / re-splitting cost between operators during the operation process is a problem that needs to be solved at present.
[0062] In the embodiment of the present invention, the operator may also be referred to as a computing node or a calculation node, which is not limited in the embodiment of the present invention.
[0063] In order to solve the above problems, a method for obtaining the optimal parallel computing strategy for a deep network is proposed in the embodiment of the present invention. Figure 1 As shown, Figure 1 This is a flow chart of a method for obtaining an optimal parallel computing strategy for a deep network according to an embodiment of the present invention. Specifically, it includes:
[0064] Step S100: construct an overall dependency tree according to the deep learning model calculation graph, wherein any tree node in the overall dependency tree corresponds to at least one operation node of the deep learning model calculation graph.
[0065] Specifically, the deep learning model calculation diagram is as follows Figure 2 As shown, Figure 2 The deep learning model calculation graph includes 14 operation nodes. The deep learning model calculation graph is a directed acyclic graph. Each operation node depends on its input nodes. Since the output of an operation node may be the input of multiple other nodes, a operation node may be dependent on multiple other nodes. This multi-dependency relationship makes the search process difficult to split, and then construct an overall dependency tree. The structure of the overall dependency tree is as follows: Figure 3 As shown, Figure 3 The tree includes 6 tree nodes, each of which corresponds to a point set in the deep learning model calculation graph, and the point set is at least one operation node corresponding to each tree node. Figure 3 The fill pattern of each tree node in Figure 2 The filling pattern of at least one operation node corresponding to it is the same.
[0066] Step S101, obtaining any tree node in the overall dependency tree as a target root node, and the target root node and its subtree nodes constitute a target dependency tree.
[0067] For example, Figure 4 As shown, Figure 3 A target dependency tree with a tree node as the root, Figure 4 The fill pattern of the tree nodes in Figure 3 The fill pattern of the tree nodes is consistent.
[0068] Step S102: Calculate the minimum aggregated redistribution cost of the target dependency tree, wherein the aggregated redistribution cost represents the aggregated redistribution cost of the tensor splitting strategy of at least one computing node corresponding to the target root node.
[0069] Specifically, the sum of the internal aggregated redistribution costs of the target root node and the sum of the accumulated values are determined as the minimum aggregated redistribution cost of the target dependency tree, wherein the accumulated value is the accumulated value of the minimum value of the sum of the aggregated redistribution costs between the target root node and any of its subtree nodes in the target dependency tree and the minimum sum of the minimum aggregated redistribution costs of the dependency tree when any of its subtree nodes in the target dependency tree is the root.
[0070] In one possible implementation, the internal summary redistribution cost sum of the target root node specifically includes: when the tensor splitting strategy of at least one computing node corresponding to the target root node is the first tensor splitting strategy, the internal cost sum of at least one computing node corresponding to the target root node, wherein the first tensor splitting strategy is the optimal tensor splitting strategy of at least one computing node corresponding to the target root node.
[0071] In one possible implementation, the aggregated redistribution cost between the target root node and any of its subtree nodes in the target dependency tree specifically includes: determining the first tensor splitting strategy of at least one operation node corresponding to the target root node in the target dependency tree, and the second tensor splitting strategy combination of at least one operation node corresponding to any of its subtree nodes in the target dependency tree, wherein the second tensor splitting strategy is the optimal tensor splitting strategy for at least one operation node corresponding to any of its subtree nodes; when determining that the target root node is the first tensor splitting strategy and any of its subtree nodes is the second tensor splitting strategy, the aggregated redistribution cost between any of its subtree nodes in the target dependency tree and the target root node.
[0072] For example, the target dependency tree with T as the root, the tensor splitting strategy of the operation node corresponding to the root node T is the minimum summary redistribution cost of the target dependency tree S, which is recorded as DP[T, S]; when calculating the DP[T, S], the entire target dependency tree is divided into n+1 parts, each of the n subtree nodes of T is a part, and the operation node corresponding to the root node T is a part. Due to the tree structure, the edges between the parts only exist between the root node T and its subtree nodes. Therefore, the calculation formula of the DP[T, S] can be expressed as:
[0073]
[0074] Wherein, intra(T,S) is the sum of the internal aggregated redistribution costs of the target root node, that is, the sum of the internal costs when the target root node T adopts the first tensor splitting strategy S; wherein, inter(T,C,S,S ′ ) represents the aggregated redistribution cost between the target root node and any of its subtree nodes in the target dependency tree, wherein C is a point set of any subtree node of the root node T, the point set being a set of at least one computation node, and S ′
[0075] is the second tensor splitting strategy for the subtree node C; the DP[C,S′] is the minimum aggregate redistribution cost of the dependency tree when any subtree node in the target dependency tree is the root.
[0076] Step S103: Determine the tensor splitting strategy corresponding to the minimum summary redistribution cost as the parallel computing strategy of at least one computing node corresponding to the target root node.
[0077] Specifically, the tensor splitting strategy S corresponding to the minimum summary redistribution cost is determined as the parallel computing strategy of at least one operation node corresponding to the target root node T, and the product relationship of the combination can be converted into an addition relationship, which greatly reduces the solution space.
[0078] In the embodiment of the present invention, before step S102, other steps are also included, specifically as follows: Figure 5 As shown, Figure 5 This is a flow chart of a method for obtaining an optimal parallel computing strategy for a deep network according to an embodiment of the present invention. Specifically, it includes:
[0079] Step S104: Determine a first tensor splitting strategy for at least one computing node corresponding to the target root node.
[0080] Specifically, multiple cost upper bounds of multiple candidate tensor splitting strategies for at least one operation node corresponding to the target root node are calculated; the minimum cost upper bound value among the multiple cost upper bounds is determined; the candidate tensor splitting strategy corresponding to the minimum cost upper bound value is determined as the first tensor splitting strategy; wherein the cost upper bound is the sum of a ratio and the number of bytes of the input or output tensor, wherein the ratio is the ratio of the number of bytes of the input or output tensor to the total number of operation nodes used by the distribution method corresponding to the candidate tensor splitting strategy.
[0081] For example, the combination of parallel computing strategies of all operation nodes is a huge solution space. If there are n operators with two parallel computing strategies in the deep network model, the entire deep network model network has 2^n parallel computing strategy combinations. For some budget nodes that calculate element by element, each element can be calculated independently, and any split parallel computing strategy is feasible, which further increases the solution space of the search. In order to obtain a global parallel computing strategy within an acceptable time, the search needs to be pruned; taking a basic pruning strategy as an example, for a subgraph G composed of any operation nodes, such as Figure 6As shown in the operation node of filling mode 1, it includes 4 operation nodes, which have a feasible parallel computing strategy S, determine the summary redistribution cost Cost(S) of the parallel computing strategy S, and all external input tensors required by the subgraph G and all output tensors provided to the outside can be known according to the deep learning model calculation graph. According to the input and the output, the cost upper bound U(s) when the parallel computing strategy S is adopted can be calculated; for the external tensor X, its summary redistribution upper bound is Bytes(X) / CU(DX)+Bytes(X); wherein, Bytes(X) / CU(DX) is the summary cost, and the Bytes(X) is the redistribution cost. If tensor X is used as output, in the worst case, it needs to be aggregated by the DX distribution method in the subgraph and then broadcast to all computing units. If tensor X is used as input tensor, the aggregation and redistribution process is opposite, but the calculation formula of the aggregation and redistribution cost is the same. If the aggregation and redistribution cost of another feasible parallel computing strategy S' of subgraph G is Cost(S'), and Cost(S')>U(s), it means that the parallel computing strategy S is still stronger than the best case of the parallel computing strategy S' in the worst case, then S' can be discarded and no longer participate in subsequent searches, that is, the optimal tensor splitting strategy of the subgraph G is determined.
[0082] In the embodiment of the present invention, the aggregate redistribution cost Cost(S) is cost(D X1 , D X2 ), which means that the tensor X is transformed from D X1 The distribution method is summarized into the overall tensor, and then according to D X2 The cost of transferring data to each computing unit is distributed in this way, that is, cost (D X1 ) is the aggregate cost of the tensor X, cost(D X2 ) is the redistribution cost of tensor X. Calculate the cost (D X1 , D X2 ), it is necessary to clarify the split representation of tensors, the representation of parallel computing strategies of operators, the unification of parallel computing strategies, and the search for parallel computing strategies, as follows:
[0083] 1. Split Representation of Tensors
[0084] The tensor X can be represented by an n-dimensional array Dx, wherein the size of each dimension of the Dx represents the number of splits of the corresponding dimension of the tensor X, and each element in the Dx represents the target computing unit number to which the sub-tensor after the tensor X is split is distributed; for example, the shape of the tensor X is 16x8x7, and the Dx=[[[4, 2, 1, 7]], [[0, 5, 3, 6]]], with a shape of 2 rows×4 columns, means that the tensor X is cut into 2 parts along the 0-axis, and each part is further cut into 4 parts along the 1-axis to form 8 sub-tensors of size 8x2x7, and then distributed to computing units numbered 4, 2, 1, 7, 0, 5, 3 and 6 in row-first order; it can be seen that the elements of the Dx are natural numbers less than the total number of computing units, and there are no repeated elements in the Dx.
[0085] In a possible implementation, if Dx is an empty array, that is, Dx=[], the tensor X is not split, and the tensor X is broadcast and distributed to all computing units.
[0086] 2. Parallel Computing Strategy Representation of Operators
[0087] When expressing the parallel computing strategy of an operator, it is necessary to express the splitting method of each input tensor and output tensor under the parallel computing strategy. For example, when calculating C = matmul (A, B), with 4 computing units, there are two feasible parallel computing strategies:
[0088] aStrategy.D A =[0, 1, 2, 3], D B =[],D C = [0, 1, 2, 3];
[0089] bStrategy.D A =[],D B =[[0, 1, 2, 3]], D C = [[0, 1, 2, 3]];
[0090] The number of the computational units is not important. For example, the a strategy is similar to D A =[1,3,2,0], D B =[],D C =[1,3,2,0], there is no essential difference, but there will be differences when actually distributing to specific computing units. As long as D A With D C The correctness of the operation can be guaranteed if there is a specific relationship between the computing unit numbers in the output. That is, we only need to know which part of the input sub-tensor is used to calculate a certain sub-tensor after the split. Therefore, the parallel computing strategy can be expressed as follows:
[0091] Form 1, D A =[0,1,2,3](D C ), D B =[](D C );
[0092] Form 2, D A =[](D C ), D B =[[0, 1, 2, 3]](D C );
[0093] Among them, the D C is empty, which can be represented as D C =[? ,? ,? ,? ](), each tensor split is represented by D Y =F(D X ), F is an n-dimensional array, the size of each dimension represents the number of splits of the corresponding dimension of Y, and the elements represent the sub-tensors of Y after splitting and D X The corresponding relationship of the sub-tensors; when the D X When it is empty, D Y =F(), then it means that D Y is a free split variable that can take any tensor splitting method with the same shape as F.
[0094] For example, in the first form, D A =[0,1,2,3](D C ), which means that the tensor A is divided into 4 parts along the 0-axis. The 0th part and the 0th part of the calculation result C are in the same calculation unit, the 1st part and the 1st part of the calculation result C are in the same calculation unit, and so on. If the calculation result C is a free split with a shape of 4, when Dc = [3, 1, 2, 0], then D A =[0,1,2,3]([3,1,2,0])=[3,1,2,0].
[0095] In a possible implementation, if Y = transpose(X), the possible parallel computing strategy is as follows: Y =[[0,2],[1,3]](D X ), D X =[[? , ? ], [? , ? ]](), where D X If the upper left, upper right, lower left, and lower right are respectively distributed to computing units 0, 1, 2, and 3, then each computing unit will transpose the corresponding submatrix, and the upper left, upper right, lower left, and lower right of the entire matrix after transposition will be distributed to computing units 0, 2, 1, and 3 respectively. Even if the distribution numbers of the various parts of the input tensor X are different, according to D Y=[[0,2],[1,3]](D X ) can also calculate the correct correspondence.
[0096] 3. Unification of Parallel Computing Strategies
[0097] When there are two consecutive operators Y=op1(X), Z=op2(Y), where Y is the output tensor of the preceding operator op1 and the input tensor of the succeeding operator op2, the parallel computing strategies of op1 and op2 both have a splitting method of Y, which is as follows:
[0098] In op1, D Y =F1(D1); in op2, D2=F2(D Y ), determine D by changing Y2 =F2 inverse (D2); if a split of Y is found, it means D Y = F(?) so that it is consistent with D in op1 Y = F1(D1) and D in op2 Y = F2 inverse (D2) are the same, then the unified D Y =F(?) replaces the tensor split representation of Y in the op1 and the op2, thereby obtaining a splitting method that satisfies the parallelization requirements of both op1 and op2.
[0099] In the embodiment of the present invention, in order to find the D Y The unified form of , assuming that the splitting modes of Y in op1 and op2 are D Y1 =F1(D1),D Y2 =F2(D2); if D1 is empty, D1 is free to split. As long as the shape of F1 remains unchanged, D Y1 The split value can be adjusted by changing the calculation unit of D1. As long as F2 and F1 have the same shape, they can be combined into D Y =F2(D2), otherwise the unification fails; when D2 is empty, it is the same as when D1 is empty, and no further description is given here; if D1 is not empty, D Y1 Convert to the form of F1'(D1'), where D1' is a free split in op1. Since D1 is not a free split, D1 is in the form of F'(D1'-). Y1 =F1(D1)=F1(F'(D1'))=(F1·F')(D1')=F1'(D1'), the above transformation method can be repeated until D1' is free to split; similarly, D Y2 Converted to F2'(D2') form, where D2' is a free split in op2, and the subsequent processing is the same as that of converting D Y1The subsequent process of converting to the form of F1'(D1') is the same and will not be described here; if D1 is not empty and D1' is the same as D2', then F1'=F2' is required, otherwise the unification fails; if D1' is different from D2' and D2' uses no less than D1' computing units, then D1'=G(D2'), where G is a composition parameter, so that D Y1 =F1'(D1')=F1'(G(D2')=F2'(D2')=D Y2 , that is, for any index i, G[F1'[i]] = F2[i]. Similarly, when D2' uses fewer computing units than D1', we can construct D2' = G(D1'), so that D Y2 =F2'(D2')=F2'(G(D1')=F1'(D1')=D Y1 So as to satisfy D Y =D Y1 =D Y2 .
[0100] 4. Parallel Computing Strategy Search
[0101] For all operators in the deep learning model, all feasible parallelization strategies for the operator can be obtained based on the operator operation logic and the total number of computing units, and various combinations of different parallelization strategies for each operator can be enumerated; for specific combinations, try to unify the split representations of the tensors between the previous and next operators; if the unification is successful, it means that the tensor split can reuse the calculation results on each computing unit for subsequent calculations; if the unification fails, it means that the calculation results cannot be reused and the results need to be summarized and distributed.
[0102] Define the aggregate redistribution cost as cost(D X1 , D X2 ), which means that the tensor X is transformed from D X1 The distribution method is summarized into the overall tensor, and then according to D X2 The cost of moving data to each computing unit in a distributed manner; if each computing unit aggregates and redistributes data in parallel, and the cost is proportional to the amount of movement, then cost (D X1 , D X2 )=Bytes(X) / CU(D X1 )+Bytes(X) / CU(D X2 ), where Bytes(X) represents the number of bytes occupied by tensor X, and CU(D) represents the total number of computing units used by the D distribution method. The global parallel strategy selection can be achieved by searching for the parallel strategy combination of all operators and finding the one with the lowest aggregation / redistribution cost of all intermediate tensors.
[0103] Figure 7Schematic diagram of a device for obtaining an optimal parallel computing strategy for a deep network according to an embodiment of the present invention. Figure 7 As shown, the device of this embodiment includes a construction unit 701, an acquisition unit 702, a calculation unit 703 and a determination unit 704.
[0104] Among them, the construction unit 701 is used to construct an overall dependency tree according to the deep learning model calculation graph, wherein any tree node in the overall dependency tree corresponds to at least one operation node of the deep learning model calculation graph; the acquisition unit 702 is used to acquire any tree node in the overall dependency tree as a target root node, and the target root node and its subtree nodes constitute a target dependency tree; the calculation unit 703 is used to calculate the minimum aggregate redistribution cost of the target dependency tree, wherein the aggregate redistribution cost represents the aggregate redistribution cost of the tensor splitting strategy of at least one operation node corresponding to the target root node; the determination unit 704 is used to determine the tensor splitting strategy corresponding to the minimum aggregate redistribution cost as the parallel computing strategy of at least one operation node corresponding to the target root node.
[0105] Furthermore, the calculation unit 703 is specifically used to: determine the sum of the internal aggregated redistribution costs of the target root node and the sum of the accumulated values as the minimum aggregated redistribution cost of the target dependency tree, wherein the accumulated value is the accumulated value of the minimum value of the sum of the aggregated redistribution costs between the target root node and any of its subtree nodes in the target dependency tree and the minimum sum of the minimum aggregated redistribution costs of the dependency tree when any of its subtree nodes in the target dependency tree is the root.
[0106] Furthermore, the internal summary redistribution cost sum of the target root node specifically includes: when the tensor splitting strategy of at least one computing node corresponding to the target root node is the first tensor splitting strategy, the internal cost sum of at least one computing node corresponding to the target root node, wherein the first tensor splitting strategy is the optimal tensor splitting strategy of at least one computing node corresponding to the target root node.
[0107] Furthermore, the redistribution cost between the target root node and any of its subtree nodes in the target dependency tree is summarized, specifically including:
[0108] Determine the first tensor splitting strategy of at least one computing node corresponding to the target root node in the target dependency tree, and the second tensor splitting strategy combination of at least one computing node corresponding to any subtree node in the target dependency tree, wherein the second tensor splitting strategy is the optimal tensor splitting strategy of at least one computing node corresponding to any subtree node;
[0109] When it is determined that the target root node is the first tensor splitting strategy and any of its subtree nodes is the second tensor splitting strategy, the aggregate redistribution cost between any of its subtree nodes and the target root node in the target dependency tree.
[0110] Furthermore, the determining unit is further configured to:
[0111] Determine a first tensor splitting strategy for at least one computing node corresponding to the target root node.
[0112] Furthermore, the determining unit is specifically configured to:
[0113] Calculating multiple cost upper bounds of multiple candidate tensor splitting strategies for at least one computing node corresponding to the target root node;
[0114] Determining a minimum upper cost bound value among the plurality of upper cost bounds;
[0115] The candidate tensor splitting strategy corresponding to the minimum cost upper bound value is determined as the first tensor splitting strategy.
[0116] Furthermore, the upper bound of the cost is the sum of the ratio and the number of bytes of the input or output tensor, wherein the ratio is the ratio of the number of bytes of the input or output tensor to the total number of computing nodes used by the distribution method corresponding to the candidate tensor splitting strategy.
[0117] In an embodiment of the present invention, a computer program instruction is further provided. When the computer program instruction is executed by a processor, the method described in any one of the above embodiments is implemented.
[0118] In an embodiment of the present invention, a computer-readable storage medium is further provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in any one of the above embodiments is implemented.
[0119] An embodiment of the present invention provides a chip, including a memory and a processing core, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processing core to implement any of the methods described in the above embodiments.
[0120] An embodiment of the present invention provides a board, which includes the chip.
[0121] An embodiment of the present invention provides a server, and the server includes the board.
[0122] As will be appreciated by those skilled in the art, various aspects of the embodiments of the present invention may be implemented as a system, method or computer program product. Therefore, various aspects of the embodiments of the present invention may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or an implementation that combines software aspects with hardware aspects, which may generally be referred to herein as a "circuit," "module," or "system." In addition, various aspects of the embodiments of the present invention may take the form of a computer program product implemented in one or more computer-readable media having a computer-readable program code implemented thereon.
[0123] Any combination of one or more computer-readable media can be utilized. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, (but not limited to) electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or apparatuses, or any suitable combination of the foregoing. More specific examples (non-exhaustive enumeration) of computer-readable storage media will include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of an embodiment of the present invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by an instruction execution system, device or apparatus or a program used in conjunction with an instruction execution system, device or apparatus.
[0124] A computer readable signal medium may include a propagated data signal having a computer readable program code implemented therein, such as in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any of the following computer readable media: not a computer readable storage medium, and may communicate, propagate, or transmit a program used by or in conjunction with an instruction execution system, device, or apparatus.
[0125] Program code embodied on a computer readable medium may be transmitted using any appropriate medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0126] The computer program code for performing operations for various aspects of the embodiments of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, etc., and conventional process programming languages such as "C" programming language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer as a stand-alone software package; partially on the user's computer and partially on a remote computer; or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., by using the Internet of an Internet service provider).
[0127] The flowchart legend and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present invention described above describe various aspects of the embodiment of the present invention.It will be understood that each block of the flowchart legend and / or block diagram and the combination of the blocks in the flowchart legend and / or block diagram can be implemented by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that the instructions (executed by the processor of the computer or other programmable data processing device) create a device for implementing the function / action specified in the flowchart and / or block diagram block or block.
[0128] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing device, or other apparatus to operate in a particular manner, so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.
[0129] The computer program instructions may also be loaded onto a computer, other programmable data processing device or other apparatus so that a series of operable steps are performed on the computer, other programmable device or other apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions / actions specified in the flowchart and / or block diagram block or blocks.
[0130] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for obtaining an optimal parallel computing strategy for a deep network, characterized in that: The method includes: Constructing an overall dependency tree according to the deep learning model calculation graph, wherein any tree node in the overall dependency tree corresponds to at least one operation node of the deep learning model calculation graph; Obtain any tree node in the overall dependency tree as a target root node, wherein the target root node and its subtree nodes constitute a target dependency tree; Calculate the minimum aggregated redistribution cost of the target dependency tree, wherein the aggregated redistribution cost represents the aggregated redistribution cost of the tensor splitting strategy of at least one computing node corresponding to the target root node; The tensor splitting strategy corresponding to the minimum summary redistribution cost is determined as the parallel computing strategy of at least one computing node corresponding to the target root node.
2. The method according to claim 1, characterized in that The calculating of the minimum aggregated redistribution cost of the target dependency tree specifically includes: The sum of the internal aggregated redistribution costs of the target root node and the accumulated value is determined as the minimum aggregated redistribution cost of the target dependency tree, wherein the accumulated value is the accumulated value of the minimum value of the sum of the aggregated redistribution costs between the target root node and any of its subtree nodes in the target dependency tree and the minimum sum of the aggregated redistribution costs of the dependency tree when any of its subtree nodes in the target dependency tree is the root.
3. The method according to claim 2, characterized in that The sum of the internal aggregated redistribution costs of the target root node specifically includes: When the tensor splitting strategy of at least one computing node corresponding to the target root node is the first tensor splitting strategy, the sum of the internal costs of at least one computing node corresponding to the target root node, wherein the first tensor splitting strategy is the optimal tensor splitting strategy of at least one computing node corresponding to the target root node.
4. The method according to claim 3, characterized in that The aggregate redistribution cost between the target root node and any of its subtree nodes in the target dependency tree specifically includes: Determine the first tensor splitting strategy of at least one computing node corresponding to the target root node in the target dependency tree, and the second tensor splitting strategy combination of at least one computing node corresponding to any subtree node in the target dependency tree, wherein the second tensor splitting strategy is the optimal tensor splitting strategy of at least one computing node corresponding to any subtree node; When it is determined that the target root node is the first tensor splitting strategy and any of its subtree nodes is the second tensor splitting strategy, the aggregate redistribution cost between any of its subtree nodes and the target root node in the target dependency tree.
5. The method according to claim 3 or 4, characterized in that The method further includes: Determine a first tensor splitting strategy for at least one computing node corresponding to the target root node.
6. The method according to claim 5, characterized in that The determining of a first tensor splitting strategy of at least one computing node corresponding to the target root node specifically includes: Calculating multiple cost upper bounds of multiple candidate tensor splitting strategies for at least one computing node corresponding to the target root node; Determining a minimum upper cost bound value among the plurality of upper cost bounds; The candidate tensor splitting strategy corresponding to the minimum cost upper bound value is determined as the first tensor splitting strategy.
7. The method according to claim 6, characterized in that The upper bound of the cost is the sum of the ratio and the number of bytes of the input or output tensor, wherein the ratio is the ratio of the number of bytes of the input or output tensor to the total number of computing nodes used by the distribution method corresponding to the candidate tensor splitting strategy.
8. A device for obtaining an optimal parallel computing strategy for a deep network, characterized in that: The device includes: A construction unit, configured to construct an overall dependency tree according to a deep learning model calculation graph, wherein any tree node in the overall dependency tree corresponds to at least one operation node of the deep learning model calculation graph; An acquisition unit, used for acquiring any tree node in the overall dependency tree as a target root node, wherein the target root node and its subtree nodes constitute a target dependency tree; A computing unit, configured to compute a minimum aggregated redistribution cost of the target dependency tree, wherein the aggregated redistribution cost represents an aggregated redistribution cost of a tensor splitting strategy of at least one computing node corresponding to the target root node; A determination unit is used to determine the tensor splitting strategy corresponding to the minimum summary redistribution cost as the parallel computing strategy of at least one computing node corresponding to the target root node.
9. A computer program instruction, characterized in that: The computer program instructions implement the method according to any one of claims 1 to 7 when executed by a processor.
10. A computer-readable storage medium storing computer program instructions, characterized in that: The computer program instructions implement the method according to any one of claims 1 to 7 when executed by a processor.
