Method, device and equipment for multivariate operator optimization, graph optimization and graph computing

Abstract

The present invention provides a multivariate operator optimization method, a graph optimization method, a graph computing method, an apparatus and a device. The method comprises: splitting the multivariate operator to be optimized into a plurality of sub-operators with a preset number of input elements, so that the total number of input elements of the multiple sub-operators obtained by splitting is not less than the number of input elements of the multivariate operator when performing layered calculations, and the number of sub-operators obtained by splitting is the least; and taking the multiple sub-operators obtained after splitting as the optimization result of the multivariate operator. The embodiment of the present invention optimizes the multivariate operator into a plurality of sub-operators with a fixed number of input elements. On the one hand, the multiple sub-operators obtained after optimization have a smaller number of sub-operators, a smaller number of slices of the input tensor, a reduced number of I / Os, and a higher efficiency in moving data from the DDR, thereby improving the computing speed of the device.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a multivariate operator optimization method, a graph optimization method, a graph computing method, a device and equipment. Background Art

[0002] Convolutional Neural Networks (CNN) are a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning.

[0003] The convolutional neural network engine is a computing engine that performs forward calculations on convolutional neural networks. In the forward calculation process, layered calculations are its characteristic. Data is input from the input layer, and operations are performed layer by layer according to the calculation graph until the output layer. Among them, a graph is an abstract data structure that represents the relationship between objects. Graph calculations are performed according to the calculation graph, that is, using the graph as a data model to express and solve problems.

[0004] In the process of forward calculation according to the calculation graph, each operator is a calculation node in the calculation graph. From a broad perspective, an operator is an operation performed on any function. For example, the Concat operator implements the operation of splicing multiple operators. Even exponentiation and square root can be considered as an operator. Different operators can implement operations from one input element to multiple input elements. The N-element operator implements the operation of performing a certain operation on N input operators (elements or sets) to form another operator (element or set).

[0005] In a typical graph computing method, on the device side, for multivariate operators, the storage space on the RAM is first dynamically calculated based on the input tensor length, the number of tensors, and the number of cycles to move data from DDR to RAM. Its implementation structure is complex, consumes many registers, and has high performance requirements for the device. It is obviously not friendly to performance-intensive devices. In addition, any number of input elements will cause each input element to be finely divided into blocks, greatly increasing the number of input and output (I / O), making the overall computing efficiency low.

[0006] Therefore, in order to improve the computing speed of the device, make full use of the registers on the device side, and reduce the number of I / Os, it is very necessary to optimize the multivariate operators. How to optimize the multivariate operators to achieve an overall acceleration of the processing speed of the inference engine has become a hot research and development issue in this field. Summary of the invention

[0007] In view of the above problems, the present invention is proposed to provide a multivariate operator optimization method, graph optimization method, graph computing method, device and equipment that overcome the above problems or at least partially solve the above problems.

[0008] In a first aspect, an embodiment of the present invention provides an optimization method for a multivariate operator, comprising:

[0009] Splitting the multivariate operator to be optimized into a plurality of sub-operators with a preset number of input elements, so that when the plurality of sub-operators obtained by splitting are subjected to hierarchical calculation, the total number of input elements is not less than the number of input elements of the multivariate operator, and the number of sub-operators obtained by splitting is the least;

[0010] The multiple sub-operators obtained after the splitting are used as the optimization results of the multivariate operator.

[0011] In one embodiment, the multivariate operator to be optimized is split into a plurality of sub-operators with a preset number of input vertices, including:

[0012] According to the input element number of the multivariate operator to be optimized and the input element number of the sub-operators in the first sub-operator set, a sub-operator is selected from the first sub-operator set according to a preset greedy strategy; the first sub-operator set includes at least two preset sub-operators with different input element numbers;

[0013] A second sub-operator set is generated based on the selected sub-operators, so that when the sub-operators in the second sub-operator set are used for hierarchical calculation, the total number of input elements is not less than the number of input elements of the multivariate operator, and the number of sub-operators obtained by splitting is the least.

[0014] In one embodiment, according to the greedy strategy, selecting a sub-operator from the first sub-operator set and generating a second sub-operator set according to the selected sub-operator includes:

[0015] From the first sub-operator set, select sub-operators in order of size, starting from the one with the largest input element number;

[0016] For each selected current sub-operator, determine whether the remaining input element number of the multi-element operator is greater than zero, except for the input element number provided by all the selected sub-operators for the hierarchical calculation;

[0017] If it is determined that the remaining input element number is greater than zero, the current sub-operator is placed in the second sub-operator set; and the current sub-operator is selected again, and the steps of calculating the remaining input element number of the multivariate operator and determining again whether it is greater than zero are repeated until the remaining input element number of the multivariate operator is less than or equal to zero;

[0018] If it is determined that the remaining input element number of the multivariate operator is less than or equal to zero, the current sub-operator is abandoned, and a sub-operator with an input element number smaller than the current sub-operator is selected from the first sub-operator set, and the steps of calculating the remaining input element number of the multivariate operator and determining again whether it is greater than zero are repeated until all sub-operators in the first sub-operator set have been selected.

[0019] In one embodiment, the method further comprises:

[0020] Starting from the sub-operator with the smallest number of input elements in the second sub-operator set, the sum of the number of input elements of the sub-operators is accumulated in sequence;

[0021] Searching in the first sub-operator set whether there is a minimum sub-operator whose input element number is greater than the sum of the input element numbers;

[0022] If it exists, then compare the sum of the running times of the sub-operators participating in the accumulation of the input number of elements with the running time of the minimum sub-operator;

[0023] If the sum of the running times is greater than the running time of the smallest sub-operator, then in the second sub-operator set, the smallest sub-operator is used to replace all the sub-operators participating in the accumulation of the input element number.

[0024] In one embodiment, if the sum of the running times is less than or equal to the running time of the smallest sub-operator, the second sub-operator set remains unchanged.

[0025] In a second aspect, an embodiment of the present invention provides a graph optimization method, including:

[0026] Scan the operators in the computation graph;

[0027] When an operator whose input element number is greater than 2 is scanned, the operator is optimized by executing the optimization method of the multivariate operator as described above;

[0028] In the computation graph, the corresponding operators are replaced with the second sub-operator set.

[0029] In a third aspect, an embodiment of the present invention provides a graph computing method, including:

[0030] Use the above graph optimization method to optimize the computational graph;

[0031] Perform graph calculation according to the optimized calculation graph.

[0032] In a fourth aspect, an embodiment of the present invention provides an optimization device for a multivariate operator, including:

[0033] A splitting module is used to split the multivariate operator to be optimized into multiple sub-operators with a preset number of input elements, so that when the multiple sub-operators obtained by splitting are subjected to hierarchical calculation, the total number of input elements is not less than the number of input elements of the multivariate operator, and the number of sub-operators obtained by splitting is the least;

[0034] The optimization module is used to use the multiple sub-operators obtained after the splitting as the optimization results of the multivariate operator.

[0035] In a fifth aspect, an embodiment of the present invention provides a computational graph model, in which operators with input elements greater than 2 contained in the graph model are replaced by a second sub-operator set; the second sub-operator set is obtained using the optimization method of the multivariate operator as described above.

[0036] In a sixth aspect, an embodiment of the present invention provides a processor, which is used to implement the aforementioned multivariate operator optimization method, or the aforementioned graph optimization method, or the aforementioned graph computing method.

[0037] In the seventh aspect, an embodiment of the present invention provides a device, comprising: a memory and a processor; wherein the memory stores a computer program, and when the program is executed by the processor, it can implement the optimization method of the multivariate operator as described above, or the graph optimization method as described above, or the graph calculation method as described above.

[0038] In an eighth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, can implement the aforementioned multivariate operator optimization method, or the aforementioned graph optimization method, or the aforementioned graph calculation method.

[0039] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0040] The multivariate operator optimization method, graph optimization method, graph computing method, device and equipment provided in the embodiments of the present invention split the multivariate operator to be optimized into multiple sub-operators with a preset input number, so that the total input number of the multiple sub-operators obtained by splitting is not less than the input number of the multivariate operator when performing layered calculations, and the number of sub-operators obtained by splitting is the least, and the multiple sub-operators after splitting are used to replace the multivariate operator, thereby realizing the optimization of the multivariate operator; the embodiment of the present invention optimizes the multivariate operator into several sub-operators with a fixed input number. On the one hand, the multiple sub-operators obtained after optimization have a small number of sub-operators, and the input tensor With fewer slices and a reduced number of I / Os, the efficiency of moving data from DDR is higher, thereby improving the computing speed of the device. On the other hand, using a set of sub-operators with a fixed number of input elements to replace the original multi-operator can also simplify the difficulty of the device's judgment of the number of input elements, memory slicing and loop number calculation operations, allowing the device to focus more on the operator operation itself, saving running time and computing resources, reducing device performance requirements, and improving the device's computing efficiency, avoiding the problems of increased I / O times, low computing efficiency and high device performance requirements caused by the arbitrary number of input elements brought about by the existing unoptimized multi-operators.

[0041] Furthermore, the optimization method for multi-operators provided in an embodiment of the present invention, after obtaining the second sub-operator set, again compares the sum of the running times of the smaller sub-operators in the second sub-operator set with the running time of the sub-operators in the first sub-operator set that are only greater than the sum of the input elements of these smaller sub-operators. If the latter is smaller than the former, the sub-operator with the larger input element is used to replace all these smaller sub-operators. This is because in the device, in the total time overhead of the multi-operators, the time spent on moving data from the DDR accounts for a large proportion, while the time spent on processor calculation accounts for a relatively low proportion. Therefore, this method of replacing multiple smaller sub-operators with a larger sub-operator is adopted. Although some sacrifices are made in storage space, on the premise of meeting the number of input elements, the total number of operators in the second sub-operator set is reduced, which can further be exchanged for a reduction in the number of I / Os, thereby reducing I / O losses as a whole and improving the computing speed of the device.

[0042] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0043] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0045] Figure 1 A flowchart of a multivariate operator optimization method provided by an embodiment of the present invention;

[0046] Figure 2 A specific implementation diagram of step S11 in the multivariate operator optimization method provided in an embodiment of the present invention;

[0047] Figure 3 A calculation flow chart of a specific example provided for an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of a stack S provided in an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of a plurality of different sub-operator operation processes provided by an embodiment of the present invention;

[0050] Figure 6 Another calculation flow chart of a specific example provided by an embodiment of the present invention;

[0051] Figure 7 A flowchart of a graph optimization method provided by an embodiment of the present invention;

[0052] Figure 8 An example diagram of a computational graph provided for an embodiment of the present invention;

[0053] Fig. 9 A flowchart of a graph computing method provided by an embodiment of the present invention;

[0054] Fig.10 A structural block diagram of a multivariate operator optimization device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0056] In conjunction with the accompanying drawings, specific implementation modes of the multivariate operator optimization method, graph optimization method, graph computing method, device and apparatus provided in the embodiments of the present invention are described below.

[0057] The optimization method of the multivariate operator provided in the embodiment of the present invention is referred to Figure 1As shown, the following steps are included:

[0058] S11, splitting the multivariate operator to be optimized into a plurality of sub-operators with a preset number of input elements, so that when the plurality of sub-operators obtained by splitting are subjected to hierarchical calculation, the total number of input elements is not less than the number of input elements of the multivariate operator, and the number of sub-operators obtained by splitting is the least;

[0059] S12. Using the multiple sub-operators obtained after splitting as optimization results of the multivariate operator.

[0060] The above-mentioned multivariate operator optimization method provided by the embodiment of the present invention splits the multivariate operator to be optimized into multiple sub-operators with a preset number of input elements, so that the total number of input elements of the multiple sub-operators obtained by splitting is not less than the number of input elements of the multivariate operator when performing layered calculations, and the number of sub-operators obtained by splitting is the least, and the multiple sub-operators after splitting are used to replace the multivariate operator, thereby realizing the optimization of the multivariate operator. The multivariate operator is optimized into, on the one hand, the multiple sub-operators obtained after optimization, due to the small number of sub-operators, the slices of the input tensor become fewer, the number of I / Os is reduced, and the efficiency of moving data from DDR is higher, thereby improving the computing speed of the device. On the other hand, using a set of sub-operators with a fixed number of input elements to replace the original multivariate operator can also simplify the difficulty of the device's judgment on the number of input elements, memory slicing and loop number calculation operations, so that the device can focus more on the operator operation itself, save running time and computing resources, reduce the performance requirements of the device, and improve the computing efficiency of the device, avoiding the problems of increased I / O times, low computing efficiency and high performance requirements for the device caused by the arbitrary number of input elements brought by the existing unoptimized multivariate operator.

[0061] Specifically, in the above step S11, the multivariate operator to be optimized is split into a plurality of sub-operators with a preset number of input elements. Figure 2 As shown, this can be achieved through the following process:

[0062] S21, according to the input element number of the multivariate operator to be optimized and the input element number of the sub-operators in the first sub-operator set, according to a preset greedy strategy, selecting a sub-operator from the first sub-operator set; the first sub-operator set includes at least two preset sub-operators with different input element numbers;

[0063] In order to optimize the multivariate operator, it is necessary to prepare several sub-operators with fixed input numbers according to the characteristics of the multivariate operator and the usage scenarios. For example, for the Concat operator, various forms of sub-operators with 2-input, 3-input, 4-input and 8-input numbers can be preset. These sub-operators constitute a sub-operator set, which is referred to as the first sub-operator set in the embodiment of the present invention for convenience.

[0064] The input elements of the sub-operators in the first sub-operator set are different from each other.

[0065] S22. Generate a second sub-operator set based on the selected sub-operators, so that when the sub-operators in the second sub-operator set are used for hierarchical calculation, the total number of input elements is not less than the number of input elements of the multivariate operator, and the number of sub-operators obtained by splitting is the least.

[0066] In the first sub-operator set, different sub-operators have different numbers of input elements. From the perspective of the time it takes the processor to calculate the operator (also known as the time it takes for the operator to run independently), sub-operators with smaller numbers of input elements usually have shorter running times and occupy less storage space. Conversely, sub-operators with larger numbers of input elements usually have longer running times and occupy more storage space.

[0067] However, from the perspective of the entire operator calculation process, the total time overhead of the operator calculation not only includes the time the processor takes to calculate each operator, but also includes the process of moving data from DDR to registers (such as RAM). Moreover, the latter takes a larger proportion of time than the former. To achieve the same number of input elements, if you want to have high overall computing efficiency, you will require the number of I / Os to be as small as possible. In other words, when selecting a sub-operator, start with the sub-operator with the largest number of input elements, and then select the sub-operator with a smaller number of input elements according to the number of input elements.

[0068] The above step S21 can be implemented using a greedy algorithm. A greedy algorithm means that when solving a problem, the best choice is always made at the moment. In other words, it does not consider the overall optimal solution, and it only makes a local optimal solution in a certain sense. In a specific application scenario of the greedy algorithm, in the scenario of shopping and making change, because only coins of fixed denominations can be used, in order to minimize the number of coins in the change, all solutions for making change are not required, but starting from the currency with the largest denomination, each denomination is considered in descending order, and the denomination with the largest denomination is used as much as possible first. When the remaining amount is less than the large denomination, the next smaller denomination is considered.

[0069] Returning to the scenario of multi-operator optimization in the embodiment of the present invention, similar to the principle of making change, it is necessary to try to select the sub-operator with the largest number of input elements that can be selected at each selection. Only when the remaining number of input elements is insufficient to use the sub-operator, the sub-operator with the next largest number of input elements is considered, and so on, until the total number of input elements can meet the requirement of the number of input elements of the multi-operator to be optimized when all the selected sub-operators are calculated in layers, and the algorithm stops.

[0070] Specifically, if a greedy algorithm is used, in the above steps S21 and S22, sub-operators may be selected from the first sub-operator set in the order of the largest input element number, starting from the first sub-operator set;

[0071] For each selected sub-operator, determine whether the remaining input element number of the multi-operator is greater than zero, except for the input element number provided by all selected sub-operators for hierarchical calculation;

[0072] If it is determined that the remaining input element number is greater than zero, the current sub-operator is placed in the second sub-operator set; and the current sub-operator is selected again, and the steps of calculating the remaining input element number of the multivariate operator and determining again whether it is greater than zero are repeated until the remaining input element number of the multivariate operator is less than or equal to zero;

[0073] If it is determined that the remaining number of input elements of the multivariate operator is less than or equal to zero, the current sub-operator is abandoned, and a sub-operator with a smaller number of input elements than the current sub-operator is selected from the first sub-operator set. The steps of calculating the remaining number of input elements of the multivariate operator and determining whether it is greater than zero are repeated until all sub-operators in the first sub-operator set have been selected.

[0074] The above implementation process is explained by taking a specific implementation process as an example. Because in the first sub-operator set, each sub-operator corresponds to a different number of input elements, so different input element numbers represent different sub-operators in the first sub-operator set. In order to simplify the description, in this process, the sub-operator selection process is simplified to the calculation process of the sub-operator input element number. Assume that the multivariate operator to be optimized is an N-ary operator. For this N-ary operator, n seed operators are pre-set, and their input element numbers are arranged in descending order from large to small as follows: x1, x2, x3...x n The set of these input numbers is called set U. In set U, we select in order starting from the number with the largest input number, that is, x1.

[0075] In this reference Figure 3 As shown, the process includes the following steps:

[0076] S31. Select x from the set U i ;

[0077] x i The initial value of the subscript i in is 1;

[0078] S32, calculate N=N-(x i -1);

[0079] S33, determine whether N is less than or equal to 0; if the determination is yes, execute the following step S35, otherwise execute the following step S34;

[0080] S34, push the sub-operator xi into the stack S, and execute S31 in a loop;

[0081] S35, calculate N=N+(xi-1), and continue to execute the following step S36;

[0082] S36, execute the calculation of i adding 1, and continue to execute S37;

[0083] S37, judging whether i exceeds the preset number of sub-operators; if so, ending the process; if not, repeating S31.

[0084] The following takes the optimization process of a 20-element operator as an example. The preset sub-operators are four sub-operators with 2-input, 3-input, 4-input, and 8-input arities. Corresponding to the above process, N=20, and the set U is {8, 4, 3, 2}. The implementation process is as follows:

[0085] 1) i starts from 1, select x1=8 from the set U, i=1, and calculate N=20-(8-1)=13. x1 minus 1 equals 7, and the difference of 20 minus 7 is assigned to N. The current value of N changes to 13. Since 13 is greater than 0, the input number x1 is pushed onto stack S. The current content of stack S is 8, and the algorithm continues;

[0086] 2) Continue to select x1 from the set, x1=8, i=1, calculate N=13-(8-1)=6, xi minus 1 equals 7, assign the difference of 13 minus 7 to N, the current value of N changes to 6, because 6 is greater than 0, the input number x1 is pushed to the stack S again, the current content of stack S is: 8, 8, the algorithm continues to execute;

[0087] 3) Continue to select x1 from the set, x1=8, i=1, calculate N=6-(8-1)=-1; xi is equal to 8, xi minus 1 is equal to 7, assign the difference of 6 minus 7 to N, and the current value of N changes to -1. Since -1 is less than 0, continue to calculate N=-1+(8-1)=6, assign the sum of -1 plus 7 to N, and the current value of N changes to 6. Then execute i to add 1, and the current value of i changes to 2. The capacity of set U is 4. Since 2 is still less than 4 and does not exceed the capacity of set U, the current content of stack S is 8, 8, and the algorithm continues;

[0088] 4) Select x2 from the set, x2=4, i=2, calculate N=6-(4-1)=3; x2 minus 1 equals 3, assign the difference of 6 minus 3 to N, and the current value of N changes to 3. Since 3 is greater than 0, push the input number x2 to stack S. The current content of stack S is 4, 8, 8, and the algorithm continues;

[0089] 5) Select x2 from the set, x2=4, i=2, calculate N=3-(4-1)=0, x2 minus 1 equals 3, assign the difference of 3 minus 3 to N, the current value of N changes to 0, since 0 is equal to 0, calculate N=0+(4-1)=3 again, assign the sum of 0 plus 3 to N, the current value of N changes to 3, execute i to increase by 1, the current value of i changes from 2 to 3, the capacity of set U is 4, since 3 is less than 4, it does not exceed the capacity of set U, the current content of stack S is 4, 8, 8, the algorithm continues;

[0090] 6) Select x3 from the set, x3=3, i=3, calculate N=3-(3-1)=1, x3 minus 1 equals 2, assign the difference of 3 minus 2 to N, the current value of N changes to 1, since 1 is greater than 0, push the input number x3 to stack S, the current content of stack S is {3, 4, 8, 8}, and the algorithm continues;

[0091] 7) Select x3 from the set, x3=3, i=3; calculate N=1-(3-1)=-1, X3 minus 1 equals 2, assign the difference of 1 minus 2 to N, the current N value changes to -1, since -1 is less than 0, calculate N=-1+(3-1)=1 again; assign the sum of -1 plus 2 to N, the current N value changes to 1, i increases by 1, the current i value changes to 4, the capacity of the set U is 4, since 4 is equal to 4, it does not exceed the capacity of the set U, the current contents of the stack S are 3, 4, 8, 8, and the algorithm continues;

[0092] 8) Select x4 from the set, x4=2, i=4, calculate N=1-(2-1)=0, x4 minus 1 equals 1, assign the difference of 1 minus 1 to N, the current N value changes to 0, since 0 is equal to 0, continue to calculate N=0+(2-1)=1; assign the sum of 0 plus 1 to N, the current N value changes to 1, i increases by 1, the current i value changes to 5, the capacity of set U is 4, since 5 is greater than 4, it exceeds the capacity of set U, so the algorithm ends, refer to Figure 4 As shown, the contents of stack S are {3, 4, 8, 8} from top to bottom.

[0093] In stack S, from top to bottom, each input arity corresponds to the 3-input arity sub-operator, 4-input arity sub-operator, 8-input arity sub-operator and 8-input arity sub-operator respectively. During calculation, the operation order is the order of stack S from bottom to top, that is, the 8-input arity sub-operator, 8-input arity sub-operator, 4-input arity sub-operator and 3-input arity sub-operator are executed in turn.

[0094] Reference Figure 5As shown, during layered operations, the calculation results of the previous layer will be passed to the next layer. Therefore, when the above-mentioned 8-input numeral sub-operator, 8-input numeral sub-operator, 4-input numeral sub-operator and 3-input numeral sub-operator are layered in sequence, the first layer needs to input 8 numbers, the second layer needs to input 7 numbers, the third layer needs to input 3 numbers, and the fourth layer needs to input 2 numbers. The sum of the required input elements is: 8+7+3+2=20 elements, which is consistent with the number of input elements of the 20-element operator to be optimized.

[0095] As mentioned above, in the total time cost of operator calculation, the time cost of moving data from DDR to registers (such as RAM) is much greater than the time cost of the processor calculating the operator alone. Therefore, on the basis of meeting the total input element number requirement, small terminal operators can be further merged (such as the 3-input element number sub-operator and the 4-input element number sub-operator in the above example), and replaced with sub-operators whose input element numbers are slightly larger than the sum of the two input element numbers. Although a little storage space of the register is sacrificed, the number of sub-operators in the second sub-operator set can be further reduced, which further reduces the number of I / Os, makes full use of hardware resources, balances the loss of computing resources and storage overhead, and improves the overall computing efficiency.

[0096] In the embodiment of the present invention, further, starting from the sub-operator with the smallest input number in the second sub-operator set, the sum of the input numbers of the sub-operators can be accumulated in sequence; then, in the first sub-operator set, it is searched whether there is a minimum sub-operator with an input number greater than the sum of the input numbers (that is, a sub-operator with an input number greater than only the sum of the input numbers);

[0097] If it exists, the sum of the running time of the sub-operators participating in the accumulation of input numbers is compared with the running time of the minimum sub-operator;

[0098] If the sum of the running times is greater than the running time of the smallest sub-operator, then in the second sub-operator set, the smallest sub-operator is used to replace all sub-operators participating in the accumulation of input elements.

[0099] Specifically, the above process can be Figure 6 The algorithm flow chart shown is implemented. In this algorithm flow, the independent running time of each sub-operator is counted in advance, t=g(x). In order to simplify the calculation, the merging of small terminal operators is still achieved by calculating the number of input elements of the sub-operators.

[0100] Reference Figure 6 As shown, the flow chart includes the following steps:

[0101] S61, sequentially obtain the input element number x from the S pop stack i ;

[0102] S62, cumulative input number, that is, calculation X = X + x i ; The initial value of X is 0;

[0103] S63, cumulative sub-operator independent running time overhead T = T + g (x i ); the initial value of T is 0;

[0104] S64, x i Enqueue the preset queue S bak ;

[0105] S65. Find the minimum input number x that is only greater than X in the set U min ;

[0106] S66, determine whether X is greater than Xmin ,and Xmin belongs to the set U; if the judgment result is yes (which means that the minimum input element number x that is only greater than X cannot be found in U min ), then the process ends, otherwise, the following step S67 is executed;

[0107] S67, determine whether T is less than the minimum input element number x min The independent running time cost of the sub-operator g(x min ), if yes, repeat S61, otherwise execute step S68.

[0108] S68, x min Push the stack to S and clear S bak , and then repeat step S61.

[0109] After executing the above steps S61 to S68, bak The number in enters the S stack again, and the merged S stack is the input element number of the sub-operator finally decomposed, and the calculation order of the final sub-operator conforms to the order of the S stack from bottom to top.

[0110] Taking the above 20-element operator optimization process as an example, the preset sub-operators are four sub-operators with 2 inputs, 3 inputs, 4 inputs, and 8 inputs. Figure 3 After the process shown, the number of input elements in the S stack from top to bottom is: {3, 4, 8, 8}.

[0111] In this process, the independent running time cost of the sub-operator of each input number in the set U is calculated separately: g(8), g(4), g(3) and g(2) for backup.

[0112] Then follow the steps below:

[0113] 1) Pop \(x_i = 3\) from the S stack. The cumulative input element count is \(X = 0 + 3 = 3\); the cumulative overhead of the sub-operator is \(T = 0+g(3)\).

[0114] Put \(x_i = 3\) into the S bak queue, and find the smallest input element count \(x\) in \(U\) that is only greater than \(X\). min It is 4. Since \(X = 3\) is less than 4, execute S67 to compare the magnitudes of \(g(3)\) and \(g(4)\). Generally, the more input elements a sub-operator has, the longer the independent running time of the sub-operator. Based on this, assume \(g(3)<g(4)\), then execute step S61 again.

[0115] 2) Continue to obtain the next input element count \(x_i = 2\) from the S stack (the previous 3 has been popped from the stack). The cumulative \(X = 3 + 4 = 7\); the cumulative time overhead of the operator is \(T = g(3)+g(4)\). Put \(x_2\) into the queue S bak , and find the smallest input element count \(x\) in the set \(U\) that is only greater than \(X\). min It is 8. Execute S66 to determine whether 7 is greater than \(x\). min Since it is not greater, execute S67. Then, determine again whether \(T = g(3)+g(4)\) is less than \(g(8)\).

[0116] There will be two cases here: which are 3)' and 4)' below respectively.

[0117] 3) Since the result of \(g(3)+g(4)\) is greater than \(g(8)\), continue to execute S68 and push \(x\) min \( = 8\) onto the S stack and empty S. bak . The data in the S stack is \(\{8, 8, 8\}\);

[0118] In the case where \(g(3)+g(4)\) is greater than \(g(8)\), continue to execute S68 and pop \(x\) i \( = 8\) from S. The cumulative input element count is \(X = 7 + 8 = 15\); the cumulative time overhead of the sub-operator is \(T = g(3)+g(4)+g(8)\). Put \(x\) i \( = 8\) into the queue S. bak , and find the smallest input element count in \(U\) that is only greater than \(X\). Since it cannot be found, the process ends.

[0119] At this time, the final data in S is \(\{8, 8, 8\}\).

[0120] 4) In the other case, since the result of \(g(3)+g(4)\) is less than or equal to \(g(8)\), loop to execute S61, that is, abandon the currently selected \(x\). min .

[0121] When g(3)+g(4) is less than or equal to g(8), continue to execute S61, that is, pop xi=8 from the stack S; accumulate the number of input elements X=7+8=15; accumulate the sub-cost T=g(3)+g(4)+g(8), and put xi=8 into the queue S. bak , search for the minimum input element number that is just greater than X in the set U, but no element is found. The result of executing S66 is yes, and the process ends. bak The data in the stack are: {3, 4, 8}. After three pops, S is {8}. Combine the two, for example, S bak The data in are pushed into S in sequence, and the number of input elements in S is {3, 4, 8, 8}, and the corresponding execution order is 8-input element operator, 8-input element operator, 4-input element operator and 3-input element operator. It can be seen that the result is consistent with the previous process and the result remains unchanged.

[0122] The embodiment of the present invention also provides a method for graph optimization, referring to Figure 7 As shown, the method comprises the following steps:

[0123] S71, scanning the operators in the computation graph;

[0124] S72, when an operator whose input element number is greater than 2 is scanned, the aforementioned optimization method for the multivariate operator is executed to optimize the operator;

[0125] S73. In the computation graph, replace corresponding operators with the second sub-operator set.

[0126] This graph optimization process can be done offline. Before computing the graph, the above graph optimization method can be implemented in advance to replace the multivariate operator with a set of corresponding sub-operators to achieve the computing efficiency of the device. It can also be done during the graph computing process, while optimizing the multivariate operator and completing the computation of the computation graph.

[0127] An example of a computational graph is Figure 8 As shown, starting from the input operator node (input) of the computation graph, scan in sequence until an operator with more than two inputs is found and perform operator optimization operations on the operator.

[0128] For example, F n (A, B, C, D, E) is a multivariate operator to be optimized. If the final value in the S stack is {4, 2}, then in the computation graph, the operator can be replaced in advance with the following sub-operator set F2 (F4 (A, B, C, D), E).

[0129] Reference Fig. 9 As shown, an embodiment of the present invention further provides a graph computing method, the method comprising:

[0130] S91. Optimize the computation graph using the aforementioned graph optimization method.

[0131] S92. Perform graph calculation according to the optimized calculation graph.

[0132] As mentioned above, the graph optimization method can be completed before the calculation, or it can be completed during the calculation process, while the calculation is being performed, and the optimization can be completed before the corresponding multivariate operator is calculated.

[0133] Based on the same inventive concept, an embodiment of the present invention also provides an optimization device, a computational graph model, a processor and a device for a multivariate operator. Since the principles of solving the problems by these devices and clients are similar to those of the aforementioned optimization method for multivariate operators, the implementation of the device and client can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0134] The embodiment of the present invention provides an optimization device for a multivariate operator, referring to Fig.10 As shown, including:

[0135] The splitting module 101 is used to split the multivariate operator to be optimized into a plurality of sub-operators with a preset number of input elements, so that when the multiple sub-operators obtained by splitting are subjected to hierarchical calculation, the total number of input elements is not less than the number of input elements of the multivariate operator, and the number of sub-operators obtained by splitting is the least;

[0136] The optimization module 102 is used to use the multiple sub-operators obtained after the splitting as the optimization results of the multivariate operator.

[0137] The splitting module 101 is used to select sub-operators from the first sub-operator set according to the input element number of the multi-operator to be optimized and the input element number of the sub-operators in the first sub-operator set according to a preset greedy strategy; the first sub-operator set contains at least two preset sub-operators with different input element numbers; and generate a second sub-operator set according to the selected sub-operators, so that when the sub-operators in the second sub-operator set are used for hierarchical calculation, the total input element number is not less than the input element number of the multi-operator, and the number of sub-operators obtained by splitting is the least.

[0138] The splitting module 101 is further used to select sub-operators from the first sub-operator set in order of size, starting from the one with the largest input element number; for each selected current sub-operator, determine whether the remaining input element number of the multi-operator is greater than zero in addition to the input element number provided by all selected sub-operators for the hierarchical calculation; if it is determined that the remaining input element number is greater than zero, put the current sub-operator into the second sub-operator set; and select the current sub-operator again, repeat the steps of calculating the remaining input element number of the multi-operator and judging whether it is greater than zero again, until the remaining input element number of the multi-operator is less than or equal to zero; if it is determined that the remaining input element number of the multi-operator is less than or equal to zero, abandon the current sub-operator, and select a sub-operator with an input element number smaller than the current sub-operator from the first sub-operator set, repeat the steps of calculating the remaining input element number of the multi-operator and judging whether it is greater than zero again, until all sub-operators in the first sub-operator set have been selected.

[0139] Reference Fig.10 As shown, the optimization device for multiple operators also includes: an operator merging module 103, which is used to accumulate the total number of input elements of sub-operators in sequence starting from the sub-operator with the smallest input element number in the second sub-operator set; search in the first sub-operator set whether there is a minimum sub-operator with an input element number greater than the total number of input elements; if so, compare the total running time of the sub-operators participating in the accumulation of input elements with the running time of the minimum sub-operator; if the total running time is greater than the running time of the minimum sub-operator, use the minimum sub-operator in the second sub-operator set to replace all the sub-operators participating in the accumulation of input elements.

[0140] An embodiment of the present invention further provides a processor, which is used to implement the aforementioned multivariate operator optimization method, or the aforementioned graph optimization method, or the aforementioned graph computing method.

[0141] In one embodiment, the processor is a CPU or a digital signal processor (DSP).

[0142] An embodiment of the present invention also provides a device, which includes a memory and a processor; wherein the memory stores a computer program, and when the program is executed by the processor, it can implement the optimization method of the multivariate operator as described above, or the graph optimization method as described above, or the graph calculation method as described above.

[0143] An embodiment of the present invention also provides a non-temporary computer-readable storage medium storing computer instructions, which, when executed by a processor, can implement the aforementioned multivariate operator optimization method, or the aforementioned graph optimization method, or the aforementioned graph calculation method.

[0144] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0145] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A graph computing method applied to a forward computing process of a convolutional neural network, the forward computing process comprising: When data is input from the input layer, the calculation is performed layer by layer according to the calculation graph until the output layer. The characteristic of the calculation is that the calculation is performed according to the calculation graph, including: Scan the operators in the computation graph; When an operator with an input element number greater than 2 is scanned, the following method is performed to optimize the operator: the multivariate operator to be optimized is split into multiple sub-operators with a preset input element number, so that the total input element number of the multiple sub-operators obtained by splitting is not less than the input element number of the multivariate operator when performing hierarchical calculations, and the number of sub-operators obtained by splitting is the least; the multiple sub-operators obtained after splitting are used as the optimization results of the multivariate operator; In the computation graph, using a second sub-operator set to replace the corresponding operator to optimize the computation graph; Graph calculation is performed according to the optimized calculation graph; during the graph calculation process, on the device side, based on the optimized multivariate operator, according to the tensor length and the number of tensors, how the storage space on the RAM is arranged and the number of cycles for moving data from the DDR to the RAM are dynamically calculated, and according to the optimized multivariate operator, the block operation of the input element is determined.

2. The method according to claim 1, characterized in that Split the multivariate operator to be optimized into multiple sub-operators with preset input numbers, including: According to the input element number of the multivariate operator to be optimized and the input element number of the sub-operators in the first sub-operator set, a sub-operator is selected from the first sub-operator set according to a preset greedy strategy; the first sub-operator set includes at least two preset sub-operators with different input element numbers; A second sub-operator set is generated based on the selected sub-operators, so that when the sub-operators in the second sub-operator set are used for hierarchical calculation, the total number of input elements is not less than the number of input elements of the multivariate operator, and the number of sub-operators obtained by splitting is the least.

3. The method according to claim 2, characterized in that According to the greedy strategy, the sub-operator is selected from the first sub-operator set, and the second sub-operator set is generated according to the selected sub-operator, including: From the first sub-operator set, select sub-operators in order of size, starting from the one with the largest input element number; For each selected current sub-operator, determine whether the remaining input element number of the multi-element operator is greater than zero, except for the input element number provided by all the selected sub-operators for the hierarchical calculation; If it is determined that the remaining input element number is greater than zero, the current sub-operator is placed in the second sub-operator set; and the current sub-operator is selected again, and the steps of calculating the remaining input element number of the multivariate operator and determining again whether it is greater than zero are repeated until the remaining input element number of the multivariate operator is less than or equal to zero; If it is determined that the remaining input element number of the multivariate operator is less than or equal to zero, the current sub-operator is abandoned, and a sub-operator with an input element number smaller than the current sub-operator is selected from the first sub-operator set, and the steps of calculating the remaining input element number of the multivariate operator and determining again whether it is greater than zero are repeated until all sub-operators in the first sub-operator set have been selected.

4. The method according to claim 2 or 3, characterized in that The method further comprises: Starting from the sub-operator with the smallest number of input elements in the second sub-operator set, the sum of the number of input elements of the sub-operators is accumulated in sequence; Searching in the first sub-operator set whether there is a minimum sub-operator whose input element number is greater than the sum of the input element numbers; If it exists, then compare the sum of the running times of the sub-operators participating in the accumulation of the input number of elements with the running time of the minimum sub-operator; If the sum of the running times is greater than the running time of the smallest sub-operator, then in the second sub-operator set, the smallest sub-operator is used to replace all the sub-operators participating in the accumulation of the input element number.

5. The method according to claim 4, characterized in that If the sum of the running times is less than or equal to the running time of the smallest sub-operator, the second sub-operator set remains unchanged.

6. A processor, characterized in that: The processor is used to implement the method for graph calculation applied to the forward calculation process of a convolutional neural network as described in any one of claims 1-5.

7. A device, characterized in that: include: A memory and a processor; wherein the memory stores a computer program, and when the program is executed by the processor, it can implement the method for graph calculation applied to the forward calculation process of a convolutional neural network as described in any one of claims 1 to 5.

8. A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, can implement the method for graph calculation applied to the forward calculation process of a convolutional neural network as described in any one of claims 1 to 5.

Images