Methods, systems, devices, and media for optimizing the layout format of computation graphs

Abstract

A method, system, electronic device, and non-temporary storage medium for optimizing the layout format of a computation graph are provided. The method includes: calculating multiple total cost functions for each operator under multiple layout formats based on information about the expected number of times operators in the computation graph are executed; and determining the layout format corresponding to the minimum total cost function among the multiple layout formats to optimize the computation graph by comparing the minimum total cost functions among the multiple layout formats. Thus, by considering the expected number of times operators in the computation graph are executed to calculate the multiple total cost functions for each operator under multiple layout formats, the actual performance impact of each operator due to the expected number of executions can be more accurately considered, thereby determining which layout format is more optimized and using that layout format to optimize the computation graph.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically to methods, systems, electronic devices, and non-temporary storage media for optimizing the layout format of computation graphs. Background Technology

[0002] The core of artificial intelligence learning tasks lies in the definition of the model and the method of solving for its parameters. These two can be abstracted into a single computational logic, namely, a computational graph. Nodes in the computational graph represent operators (Ops). Operators correspond to computational logic; for example, a convolutional layer is an operator; the weight summation process in a fully-connected layer (FC layer) is an operator; and operators like tanh and ReLU are used as activation functions. The connections between nodes represent the flow relationships between them. During the creation of the computational graph, a series of processing steps are required, including device placement, graph optimization, and graph partitioning. Graph optimization, which runs through different stages of the computational graph processing, is one of the most important steps in saving computational resources. Summary of the Invention

[0003] According to one aspect of this application, a method for optimizing the layout format of a computation graph is provided, comprising: calculating multiple total cost functions of each operator under multiple layout formats based on information about the number of times operators in the computation graph are expected to be executed; and determining a layout format corresponding to the minimum total cost function among the multiple layout formats to optimize the computation graph by comparing the minimum total cost function among the multiple layout formats.

[0004] According to another aspect of this application, a system for optimizing the layout format of a computation graph is provided, comprising: a computing device configured to calculate a plurality of total cost functions for each operator under a plurality of layout formats based on information about the number of times operators in the computation graph are expected to be executed; and a determining device configured to determine, by comparing and obtaining the minimum total cost function among the plurality of total cost functions, a layout format corresponding to the minimum total cost function among the plurality of layout formats to optimize the computation graph.

[0005] According to another aspect of this application, an electronic device is provided, comprising: a memory for storing instructions; and a processor for reading the instructions from the memory and executing a method according to an embodiment of this application.

[0006] According to another aspect of this application, a non-temporary storage medium is provided, on which instructions are stored, wherein, when read by a processor, the instructions cause the processor to perform a method according to an embodiment of this application.

[0007] In this way, by taking into account the expected number of times operators in the computation graph are executed, multiple total cost functions of each operator under multiple layout formats can be calculated. This allows for a more accurate consideration of the actual performance impact of each operator due to the expected number of times each operator is executed, thereby determining which layout format is more optimized and using that layout format to optimize the computation graph. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1A This diagram illustrates a branched computation graph with layout constraint conflicts between operators A and B.

[0010] Figure 1B and 1C The diagrams show two alternative solutions for resolving layout constraint conflicts by inserting the corresponding ReorderOp operator.

[0011] Figure 2 This diagram illustrates the basic information of the TensorFlow software.

[0012] Figure 3 Specific examples of branching operators and their branches in a computation graph are shown.

[0013] Figure 4 A schematic diagram of a computation graph with loops is shown.

[0014] Figure 5A It shows that in such Figure 4 The diagram shown illustrates Scheme 1, in which the R operator in the computational graph with loops converts layout format L0 to layout format L1.

[0015] Figure 5B It shows that in such Figure 4 The diagram shown illustrates Scheme 2, in which the R operator in the computational graph with loops converts layout format L0 to layout format L2.

[0016] Figure 6 A flowchart illustrating a method for optimizing the layout format of a computation graph according to an embodiment of this application is shown.

[0017] Figure 7 A block diagram of a system for optimizing the layout format of a computation graph according to an embodiment of this application is shown.

[0018] Figure 8 A block diagram of an exemplary electronic device suitable for implementing embodiments of this application is shown.

[0019] Figure 9 A schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of this application is shown. Detailed Implementation

[0020] Specific embodiments of this application will now be described in detail, with examples of the application illustrated in the accompanying drawings. Although this application will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit this application to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of this application as defined by the appended claims. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0021] To ensure operators achieve optimal performance on a specific chip, a specific data layout / format can be designed for that operator. Executing an operator with the appropriate data layout / format will result in better performance. Operators can sometimes support multiple data layouts / formats (hereinafter collectively referred to as layout formats). For example, data layout formats include COLMAJOR (column-major), ROWMAJOR (row-major), and BUFFER, while data formats include NCHW, NHWC, HWIO, and OIHW, where N represents the batch size (the number of elements selected at once); C represents the number of feature maps, also known as the number of channels; H represents the height; W represents the width; I represents the input channel; and O represents the output channel.

[0022] When two or more data layout formats are available, existing techniques determine which data layout format to use to optimize the computation graph to obtain a better-performing solution by directly calculating the sum of the performance parameters of each operator under the two available data layout formats (e.g., represented by a cost function).

[0023] However, this method of determination is not accurate enough, as it does not take into account the expected number of times each operator is executed. Therefore, it cannot accurately account for the actual performance impact of each operator due to the expected number of times each operator is executed.

[0024] Figure 6 A flowchart of a method 600 for optimizing the layout format of a computation graph according to an embodiment of this application is shown.

[0025] like Figure 6As shown, a method 600 for optimizing the layout format of a computation graph includes: step 610, calculating multiple total cost functions for each operator under multiple layout formats based on information about the number of times operators in the computation graph are expected to be executed; step 620, determining the layout format corresponding to the minimum total cost function among the multiple layout formats to optimize the computation graph by comparing and obtaining the minimum total cost function among the multiple layout formats.

[0026] In this way, by taking into account the expected number of times operators in the computation graph are executed, multiple total cost functions of each operator under multiple layout formats can be calculated. This allows for a more accurate consideration of the actual performance impact of each operator due to the expected number of times each operator is executed, thereby determining which layout format is more optimized and using that layout format to optimize the computation graph.

[0027] Step 610: Calculate multiple total cost functions for each operator under multiple layout formats based on the information about the number of times the operators in the computation graph are expected to be executed.

[0028] In one embodiment, the information about the number of times an operator in the computation graph is expected to be executed includes the probability that the operator is executed in the case of conditional branching or the number of times the operator is iterated in the case of loop iteration.

[0029] The expected number of times operators in the computation graph are executed can be obtained during the sampling phase. Sampling provides information in the following two aspects:

[0030] l-Conditional branch probability: The probability that different branch operators of the condition judgment in the computation graph will be executed.

[0031] l Iteration count: The number of times the operator is iterated in the computation graph.

[0032] Based on these two pieces of information, we can better guide the layout optimization process and select the best optimization solution.

[0033] In one embodiment, the expected number of times an operator in the computation graph is executed is calculated based on the computation graph profile. The probability that an operator is executed by a branch is equal to the number of times the operator is executed by a branch divided by the number of times the conditional branch operator is executed. The number of times an operator is iterated by a loop is equal to the number of times the operator responsible for executing the next iteration is executed plus 1, divided by the number of times the operator responsible for starting the loop is executed.

[0034] In one embodiment, calculating multiple total cost functions for each operator under multiple layout formats based on information about the number of times operators in the computation graph are expected to be executed includes: multiplying the cost function of the operator in the computation graph under each of the multiple layout formats by the information about the number of times operators in the computation graph are expected to be executed, and then adding the cost functions of the other operators.

[0035] In one embodiment, the cost function includes time consumption.

[0036] Step 620: By comparing and obtaining the minimum total cost function among multiple total cost functions, the layout format corresponding to the minimum total cost function among multiple layout formats is determined to optimize the computation graph.

[0037] Next, the various embodiments according to this application and their improvement effects on the prior art will be described in detail in two cases: the case of conditional branching and the case of loop iteration.

[0038] Figure 1A This diagram illustrates a branched computation graph with layout constraint conflicts between operators A and B.

[0039] like Figure 1A As shown, there is a layout constraint conflict between operators A and B. A layout processing method in a tensor flow software involves inserting the corresponding ReorderOp operator (…). Figure 1B and 1C The R operator in TensorFlow resolves layout constraint conflicts. Note that TensorFlow is a deep learning framework that can train and run deep neural networks, thus enabling optimization and other processing of the tensor's computation graph.

[0040] Figure 1B and 1C The diagrams show two alternative solutions for resolving layout constraint conflicts by inserting the corresponding ReorderOp operator.

[0041] like Figure 1B and 1C As shown, there are two options. Figure 1B The example demonstrates how to resolve layout conflicts by converting layout format L0 to layout format L1 using the ReorderOp operator (Solution 1).

[0042] In this case, the input layout of operator A is L0, and the output layout is L0. The ReorderOp operator converts the layout format L0 to the layout format L1. Therefore, the input layout of subsequent operators B, C, and D is L1, and the output layout is L1. Operator S is a control flow operator, and the C operator channel or the D operator channel can be selected.

[0043] Figure 1C The example demonstrates how to resolve layout conflicts by converting layout format L0 to layout format L2 using the ReorderOp operator (Solution 2).

[0044] In this case, the input layout of operator A is L0, and the output layout is L0. The ReorderOp operator converts the layout format L0 to the layout format L2. As a result, the input layout of subsequent operators B, C, and D is L2, and the output layout is L2. Operator S is a control flow operator, and the C operator channel or the D operator channel can be selected.

[0045] The specific layout constraints are shown in Table 1 below:

[0046]

[0047] Table 1

[0048] Operators S and M are control flow operators, and their performance is insensitive to layout. These two operators can be disregarded when weighing performance below. Assuming that the optimization according to the embodiments of this application is not used, the time consumption of choosing Scheme 1 or Scheme 2 can be measured by the opCost function (operator cost function).

[0049] The first parameter of the opCost function is the corresponding operator, and the second parameter is the layout scheme used for the operator's calculation. The value of the opCost function is the time taken to calculate the corresponding operator using the corresponding layout scheme (specifying the input and output layouts).

[0050] Combination Figure 1B The total cost function for Option 1 is: opCost(A, {L0, L0}) + opCost(R, {L0, L1}) + opCost(B, {L1, L1}) + opCost(C, {L1, L1}) + opCost(D, {L1, L1})

[0051] This means that the total cost function includes the cost function of operator A in L0 layout operation + the cost function of operator R in L1 layout conversion + the cost function of operator B in L1 layout operation + the cost function of operator C in L1 layout operation + the cost function of operator D in L1 layout operation.

[0052] Combination Figure 1C The total cost function for Option 2 is: opCost(A, {L0, L0}) + opCost(R, {L0, L2}) + opCost(B, {L2, L2}) + opCost(C, {L2, L2}) + opCost(D, {L2, L2})

[0053] This means that the total cost function includes the cost function of operator A for L0 layout operation + the cost function of operator R for converting from L0 layout to L2 layout + the cost function of operator B for L2 layout operation + the cost function of operator C for L2 layout operation + the cost function of operator D for L2 layout operation.

[0054] To simplify the discussion, assume that operators R and B have the same performance in scheme 1 and scheme 2.

[0055] For ease of description, let's assume that the computation cost function is the computation time.

[0056] Assuming that, without considering branch probabilities, operators C and D are executed with the same probability, then if we choose option 1 (converting operator R to L1 layout), the total execution time for operators C and D is 10ms + 90ms = 100ms. If we choose option 1 (converting operator R to L1 layout), the total execution time for operators C and D is 20ms + 30ms = 50ms.

[0057] Assume that opCost(A, {L0, L0}) = 2ms, opCost(R, {L0, L1}) and opCost(R, {L0, L2}) are equal and = 2ms, and opCost(B, {L2, L2}) = 1ms.

[0058] Therefore, the total time for Scheme 1 is 2 + 2 + 1 + 10 + 90 = 105 ms. The total time for Scheme 2 is 2 + 2 + 1 + 20 + 30 = 55 ms.

[0059] In this case, option 2 would be considered to optimize the computation graph because the total time of 55ms is less.

[0060] However, this optimization method does not take into account the expected number of times branch operators C and D are executed.

[0061] The following describes an embodiment of this application that uses sampling conditional branch probabilities to guide the process in selecting a better layout scheme.

[0062] Considering branch probabilities, see [link / reference] Figure 1B and 1C Assume the probability of the S operator branching into the C operator is 90%, and the probability of the S operator branching into the D operator is 10%. Then, when calculating the total time, we need to multiply by the corresponding probabilities to obtain the expected time for the C operator and the expected time for the D operator.

[0063] In other words, the total time for Scheme 1 is 2 + 2 + 1 + 0.9 * 10 + 0.1 * 90 = 23 ms. The total time for Scheme 2 is 2 + 2 + 1 + 0.9 * 20 + 0.1 * 30 = 26 ms.

[0064] Therefore, after considering the branch probability, it was found that Scheme 1 takes less time in total.

[0065] Therefore, after considering the branch probability, Scheme 1 should be chosen to optimize the computation graph, that is, the computation graph should be optimized by converting to the L2 layout format, rather than converting to the L1 layout format.

[0066] It is evident that considering branch probabilities allows for a more accurate estimation of the expected total time for both solutions, making the optimization strategy determined in this way superior to the optimization strategy made without considering branch probabilities.

[0067] The following describes how to calculate the branch probability according to an embodiment of this application.

[0068] Figure 2 This diagram illustrates the basic information of the TensorFlow software.

[0069] like Figure 2 As shown, the TensorFlow software profile displays statistics for each input operation and their categorization by execution time. Note that the TensorFlow software profile information is obtained by tracking the performance of the TensorFlow software model using tools included with the profiler.

[0070] Each entry in the profile contains information in the following columns:

[0071] Input Operator: A type of operation in tensor flow software that displays input operations. Figure 2 The input operator shown is Interator::Prefetch::Generator.

[0072] Count: Displays the total number of instances of the computation executed during the performance profiling. For example, Figure 2 The total number of instances executed by the input operator shown is 80.

[0073] Total Time (in ms): Displays the cumulative time used by each instance. For example, Figure 2 The cumulative time taken for each instance of the input operator shown is 89,282 ms.

[0074] Total Time (%): Displays the percentage of total time spent on a single operation relative to the total input processing time. For example, Figure 2 The input operator shown spends 43.3% of the total time on a single operation as a percentage of the total input processing time.

[0075] Total Self Time (in ms) - Displays the cumulative self time used by each instance. Self time here refers to the time spent within the function body, excluding the time spent by functions it calls. For example, Figure 2 The cumulative self-time used by each instance of the input operator shown is 89,282 ms. Since this input operator does not call any other functions, its self-time and the time used are the same.

[0076] Total Self Time (%) - Displays the percentage of total self time relative to total input processing time. For example, Figure 2 The total ego time spent by the input operator on a single operation accounts for 43.3% of the total input processing time.

[0077] Category - Displays the processing category of the input operation. For example, Figure 2 The input operator shown is classified as preprocessing.

[0078] Therefore, to calculate the time consumed by each operator, the Total Time parameter in the profile information table can be used.

[0079] To calculate the branch probability of a certain operator, you can use the count parameter in the profile information table.

[0080] For example, the specific steps are as follows:

[0081] Step 1: Find all the branch operators (ConditionOp) in the computation graph: SwitchOp, and the first operator of its true branch, TrueBranchOp, and the first operator of its false branch, FalseBranchOp, forming a triplet: {ConditionOp, TrueBranchOp, FalseBranchOp}. Figure 3 Specific examples of branching operators and their branches in a computation graph are shown. For example... Figure 3 As shown, the first operator of the True branch is FusedBatchNormV3, and the first operator of the False branch is Sub. That is, the triple is {SwitchOp, FusedBatchNormV3, Sub}.

[0082] Note that in tensor flow software, each operator in the operator graph has a globally unique name, and this unique name is used in triples.

[0083] Step 2: Based on the information in the profile, count the number of times each of the three operators in the triplet is executed: the number of times the branch operator is executed (ConditionOpCount), the number of times the true branch is executed (TrueBranchOpCount), and the number of times the false branch is executed (FalseBranchOpCount). Calculate the probability that two branches are executed (called the probability of two branches).

[0084] The execution counts of the branch operator ConditionOpCount, TrueBranchOpCount (for true branches), and FalseBranchOpCount (for false branches) can all be determined by the operator's name. Figure 2 Find it in the "count" column of the brief information shown.

[0085] TrueBranchProbability = TrueBranchOpCount / ConditionOpCount

[0086] False branch probability FalseBranchProbability = FalseBranchOpCount / ConditionOpCount

[0087] Thus, the branch probabilities of the two branches and the time consumption of each operator are calculated, and then substituted into the above cost function formula to calculate the total time consumption.

[0088] Figure 4 A schematic diagram of a computation graph with loops is shown.

[0089] like Figure 4 As shown, in the computation graph of the tensor flow software, the loop begins with the Enter operator and ends with the Exit operator; the NextIter operator is responsible for executing the next iteration loop.

[0090] Assume the constraint table of the computation graph is as follows:

[0091] Operator Layout constraints A Input layout: L0, Output layout: L0 B 1) Input layout: L1, Output layout: L1; 2) Input layout: L2, Output layout: L2 gt 1) Input layout: L1, Output layout: L1; 2) Input layout: L2, Output layout: L2 add 1) Input layout: L1, Output layout: L1; 2) Input layout: L2, Output layout: L2

[0092] like Figure 4 As shown, there is a layout conflict between operator A and its downstream operators gt and add; there are two possible solutions. Inserting the corresponding ReorderOp (R operator) will convert the layout to layout format L1 or L2. Therefore, the difference between the two solutions lies in whether ReorderOp chooses to convert to layout format L1 (Solution 1) or layout format L2 (Solution 2) when resolving the conflict.

[0093] Figure 5A It shows that in such Figure 4 The diagram shown illustrates Scheme 1, in which the R operator in the computational graph with loops converts layout format L0 to layout format L1. Figure 5B It shows that in such Figure 4 The diagram shown illustrates Scheme 2, in which the R operator in the computational graph with loops converts layout format L0 to layout format L2.

[0094] To simplify the discussion, assume that the R operator has the same performance in schemes 1 and 2.

[0095] Since operator A is the same in both schemes 1 and 2, the time consumption of operator A does not need to be considered when selecting a scheme.

[0096] Assume that the time taken by the gt operator to switch from layout format L1 to layout format L1 in scheme 1 is 30ms, the time taken by the add operator to switch from layout format L1 to layout format L1 in scheme 1 is 30ms, and the time taken by the B operator to switch from layout format L1 to layout format L1 in scheme 1 is 30ms.

[0097] Assume that the gt operator takes 10ms to go from layout format L2 to layout format L2 in scheme 2, the add operator takes 10ms to go from layout format L2 to layout format L2 in scheme 2, and the B operator takes 90ms to go from layout format L2 to layout format L2 in scheme 2.

[0098] As can be seen, if we do not consider the number of iterations of each operator, and assume that the number of iterations of each operator is 1, then we choose scheme 1, because the total time of 30+30+30=90ms is less than the total time of 10+10+90=110ms of scheme 2.

[0099] However, the above scheme does not take into account the number of times the operator is expected to be executed.

[0100] The following describes an embodiment of this application that guides the process to select a better layout scheme by considering the number of loop iterations of the operator.

[0101] First, the operators gt and add will be looped in the computation graph. Considering the number of iterations, if the number of iterations iterCount for operators gt and add is 10, the total time consumption of scheme 1 can be calculated as iterCount*(opCost(gt,{L1,L1})+opCost(add,{L1,L1}))+opCost(B,{L1,L1})=10*(30+30)+30=630ms.

[0102] The total time for Scheme 2 can be calculated as follows: iterCount*(opCost(gt,{L2,L2})+opCost(add,{L2,L2}))+opCost(B,{L2,L2})=10*(10+10)+90=290ms.

[0103] It is evident that Scheme 2 should be chosen to optimize the computation graph, as its total time of 290ms is significantly less than that of Scheme 1 (630ms). Therefore, the computation graph should be optimized by converting to the L2 layout format, rather than the L1 layout format.

[0104] It is evident that considering the number of iterations allows for a more accurate estimation of the expected total time for both schemes, and the optimization strategy determined in this way is superior to the optimization strategy made without considering the number of iterations.

[0105] The number of loop iterations can also be calculated using... Figure 2 The calculation is based on the information shown in the profile. Specifically, AvgIterCount = (NextIterCount + 1) / EnterCount.

[0106] Where NextIterCount represents the number of times the NextIter operator is executed, which can be derived from... Figure 2 The EnterCount value, obtained from the count column of the NextIter operator, indicates the number of times the Enter operator has been executed. This can be found in... Figure 2 The count is obtained from the Enter operator's count column. Because the computation graph loop begins with the Enter operator and ends with the Exit operator; the NextIter operator is responsible for executing the next iteration, so in reality, for each Enter operator, the number of times the gt and add operators are executed is the first iteration plus the number of times the NextIter operator is executed, i.e., NextIterCount + 1. (NextIterCount + 1) / EnterCount can represent the number of times the gt and add operators are executed for each Enter operator.

[0107] Thus, the number of iterations of the looped operator and the time consumed by each operator are calculated, and then substituted into the above cost function formula to calculate the total time consumed.

[0108] Therefore, to optimize computational graphs, this application proposes Profile-guided optimization (PGO) technology. This technology can help obtain better-performing solutions during the optimization of the data layout format of the computational graph. In the compiler field, the PGO technology proposed in this application can improve program performance by an average of 5% to 8%.

[0109] Note that in this article, the cost function opCost is designed to calculate the cost of time consumption, but it is not limited to this. Depending on the actual cost to be concerned, the cost function can also be designed to calculate other costs, such as resource utilization, storage space, computational load, etc.

[0110] The branch probability and loop iteration count of the above operators are based on the expected number of times the operator is executed. However, the information based on the expected number of times the operator is executed is not limited to this. The number of executions, execution probability, or other information related to the number of times can also be used directly.

[0111] Thus, by utilizing procedural information and combining it with the characteristics of the network and training dataset, the layout of the computation graph of a tensor flow software is optimized to improve performance. The solution is generalizable and applicable to various networks and datasets.

[0112] Figure 7 A block diagram of a system 700 for an optimized layout format of a computation graph according to an embodiment of this application is shown.

[0113] like Figure 7 As shown, a system 700 for optimizing the layout format of a computation graph includes: a computing device 710 configured to calculate multiple total cost functions of each operator under multiple layout formats based on information about the number of times operators in the computation graph are expected to be executed; and a determining device 720 configured to determine a layout format corresponding to the minimum total cost function among the multiple layout formats by comparing and obtaining the minimum total cost function among the multiple total cost functions.

[0114] In this way, by taking into account the expected number of times operators in the computation graph are executed, multiple total cost functions of each operator under multiple layout formats can be calculated. This allows for a more accurate consideration of the actual performance impact of each operator due to the expected number of times each operator is executed, thereby determining which layout format is more optimized and using that layout format to optimize the computation graph.

[0115] In one embodiment, the information about the number of times an operator in the computation graph is expected to be executed includes the probability that the operator is executed in the case of conditional branching or the number of times the operator is iterated in the case of loop iteration.

[0116] In one embodiment, the computing device 710 is configured to: calculate the expected number of times an operator in the computation graph is executed based on the profile information of the computation graph, wherein the probability of an operator being branched is equal to the number of times the operator is branched and executed divided by the number of times the conditional branch operator is executed, and the number of times an operator is iterated in a loop is equal to the number of times the operator responsible for executing the next iteration is executed plus 1 and divided by the number of times the operator responsible for starting the loop is executed.

[0117] In one embodiment, the computing device 710 is configured to multiply the cost function of the operator in the computation graph under each of the plurality of layout formats by information on the number of times the operator in the computation graph is expected to be executed, plus the cost function of each of the other operators.

[0118] In one embodiment, the cost function includes time consumption.

[0119] Thus, by utilizing procedural information and combining it with the characteristics of the network and training dataset, the layout of the computation graph of a tensor flow software is optimized to improve performance. The solution is generalizable and applicable to various networks and datasets.

[0120] Figure 8 A block diagram of an exemplary electronic device suitable for implementing embodiments of this application is shown.

[0121] The electronic device may include a processor (H1); and a storage medium (H2) coupled to the processor (H1) and storing computer-executable instructions therein for performing the steps of various methods of the embodiments of this application when executed by the processor.

[0122] The processor (H1) may include, but is not limited to, one or more processors or microprocessors.

[0123] Storage media (H2) may include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disks, floppy disks, solid-state drives, removable disks, CD-ROMs, DVD-ROMs, Blu-ray discs, etc.).

[0124] In addition, the electronic device may also include (but is not limited to) a data bus (H3), an input / output (I / O) bus (H4), a display (H5), and input / output devices (H6) (e.g., keyboard, mouse, speaker, etc.).

[0125] The processor (H1) can communicate with external devices (H5, H6, etc.) via the I / O bus (H4) through a wired or wireless network (not shown).

[0126] The storage medium (H2) may also store at least one computer-executable instruction for performing the steps of various functions and / or methods in the embodiments described in this technology when executed by the processor (H1).

[0127] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0128] Figure 9 A schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of this application is shown.

[0129] like Figure 9As shown, the computer-readable storage medium 920 stores instructions, such as computer-readable instruction 910. When the computer-readable instruction 910 is executed by a processor, the various methods described above can be performed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, the computer-readable storage medium 920 can be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instruction 910 stored on the computer-readable storage medium 920, the various methods described above can be performed.

[0130] Of course, the specific embodiments described above are merely examples and not limitations. Those skilled in the art can combine and integrate some steps and devices from the various embodiments described separately above to achieve the effects of this application based on the concept of this application. Such combined and integrated embodiments are also included in this application, but will not be described one by one here.

[0131] Note that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of the various embodiments of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations; these details do not restrict this application from being implemented using the aforementioned specific details.

[0132] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0133] The flowcharts and method descriptions in this disclosure are merely illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the given order. As those skilled in the art will recognize, the steps in the above embodiments can be performed in any order. Words such as "then," "next," etc., are not intended to limit the order of the steps; these words are only used to guide the reader through the description of these methods. Furthermore, any reference to a singular element, such as the use of the articles "a," "one," or "the," is not to be construed as limiting that element to the singular.

[0134] Furthermore, the steps and apparatus in the various embodiments herein are not limited to any one embodiment. In fact, new embodiments can be conceived by combining relevant steps and apparatus in the various embodiments herein based on the concepts of this application, and these new embodiments are also included within the scope of this application.

[0135] The various operations of the methods described above can be performed by any suitable means capable of performing the corresponding functions. Such means may include various hardware and / or software components and / or modules, including but not limited to hardware circuits, application-specific integrated circuits (ASICs), or processors.

[0136] The various exemplified logic blocks, modules, and circuits described herein can be implemented or performed using a general-purpose processor, digital signal processor (DSP), ASIC, field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, but alternatively, it can be any commercially available processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, a microprocessor cooperating with a DSP core, or any other such configuration.

[0137] The steps of the methods or algorithms described in this disclosure can be directly embedded in hardware, in a software module executed by a processor, or a combination of both. The software module can reside in any form of tangible storage medium. Some examples of storage media that can be used include random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, etc. The storage medium can be coupled to the processor so that the processor can read information from and write information to the storage medium. Alternatively, the storage medium can be integral with the processor. The software module can be a single instruction or many instructions, and can be distributed across several different code segments, different programs, and across multiple storage media.

[0138] The methods disclosed herein include actions for implementing the described methods. The methods and / or actions may be interchanged without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions may be modified without departing from the scope of the claims.

[0139] The above functions can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored as instructions on a tangible computer-readable medium. The storage medium can be any available tangible medium that can be accessed by a computer. By way of example and not limitation, such a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital universal disc (DVD), floppy disk, and Blu-ray disc, wherein a disc typically magnetically reproduces data, while a disc optically reproduces data using lasers.

[0140] Therefore, this disclosure may also include computer program products that can perform the methods, steps, and operations given herein. For example, such a computer program product may be a computer software package, computer code instructions, or a computer-readable tangible medium having computer instructions tangibly stored (and / or encoded) thereon, which can be executed by a processor to perform the operations described herein. The computer program product may include packaging materials.

[0141] Software or instructions can also be transmitted via a transmission medium. For example, software can be transmitted from a website, server, or other remote source using transmission media such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, or microwave.

[0142] Furthermore, modules and / or other suitable means for carrying out the methods and techniques described herein can be downloaded and / or obtained by user terminals and / or base stations as appropriate. For example, such a device can be coupled to a server to facilitate the transmission of means for carrying out the methods described herein. Alternatively, the various methods described herein can be provided via storage components (e.g., RAM, ROM, physical storage media such as CDs or floppy disks) so that user terminals and / or base stations can obtain the various methods when coupled to the device or when storage components are provided to the device. Furthermore, any other suitable techniques for providing the methods and techniques described herein to the device can be utilized.

[0143] Other examples and implementations are within the scope and spirit of this disclosure and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Features implementing the functions can also be physically located in various places, including being distributed so that parts of the functions are implemented at different physical locations. Moreover, as used herein, including as used in the claims, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the described examples are preferred or better than other examples.

[0144] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0145] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0146] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for optimizing the layout format of a computation graph in the field of compilers, comprising: Based on the expected number of times operators in the computation graph are executed, multiple total cost functions for each operator under multiple layout formats are calculated. The expected number of times operators in the computation graph are executed includes the probability that the operator is executed by a branch in the case of conditional branching, or the number of times the operator is iterated in the case of loop iteration. The expected number of times operators in the computation graph are executed is calculated based on the profile information of the computation graph. The probability that an operator is executed by a branch is equal to the number of times the operator is executed by a branch divided by the number of times the conditional branch operator is executed. The number of times an operator is iterated in the loop is equal to the number of times the operator responsible for executing the next iteration is executed plus 1, divided by the number of times the operator responsible for starting the loop is executed. Calculating multiple total cost functions for each operator under multiple layout formats based on the expected number of times operators in the computation graph are executed includes multiplying the cost function of the operator in the computation graph for each layout format by the expected number of times the operator in the computation graph is executed, and then adding the cost functions of the other operators. The cost function includes at least one of time consumption, resource utilization, storage space, and computational load. By comparing and obtaining the minimum total cost function among the multiple total cost functions, the layout format corresponding to the minimum total cost function among multiple layout formats is determined to optimize the computation graph. The optimized computation graph is applied on the chip to an artificial intelligence learning task.

2. A system for optimizing the layout format of computation graphs in the compiler field, comprising: A computing device is configured to calculate multiple total cost functions for each operator under multiple layout formats based on information about the expected number of times operators in a computation graph are to be executed. The information about the expected number of times operators in the computation graph are to be executed includes the probability that the operator is branched in the case of conditional branching or the number of times the operator is iterated in the case of loop iteration. The computing device is configured to calculate the expected number of times operators in the computation graph are to be executed based on profile information of the computation graph. The probability that an operator is branched is equal to the number of times the operator is branched divided by the number of times the conditional branch operator is executed, and the number of times the operator is iterated in the loop is equal to the number of times the operator responsible for executing the next iteration is executed plus 1, divided by the number of times the operator responsible for starting the loop is executed. The determining device is configured to optimize the computation graph by comparing and obtaining the minimum total cost function among a plurality of total cost functions, thereby determining a layout format among a plurality of layout formats corresponding to the minimum total cost function, and to apply the optimized computation graph on a chip to an artificial intelligence learning task. The computing device is configured as follows: The cost function of the operator in the computation graph under each of the multiple layout formats is multiplied by the information on the expected number of times the operator in the computation graph is executed, plus the cost function of each other operator, wherein the cost function includes at least one of time consumption, resource utilization, storage space, and computational cost.

3. An electronic device, comprising: Memory, used to store instructions; A processor for reading instructions from the memory and executing the method as described in claim 1.

4. A non-temporary storage medium having instructions stored thereon, in, When the instruction is read by the processor, it causes the processor to execute the method as described in claim 1.

Images