Optimization method and device of a computation graph, electronic device, and storage medium

Abstract

An optimization method and device of a computation graph, an electronic device and a storage medium. The optimization method of the computation graph comprises: determining constraint information of a first parameter in the computation graph; performing minimum dependence pruning on the computation graph to obtain a plurality of subgraphs; and determining whether a conversion operator needs to be inserted in the plurality of subgraphs based on the constraint information of the first parameter. The optimization method of the computation graph can determine the constraint information of the first parameter before performing minimum dependence pruning on the computation graph, without depending on the specific structure of each subgraph after pruning, and can be compatible with constraint information in a specific chip, support flexible combination of operators, thereby greatly reducing the design difficulty and computation complexity of the computation graph, and improving the computation efficiency.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a method and apparatus for optimizing computational graphs, electronic devices, and storage media. Background Technology

[0002] In the field of Artificial Intelligence (AI), deep learning (DL) technology has become the foundation of cutting-edge technologies and a core driving force for many applications. Deep learning learns the inherent patterns and hierarchical representations of sample data. The information gained during this learning process greatly aids in the interpretation of data such as text, images, and sound. Its goal is to enable machines to possess analytical and learning capabilities like humans, allowing them to recognize data such as text, images, and sound. Deep learning technology enables machines to mimic human activities such as sight, hearing, and thought, solving many complex pattern recognition problems and leading to significant progress in AI-related technologies. Summary of the Invention

[0003] At least one embodiment of this disclosure provides a method for optimizing a computation graph. The method includes: determining constraint information of a first parameter in the computation graph; performing minimum dependency pruning on the computation graph to obtain multiple subgraphs; and determining, based on the constraint information of the first parameter, whether a transformation operator needs to be inserted into the multiple subgraphs.

[0004] For example, the optimization method for computational graphs provided in at least one embodiment of this disclosure further includes: recording the constraint information of the first parameter into a variable operator, wherein the variable operator is accessed synchronously by the plurality of subgraphs.

[0005] For example, at least one embodiment of the present disclosure provides a method for optimizing a computational graph, which further includes: registering a first optimization operator, wherein determining the constraint information of a first parameter in the computational graph includes: using the first optimization operator to determine the constraint information of the first parameter.

[0006] For example, the computation graph optimization method provided in at least one embodiment of this disclosure further includes: registering at least one second optimization operator; wherein, determining whether the transformation operator needs to be inserted into the plurality of subgraphs based on the constraint information of the first parameter includes: determining whether the transformation operator needs to be inserted into the plurality of subgraphs using the at least one second optimization operator based on the constraint information of the first parameter.

[0007] For example, in the computation graph optimization method provided in at least one embodiment of this disclosure, in response to the need to insert the transformation operator into the plurality of subgraphs, the method further includes: inserting the transformation operator into the plurality of subgraphs using the at least one second optimization operator to obtain a plurality of subgraphs for execution.

[0008] For example, in the optimization method of a computation graph provided in at least one embodiment of this disclosure, the computation graph includes multiple operators. Determining the constraint information of a first parameter in the computation graph includes: loading a constraint configuration table of the computation graph, wherein the constraint configuration table records the constraint information of each of the multiple operators; determining the constraint information of at least one first operator among the multiple operators based on the constraint configuration table, wherein the at least one first operator uses the first parameter; and determining the constraint information of the first parameter based on the constraint information of the at least one first operator.

[0009] For example, in the computation graph optimization method provided in at least one embodiment of this disclosure, the plurality of subgraphs includes a first subgraph, the first subgraph includes a plurality of operators, and determining whether the transformation operator needs to be inserted into the plurality of subgraphs based on the constraint information of the first parameter includes: determining whether there is a conflict between the plurality of operators of the first subgraph based on the constraint information of the first parameter and the constraint configuration table; and inserting the transformation operator between the conflicting operators in response to the conflict between the plurality of operators of the first subgraph.

[0010] For example, in the computation graph optimization method provided in at least one embodiment of this disclosure, the first subgraph includes at least one first operator using the first parameter among a plurality of operators. The first subgraph includes a training subgraph or a verification subgraph. The first operator in the training subgraph is used to train the first parameter and the input data of the computation graph. The first operator in the verification subgraph is used to verify the training result output by the training subgraph.

[0011] For example, in the computation graph optimization method provided in at least one embodiment of this disclosure, the plurality of subgraphs include a second subgraph, the second subgraph includes a plurality of operators, and determining whether the transformation operator needs to be inserted into the plurality of subgraphs based on the constraint information of the first parameter includes: determining whether there is a conflict between the plurality of operators in the second subgraph based on the constraint information of the first parameter and a constraint configuration table, wherein the constraint configuration table records the constraint information of each of the plurality of operators; and inserting the transformation operator into the second subgraph in response to the conflict between the plurality of operators in the second subgraph.

[0012] For example, in the computation graph optimization method provided in at least one embodiment of this disclosure, the second subgraph includes initializing the subgraph and determining whether there is a conflict between multiple operators of the second subgraph based on the constraint information of the first parameter and the constraint configuration table, including: determining the constraint information of a first value based on the constraint configuration table; and determining whether there is a conflict between the constraint information of the first value and the constraint information of the first parameter before assigning the first value to the first parameter.

[0013] For example, in the computation graph optimization method provided in at least one embodiment of this disclosure, in response to a conflict between multiple operators in the second subgraph, the transformation operator is inserted in the second subgraph, including: in response to a conflict between the constraint information of the first value and the constraint information of the first parameter, the transformation operator is inserted in the process of assigning the first value to the first parameter.

[0014] For example, in the optimization method of the computation graph provided in at least one embodiment of this disclosure, the second subgraph includes a saved subgraph, which is used to save the first parameter as a native parameter, the constraint information of the native parameter being native constraint information. Based on the constraint information of the first parameter and the constraint configuration table, determining whether there is a conflict between multiple operators of the second subgraph includes: determining the native constraint information based on the constraint configuration table; and determining whether there is a conflict between the native constraint information and the constraint information of the first parameter before saving the first parameter.

[0015] For example, in the computation graph optimization method provided in at least one embodiment of this disclosure, in response to a conflict between multiple operators in the second subgraph, the transformation operator is inserted into the second subgraph, including: in response to a conflict between the original constraint information and the constraint information of the first parameter, the transformation operator is inserted in the process of saving the first parameter.

[0016] For example, in the optimization method of the computation graph provided in at least one embodiment of this disclosure, the second subgraph includes a loading subgraph, which is used to load the native parameter as the first parameter. The constraint information of the native parameter is native constraint information. Based on the constraint information of the first parameter and the constraint configuration table, determining whether there is a conflict between multiple operators of the second subgraph includes: determining the native constraint information based on the constraint configuration table; and determining whether there is a conflict between the native constraint information and the constraint information of the first parameter before loading the native parameter as the first parameter.

[0017] For example, in the computation graph optimization method provided in at least one embodiment of this disclosure, in response to a conflict between multiple operators in the second subgraph, the transformation operator is inserted in the second subgraph, including: in response to a conflict between the native constraint information and the constraint information of the first parameter, the transformation operator is inserted in the process of loading the native parameter as the first parameter.

[0018] At least one embodiment of this disclosure also provides an optimization apparatus for a computation graph, the optimization apparatus comprising: an optimization module configured to determine constraint information of a first parameter in the computation graph; a pruning module configured to perform minimum dependency pruning on the computation graph to obtain multiple subgraphs; the optimization module is further configured to determine, based on the constraint information of the first parameter, whether a transformation operator needs to be inserted into the multiple subgraphs.

[0019] For example, the computational graph optimization apparatus provided in at least one embodiment of this disclosure further includes a registration module configured to register a first optimization operator, and the optimization module is further configured to use the first optimization operator to determine the constraint information of the first parameter.

[0020] For example, in the computation graph optimization apparatus provided in at least one embodiment of this disclosure, the registration module is further configured to register at least one second optimization operator; the optimization module is further configured to determine, based on the constraint information of the first parameter, whether the transformation operator needs to be inserted into the plurality of subgraphs using the at least one second optimization operator.

[0021] At least one embodiment of this disclosure also provides an electronic device. The electronic device includes: a processor; and a memory including one or more computer program modules; wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, the one or more computer program modules being used to implement the computational graph optimization method provided in any embodiment of this disclosure.

[0022] At least one embodiment of this disclosure also provides a storage medium storing non-transitory computer-readable instructions that, when executed by a computer, implement the computation graph optimization method provided in any embodiment of this disclosure. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure, and are not intended to limit this disclosure.

[0024] Figure 1A This is a schematic diagram of a computational graph optimization method.

[0025] Figure 1BThis is a schematic diagram of a method for clipping a computational graph;

[0026] Figure 2 An exemplary flowchart illustrating a computation graph optimization method provided in at least one embodiment of this disclosure;

[0027] Figure 3 A schematic diagram illustrating an example of a first optimization operator and a second optimization operator provided for at least one embodiment of this disclosure;

[0028] Figure 4 for Figure 2 An exemplary flowchart of an example of step S10;

[0029] Figure 5 A schematic diagram illustrating an example of a partial computational graph provided for at least one embodiment of this disclosure;

[0030] Figure 6 A schematic diagram illustrating an example of a computational graph optimization method provided in at least one embodiment of this disclosure;

[0031] Figure 7 A schematic diagram illustrating another example of a computational graph optimization method provided in at least one embodiment of this disclosure;

[0032] Figure 8 A schematic diagram illustrating yet another example of a computational graph optimization method provided in at least one embodiment of this disclosure;

[0033] Figure 9 A schematic diagram illustrating yet another example of a computational graph optimization method provided in at least one embodiment of this disclosure;

[0034] Figure 10 A schematic diagram illustrating yet another example of a computational graph optimization method provided in at least one embodiment of this disclosure;

[0035] Figure 11 A schematic block diagram of a computational graph optimization apparatus provided for at least one embodiment of the present disclosure;

[0036] Figure 12 A schematic block diagram of an electronic device provided for at least one embodiment of this disclosure;

[0037] Figure 13 A schematic block diagram of another electronic device provided for at least one embodiment of this disclosure; and

[0038] Figure 14 This is a schematic diagram of a storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0040] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0041] The present disclosure will now be described through several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and components may be omitted. When any component of the embodiments of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numerals in each drawing.

[0042] Deep learning computation tasks consist of multiple computational units, each of which can be called an operator (Op). For example, in a neural network model, an operator corresponds to the computational logic of a layer or node; for instance, the convolution computation task in a convolutional layer can be an operator, and the weight summation computation task in a fully connected layer can also be an operator. In the specific implementation of a neural network, operators can be implemented in software (e.g., through computer programs) or in hardware (e.g., through circuits or FPGAs).

[0043] In performing computational tasks such as deep learning, a computational graph can be constructed using operators and required parameters. The actual computational task is then performed by running the computational graph. Optimizing the computational graph is a method to improve the execution efficiency of deep learning computational tasks. For example, various optimization operators (such as the Pass operation, which involves traversing the computational graph and performing necessary optimizations) can be used to perform a series of optimization processes on the computational graph. These optimization processes are integrated throughout different stages of the computational graph processing.

[0044] Figure 1A This is a schematic diagram of a computational graph optimization method. Figure 1B This is a schematic diagram of a method for clipping a computational graph.

[0045] For example, such as Figure 1A As shown, a series of processes need to be performed on the computation graph during the creation process, including placement, rewrite, partitioning, and graph optimization. Depending on the different stages of computation graph processing, different optimization operators (e.g., optimization operators 1 to 6) can be used to optimize the computation graph in batches.

[0046] For example, such as Figure 1A As shown, depending on the different computational graph processing stages, different optimization operators can be registered at different effective times, and optimization operators 1 through 6 can contain different types. For example, in the pre-layout stage (PRE_PLACEMENT), optimization operator 1 is registered and run; in the post-layout stage (POST_PLACEMENT), optimization operator 2 is registered and run; in the pre-rewrite stage (PRE_REWRITE_FOR_EXE), optimization operator 4 is registered and run; and in the post-partition stage (POST_PARTITION), optimization operator 5 is registered and run. For another example, optimization operator 3 could be an operator for full-graph optimization (e.g., Grappler), and optimization operator 6 could be an operator for subgraph optimization after partitioning (e.g., GraphOptimizer).

[0047] For example, the above partitioning operation can include a pruning operation. To perform a computational task, after the computation graph is built, the execution process begins (e.g., running the `Session.run` function in the Tensorflow framework). For instance, during execution, the full computation graph can be pruned according to dependencies, resulting in a minimal dependency subgraph (Client Graph); this pruning process is also known as "minimum dependency pruning," and the resulting minimal dependency subgraph is the graph actually executed during the process.

[0048] For example, such as Figure 1B The diagram shows a portion of a computation graph, which includes multiple operators (including but not limited to operators 1-5) and multiple parameters (including but not limited to parameters a-f). For example, if there is a minimum dependency between operators 1-5 and parameters a, b, d, and f, then the portion of the computation graph including operators 1-5 and parameters a, b, d, and f can be pruned into a minimum dependency subgraph (i.e.,...). Figure 1B The circled part is used to implement specific functions in actual execution (e.g., forward computation, backward computation, and parameter training, saving, loading, etc.).

[0049] For example, the minimum dependency subgraph can include an initialization graph, a training graph, a validation graph, a save graph, and a restore graph. The initialization graph initializes the parameters in the computational model; the training graph is used for forward propagation, backpropagation, loss calculation, and optimizer operations; the validation graph validates the training results; the save graph saves the parameters in their native state; and the restore graph restores the saved parameters. For instance, after obtaining the minimum dependency subgraph, pre-defined native optimization operators (e.g., ...) can be used in the program. Figure 1A The optimization operators (such as 1 to 6) optimize the computation graph at preset activation times.

[0050] For example, each operator or parameter in the computation graph has specific constraint information, such as data format and memory layout. For instance, data with the format [N,C,H,W] can include N samples, each corresponding to multiple channels, where C represents the number of channels. The data format can also include [N,H,W,C], [C,N,H,W], etc. For example, in each channel, multiple data elements are arranged in a two-dimensional array. The height of the data in each channel in the vertical dimension is H, where H represents the number of data elements along the column direction of the array, and the width of the data in each channel in the horizontal dimension is W, where W represents the number of data elements along the row direction of the array.

[0051] For example, with Figure 1ATaking one of the optimization operators shown as an example, when the optimization operator traverses the computation graph at runtime, if it finds that the format of the input data to a certain operator is [N,H,W,C] and does not match the format [N,C,H,W] of the operator, it will insert a reorder op before the operator to convert the format of the input data from [N,H,W,C] to the required [N,C,H,W] so that the operator can obtain better computational performance.

[0052] For example, operators or parameters running on a specific chip may have specific constraint information (e.g., layout or data format). The native optimization operator may not be able to recognize this specific constraint information, thus failing to optimize the performance of the computation graph on that specific chip. If a new optimization operator is registered when the native optimization operator takes effect, the new optimization operator can perform constraint information transformation on the input data of the computation graph, but it is difficult to handle the transformation of constraint information for specific parameters (e.g., weight parameters, bias parameters, etc.).

[0053] For example, consider the weight parameter. The weight parameter is used by operators during forward propagation; during backpropagation, it is also used in the chain rule to calculate the corresponding gradient; and during optimization, the weight parameter is optimized based on the gradient. During forward propagation, backpropagation, or optimization, when an operator uses the weight parameter, the constraint information of that operator needs to match the constraint information of the weight parameter. However, the initialization of the weight parameter occurs in the initialization subgraph. Since the initialization subgraph only contains the operator that initializes the weight parameter, not the operator that uses it, the actual constraint information of the weight parameter in the computation graph cannot be determined during initialization. Therefore, the weight parameter is first initialized with its native constraint information, and then during forward propagation, it is transformed to the required constraint information. This synchronization process of different constraint information for the weight parameter significantly increases the design difficulty of the computation graph and the complexity of the actual computation.

[0054] At least one embodiment of this disclosure provides a method for optimizing a computation graph. The method includes: determining constraint information of a first parameter in the computation graph; performing minimum dependency pruning on the computation graph to obtain multiple subgraphs; and determining, based on the constraint information of the first parameter, whether a transformation operator needs to be inserted into the multiple subgraphs.

[0055] At least one embodiment of this disclosure also provides a computational graph optimization apparatus, electronic device, and storage medium for implementing the computational graph optimization method of the above embodiments.

[0056] The method, apparatus, electronic device, and storage medium provided in at least one embodiment of this disclosure can determine the constraint information of the first parameter before performing minimal dependency pruning on the computation graph, without depending on the specific structure of each subgraph after pruning. Furthermore, it can be compatible with the constraint information in a specific chip and supports the flexible combination of operators, thereby greatly reducing the design difficulty and computational complexity of the computation graph and improving computational efficiency.

[0057] At least one embodiment of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the same reference numerals will be used to refer to the same elements described in different drawings.

[0058] Figure 2 An exemplary flowchart of a computational graph optimization method provided in at least one embodiment of this disclosure.

[0059] For example, such as Figure 2 As shown, at least one embodiment of this disclosure provides a method for optimizing a computational graph, which may include the following steps S10 to S30.

[0060] Step S10: Determine the constraint information of the first parameter in the computation graph;

[0061] Step S20: Perform minimum dependency pruning on the computation graph to obtain multiple subgraphs;

[0062] Step S30: Based on the constraint information of the first parameter, determine whether it is necessary to insert transformation operators in multiple subgraphs.

[0063] For example, in step S10, the computation graph can be used to perform computational tasks such as deep learning. The computation graph may include multiple operators and multiple parameters. For example, the operators or parameters in the computation graph may include various types, and the embodiments of this disclosure do not limit this. In the embodiments of this disclosure, the number of first parameters can be one or more, and one or more first parameters can be used by one or more corresponding operators in the computation graph. For example, the first parameter can be a weight parameter, bias parameter, etc. in a neural network. The embodiments of this disclosure do not limit the type or use of the computation graph or the first parameter.

[0064] For example, constraint information includes data format and memory layout. Data format may include, for example, [N,H,W,C], [N,C,H,W], or YUV (Y represents the luminance component, U and V represent the chrominance components), RGB (R represents the red component, G represents the green component, B represents the blue component), etc. The memory layout may include the data arrangement order (e.g., linear arrangement, column-major arrangement, row-major arrangement) and the stored data dimension (e.g., four-dimensional tensor data with the format [N,C,H,W], having N samples, an output channel KC_OUT, an input channel KC_IN, and three-dimensional convolution weights of a filter with dimensions K_W*K_H, or one-dimensional vector data with only a length N parameter). The embodiments of this disclosure do not limit the type or purpose of constraint information.

[0065] For example, in step S20, minimal dependency pruning can be performed on the full computation graph according to dependencies (e.g., such as...). Figure 1B The minimal dependency pruning shown above yields a minimal dependency subgraph (Client Graph), which is then used to perform computational tasks by executing the individual minimal dependency subgraphs during the execution process (e.g., running the Session.run function in the Tensorflow framework).

[0066] For example, in step S30, for multiple subgraphs containing the first parameter, based on the constraint information of the first parameter determined in step S10, it can be determined whether the constraint information of the operators or parameters related to the first parameter in the subgraphs matches. If the constraint information does not match, the corresponding transformation operator is inserted in the corresponding process so that the related operators or parameters can obtain better computational performance.

[0067] In the computation graph optimization method provided in at least one embodiment of this disclosure, taking the initialization of a subgraph as an example, after the above steps S10 to S30, the actual constraint information of the first parameter in the computation graph can be determined when it is initialized, without having to initialize it as the original constraint information first, and then convert it into the required constraint information in other subgraphs. This greatly reduces the design difficulty and computational complexity of the computation graph and improves computational efficiency.

[0068] It should be noted that, apart from initializing the subgraph, the computation graph optimization method provided in at least one embodiment of this disclosure is applicable to multiple subgraphs obtained by minimum dependency pruning. The embodiments of this disclosure do not limit the method of minimum dependency pruning, the type and number of subgraphs obtained, etc.

[0069] In some examples, the computational graph optimization method provided in at least one embodiment of this disclosure may further include: recording constraint information of the first parameter into a variable operator. For example, the variable operator can be accessed synchronously by multiple subgraphs.

[0070] For example, in Figure 2 Before step S20, the constraint information of the first parameter can be recorded in the variable operator by adding an attribute. After the minimum dependency pruning, some subgraphs in multiple subgraphs (e.g., initializing subgraphs, saving subgraphs, loading subgraphs, etc.) contain variable operators, and the remaining subgraphs (e.g., training subgraphs, validation subgraphs, etc.) contain operators using the first parameter, so that the constraint information of the first parameter is contained in multiple subgraphs.

[0071] For example, the constraint information of the first parameter is recorded in the variable operator. As a global parameter, the variable operator can be accessed synchronously by multiple subgraphs. Therefore, each subgraph can obtain the actual constraint information of the first parameter in the computation graph without first setting it as the native constraint information and then converting it into the required constraint information in the corresponding subgraph. This greatly reduces the design difficulty and computational complexity of the computation graph and improves computational efficiency.

[0072] In some examples, the computational graph optimization method provided in at least one embodiment of this disclosure may further include: registering a first optimization operator. For example, Figure 2 Step S10 may include: using the first optimization operator to determine the constraint information of the first parameter.

[0073] In some examples, the computational graph optimization method provided in at least one embodiment of this disclosure may further include: registering at least one second optimization operator. For example, Figure 2 Step S30 may include: determining, based on the constraint information of the first parameter, whether a transformation operator needs to be inserted in multiple subgraphs using at least one second optimization operator.

[0074] For example, in response to the need to insert transformation operators into multiple subgraphs, the computation graph optimization method provided in at least one embodiment of this disclosure may further include: inserting transformation operators into multiple subgraphs using at least one second optimization operator to obtain multiple subgraphs for execution.

[0075] Figure 3 This is a schematic diagram illustrating an example of a first optimization operator and a second optimization operator provided for at least one embodiment of this disclosure. For example, Figure 3 for Figure 2 This is a specific example of an optimization method for the computational graph shown.

[0076] For example, such as Figure 3As shown, when the computation graph enters the execution phase (e.g., running the `Session.run` function in the Tensorflow framework), before minimal dependency pruning, an optimization operator's activation time (e.g., `PRE_PRUNE`) can be added, and then the first optimization operator can be registered in the `PRE_PRUNE` phase. For example, in Figure 2 In step S10, by running the first optimization operator, the information of the complete computation graph can be analyzed. Specifically, by analyzing the upstream or downstream operators of the first parameter in the computation graph, the constraint information of the first parameter can be determined (e.g., what layout or data format the first parameter should be initialized to).

[0077] For example, such as Figure 3 As shown, after the first optimization operator has finished running, the native optimization operators in the running process (e.g., such as...) Figure 1A The optimization operators 1 to 6 shown are used to achieve the following: Figure 2 The minimum dependency pruning in step S20 and the corresponding native optimization process result in multiple subgraphs.

[0078] For example, such as Figure 3 As shown, after all the original optimization operators have been executed, a second optimization operator is added. The second optimization operators 1 to 4 can each correspond to different subgraphs. For example, if step S30 determines that transformation operators need to be inserted into multiple subgraphs, the second optimization operators 1 to 4 can be run according to the constraint information of the first parameter determined in step S10. This inserts the corresponding transformation operators into the corresponding processes according to the execution order of the subgraphs, so that the constraint information between the operators or parameters related to the first parameter in each subgraph matches.

[0079] It should be noted that the number of second optimization operators can be one or more, and the number of second optimization operators can be selected according to actual needs. The embodiments of this disclosure do not limit this.

[0080] Figure 4 for Figure 2 An exemplary flowchart of an example of step S10.

[0081] For example, a computation graph can include multiple operators; such as Figure 4 As shown, Figure 2 Step S10 may further include the following steps S101 to S103:

[0082] Step S101: Load the constraint configuration table of the computation graph;

[0083] Step S102: Based on the constraint configuration table, determine the constraint information of at least one first operator among multiple operators;

[0084] Step S103: Determine the constraint information of the first parameter based on the constraint information of at least one first operator.

[0085] For example, in step S101, the constraint configuration table records the constraint information of each operator, specifically, including constraint information such as the data input to the operator and the data output to the operator (e.g., the layout or format of the data in memory).

[0086] For example, in step S102, the number of first operators can be one or more; in the computation graph or corresponding subgraph, each first operator uses a first parameter (e.g. Figure 1B Operator 4 shown uses parameter f. For example, constraint information for each first operator can be obtained by reading the constraint configuration table.

[0087] For example, in step S103, since the first operator uses the first parameter (that is, the first operator is the upstream or downstream operator of the first parameter in the computation graph), the constraint information of the first parameter used by the first operator can be determined by analyzing the constraint information of the first operator.

[0088] Figure 5 This is a schematic diagram illustrating an example of a partial computational graph provided for at least one embodiment of the present disclosure. For example, Figure 5 for Figure 2 A specific example of a portion of the computation graph.

[0089] For example, such as Figure 5 The computation graph shown includes operators A, B, and C. The constraint configuration table in Table 1 contains a partial example of the constraint information for operators A, B, and C. For example, as shown... Figure 5 As shown, the output of operator A is the input of operator B, and the output of operator B is the input of operator C. For example, when the constraint information of the output of operator A does not match the constraint information of the input of operator B, a constraint conflict will occur between operator A and operator B; when the constraint information of the output of operator B does not match the constraint information of the input of operator C, a constraint conflict will occur between operator B and operator C.

[0090] For example, as shown in Table 1, for the constraint information of each operator, the input of operator A is L0 and the output is L0, the input of operator B is L1 and the output is L1 (or the input is L2 and the output is L2), and the input of operator C is L2 and the output is L2 (or the input is L3 and the output is L3). That is, there is an unavoidable constraint conflict between operator A and operator B; however, when the input of operator B is L2 and the output is L2, and the input of operator C is L2 and the output is L2, the conflict between operator B and operator C can be avoided.

[0091] Table 1 shows some examples of constraint configuration tables.

[0092]

[0093] For example, such as Figure 5 As shown, since there is an unavoidable constraint conflict between operator A and operator B, a transformation operator R needs to be inserted between operator A and operator B; whether there is a conflict between operator B and operator C depends on what kind of transformation the constraint information of operator A is performed by the inserted transformation operator R when resolving the conflict between operator A and operator B.

[0094] For example, such as Figure 5 As shown, in Example 1, because the conversion operator R inserted between operators A and B converts the output of operator A to L2, the output L2 of operator B can match the input L2 of operator C, so there is no need to insert a conversion operator between operators B and C. In Example 2, because the conversion operator R inserted between operators A and B converts the output of operator A to L2, the output L2 of operator B does not match the input L3 of operator C, so a conversion operator R needs to be inserted between operators B and C to convert the output of operator B to L3. In Example 3... Since the conversion operator R inserted between operators A and B converts the output of operator A into L1, the output L1 of operator B does not match the input L2 of operator C. Therefore, a conversion operator R needs to be inserted between operators B and C to convert the output of operator B into L2. In Example 4, since the conversion operator R inserted between operators A and B converts the output of operator A into L1, the output L1 of operator B does not match the input L3 of operator C. Therefore, a conversion operator R needs to be inserted between operators B and C to convert the output of operator B into L3.

[0095] It should be noted that, Figure 5 The partial calculation diagram shown and the constraint configuration table and the constraint information recorded in Table 1 are only examples. The specific contents of the calculation diagram, constraint configuration table and constraint information can be selected according to actual needs, and the embodiments of this disclosure do not limit this.

[0096] In some examples, Figure 2The multiple subgraphs obtained in step S20 may include a first subgraph, which includes multiple operators; the multiple operators of the first subgraph may include at least one first operator using a first parameter. For example, subgraphs other than those processed by initialization, saving, and restoring can all be used as the first subgraph. For example, the first subgraph may include a training subgraph or a validation subgraph. The first operator in the training subgraph is used to train the first parameter and the input data of the computation graph (e.g., the training subgraph includes subgraphs for forward propagation, backpropagation, loss calculation, optimization, etc.), and the first operator in the validation subgraph is used to validate the training results output by the training subgraph. It should be noted that, in addition to the first subgraph described above, the first subgraph may also include other types of subgraphs as needed, and the embodiments of this disclosure do not limit this.

[0097] In some examples, Figure 2 The multiple subgraphs obtained in step S20 may further include a second subgraph, which includes multiple operators. The second subgraph may include an initialization subgraph, a saving subgraph, or a loading subgraph, etc. For example, the initialization subgraph can be used to initialize the first parameter, the saving subgraph can be used to save the first parameter in its native mode, and the loading subgraph can be used to restore the first parameter from its native mode to its current state. It should be noted that, in addition to the second subgraph described above, the second subgraph may also include other types of subgraphs as needed, and the embodiments of this disclosure do not impose limitations on this.

[0098] Figure 6 This is a schematic diagram illustrating an example of a computational graph optimization method provided in at least one embodiment of the present disclosure. For example, Figure 6 The first and second subgraphs in the graph are Figure 2 A specific example of the sub-image obtained by cropping in step S20.

[0099] For example, such as Figure 6 As shown, taking the first subgraph as an example of a forward subgraph, the first subgraph includes a first operator 1 and a first operator 2. First operator 1 uses a first parameter 1, and first operator 2 uses a first parameter 2. Input data and the first parameter 1 to be trained are input to first operator 1. The output of first operator 1 and the first parameter 2 to be trained are the inputs to first operator 2. The output of first operator 2 is the input to other operators. For example, first parameter 1 and first parameter 2 are continuously updated during training. In some specific examples, first operator 1, first operator 2, and other operators can be Matmul (matrix multiplication), Add (addition), and ReLU (activation function) operators, respectively, and first parameter 1 and first parameter 2 can be trainable weights and biases, respectively.

[0100] For example, such as Figure 6As shown, taking the second subgraph as an example of an initialization subgraph, the second subgraph includes variable operator 1 and variable operator 2, which are used to record the constraint information of the first parameter 1 and the first parameter 2, respectively. The second subgraph also includes second operator 1 and second operator 2, which are used to assign the first value 1 and the first value 2 to the first parameter 1 and the first parameter 2, respectively, and the assignment information is stored in variable operator 1 and variable operator 2, respectively. The second subgraph also includes node operators, used to derive minimum dependency pruning to prune the second subgraph to the required portion. In some specific examples, second operator 1 and second operator 2 can be assignment operators (Assign) corresponding to the first parameter 1 and the first parameter 2, respectively, and the first value 1 and the first value 2 can be the initial values ​​(Const) corresponding to the first parameter 1 and the first parameter 2, respectively.

[0101] For example, such as Figure 6 As shown, the data of the first parameter 1 and the first parameter 2 in the first subgraph can be equivalent to the data in variable operator 1 and variable operator 2 in the second subgraph (that is, they are the same data); this data includes, for example, the constraint information and specific assignments of the first parameter 1 and the first parameter 2, etc., and the embodiments of this disclosure do not limit this. For example, after the second subgraph completes the initialization (e.g., assignment operation) of the first parameter 1 and the first parameter 2, the data in variable operator 1 and variable operator 2 will change, and at this time the data of the first parameter 1 and the first parameter 2 in the first subgraph will also change synchronously. Furthermore, variable operator 1 and variable operator 2, as global parameters, will also be accessed synchronously by multiple other subgraphs besides the first subgraph.

[0102] For example, in order to Figure 2 In step S10, the constraint information of the first parameter 1 and the first parameter 2 is obtained, and an activation time can be added before the minimum dependency pruning (e.g. Figure 3 The PRE_PRUNE phase is shown, and then the first optimization operator is registered in the PRE_PRUNE phase. The first optimization operator can be called StatefullOpLayoutInferPass. For example, by running the first optimization operator, you can... Figure 4 In step S101, the constraint configuration table of the computation graph is loaded; this can be done... Figure 4 In step S102, the constraint information of the first operator 1 and the first operator 2 is determined by reading the constraint configuration table; this can be achieved by... Figure 4 In step S103, the constraint information of the first parameter 1 and the first parameter 2 is determined by analyzing the constraint information of the first operator 1 and the first operator 2.

[0103] In some examples, for the first subgraph, Figure 2 Step S30 may further include the following steps S301 to S302:

[0104] Step S301: Based on the constraint information of the first parameter and the constraint configuration table, determine whether there is a conflict between multiple operators of the first subgraph;

[0105] Step S302: In response to a conflict between multiple operators in the first subgraph, a transformation operator is inserted in the first subgraph.

[0106] For example, the constraint configuration table records the constraint information of each operator in the first subgraph. For example, in step S301, based on Figure 4 The constraint information of the first parameter determined in steps S101 to S103 and the constraint configuration table of the entire computation graph can determine whether there are conflicts between the constraint information of multiple operators in the first subgraph. For example, in step S302, if there are conflicts between the constraint information of multiple operators in the first subgraph, a transformation operator can be inserted between the conflicting operators to transform the constraint information of the conflicting operators to a matching one.

[0107] Figure 7 This is a schematic diagram of another example of a computational graph optimization method provided in at least one embodiment of this disclosure. For example, Figure 7 Is Figure 6 Execution in the first subgraph Figure 2 A specific example of step S30.

[0108] For example, with Figure 7 Taking the first subgraph shown as a forward subgraph as an example, after all the native optimization operators have been executed, a second optimization operator corresponding to the first subgraph can be added. This second optimization operator can be called CommGraphLayoutInferPass (for example, any part of the training subgraph, validation subgraph, or computation graph that includes the forward subgraph can be called a Common Graph); for example, this second optimization operator can be... Figure 3 The second optimization operator 1 in the process.

[0109] For example, such as Figure 7 As shown, in step S301, based on Figure 4 The constraint information of the first parameter 1 and the first parameter 2 determined in steps S101 to S103 and the constraint configuration table of the entire computation graph can determine whether there is a conflict between the constraint information of the first operator 1 and the first operator 2 of the first subgraph. In step S302, if there is a conflict between the constraint information of multiple operators in the first subgraph, a transformation operator can be inserted between the conflicting first operator 1 and the first operator 2 by running the second optimization operator 1, so as to transform the constraint information of the first operator 1 and the first operator 2 to match.

[0110] It should be noted that, except for Figure 7For first subgraphs of other types besides the forward subgraph (e.g., training or validation subgraphs used for backpropagation, loss calculation, etc.), transformation operators can be inserted between conflicting operators according to steps S301-S302. For details, please refer to [link to relevant documentation]. Figure 7 The description will not be repeated here.

[0111] In some examples, for the second subgraph, Figure 2 Step S30 may further include the following steps S310 to S320:

[0112] Step S310: Based on the constraint information of the first parameter and the constraint configuration table, determine whether there is a conflict between multiple operators in the second subgraph;

[0113] Step S320: In response to a conflict between multiple operators in the second subgraph, a transformation operator is inserted in the second subgraph.

[0114] For example, the constraint configuration table records the constraint information for each operator in the second subgraph. For example, in step S310, based on Figure 4 The constraint information of the first parameter determined in steps S101 to S103 and the constraint configuration table of the entire computation graph can determine whether there are conflicts between the constraint information of multiple operators in the second subgraph. For example, in step S320, if there are conflicts between the constraint information of multiple operators in the second subgraph, a transformation operator can be inserted between the conflicting operators to transform the constraint information of the conflicting operators to a matching one.

[0115] In some examples, the second subgraph may include an initialization subgraph. For example, step S310 may further include: determining the constraint information of the first value based on the constraint configuration table; and determining whether there is a conflict between the constraint information of the first value and the constraint information of the first parameter before assigning the first value to the first parameter. For example, step S320 may further include: inserting a transformation operator in the process of assigning the first value to the first parameter in response to a conflict between the constraint information of the first value and the constraint information of the first parameter.

[0116] Figure 8 This is a schematic diagram of yet another example of a computational graph optimization method provided in at least one embodiment of the present disclosure. For example, Figure 8 Is Figure 6 Execute in the second subgraph (initialization subgraph) Figure 2 A specific example of step S30.

[0117] For example, with Figure 8The second subgraph shown is an example of an initialization subgraph. After all the native optimization operators have been executed, a new second optimization operator corresponding to the initialization subgraph can be added. This second optimization operator corresponding to the initialization subgraph (InitializeGraph) can be called IniGraphLayoutInferPass. For example, this second optimization operator can be... Figure 3 The second optimization operator 2 in the process.

[0118] For example, such as Figure 8 As shown, the second operator 1 and the second operator 2 can be assignment operators (Assign) corresponding to the first parameter 1 and the first parameter 2, respectively, and the first value 1 and the first value 2 can be the initial values ​​(Const) corresponding to the first parameter 1 and the first parameter 2, respectively. For example, the second operator 1 and the second operator 2 are used to assign the first value 1 and the first value 2 to the first parameter 1 and the first parameter 2, respectively.

[0119] For example, such as Figure 8 As shown, taking the process of assigning the first value 1 to the first parameter 1 by the second operator 1 as an example, the constraint information of the first value 1 can be determined based on the constraint configuration table; for example, since the constraint information of the first parameter 1 is recorded in the variable operator 1, the second operator 1 can obtain the constraint information of the first parameter 1. For example, before assigning the first value 1 to the first parameter, it is determined whether there is a conflict between the constraint information of the first value 1 and the constraint information of the first parameter 1.

[0120] For example, such as Figure 8 As shown, in the process of assigning the first value 1 to the first parameter 1, if there is no conflict between the constraint information of the first value 1 and the constraint information of the first parameter 1, then the second operator 1 follows... Figure 8 The dashed line directly assigns the first value 1 to the first parameter 1; if there is a conflict between the constraint information of the first value 1 and the constraint information of the first parameter 1, a transformation operator is inserted between the first value 1 and the second operator 1.

[0121] In some examples, the second subgraph may also include a saving subgraph, which is used to save the first parameter as a native parameter, the constraint information of which is the native constraint information. For example, step S310 may further include: determining the native constraint information based on the constraint configuration table; and determining whether there is a conflict between the native constraint information and the constraint information of the first parameter before saving the first parameter. For example, step S320 may further include: inserting a transformation operator in the process of saving the first parameter in response to a conflict between the native constraint information and the constraint information of the first parameter.

[0122] Figure 9 This is a schematic diagram illustrating yet another example of a computational graph optimization method provided in at least one embodiment of the present disclosure. For example, Figure 9Executing in the saved subgraph Figure 2 A specific example of step S30.

[0123] For example, with Figure 9 The second subgraph shown is an example of a saved subgraph. After all the native optimization operators have been executed, a new second optimization operator corresponding to the saved subgraph can be added. This second optimization operator corresponding to the saved subgraph can be called SaveGraphLayoutInferPass. For example, this second optimization operator can be... Figure 3 The second optimization operator 3 in the process.

[0124] For example, such as Figure 9 As shown, variable operator 1, variable operator 2, and variable operator 3 respectively record constraint information related to the first parameter 1, first parameter 2, and first parameter 3. For example, this save subgraph is used to save the first parameter 1, first parameter 2, or first parameter 3 as the original parameters in the save operator (Save), that is, to save the constraint information of the first parameter 1, first parameter 2, or first parameter 3 as the original constraint information.

[0125] For example, such as Figure 9 As shown, taking the process of saving the first parameter 1 as an example, the original constraint information in the operator to be saved can be determined based on the constraint configuration table; since the first parameter 1 is recorded in variable operator 1, before saving the first parameter 1, it can be determined whether there is a conflict between the original constraint information and the constraint information of the first parameter 1.

[0126] For example, such as Figure 9 As shown, in the process of saving the first parameter 1, if there is no conflict between the original constraint information and the constraint information of the first parameter 1, then it follows... Figure 9 The dashed line directly saves the first parameter 1 as the native parameter, that is, saves the constraint information of the first parameter 1 as the native constraint information; if there is a conflict between the native constraint information and the constraint information of the first parameter 1, a transformation operator is inserted in the process of saving the first parameter 1.

[0127] For example, such as Figure 9 As shown, when saving the first parameter 1, there may be cross-device transmission. The first parameter 1 can be sent to an external device first using the sending operator to perform the required processing, and then the first parameter 1 can be received and saved as the original parameter using the receiving operator.

[0128] In some examples, the second subgraph may further include a loading subgraph used to load the native parameter as the first parameter, the constraint information of which is the native constraint information. For example, step S310 may further include: determining the native constraint information based on the constraint configuration table; and determining whether there is a conflict between the native constraint information and the constraint information of the first parameter before loading the native parameter as the first parameter. For example, step S320 may further include: in response to a conflict between the native constraint information and the constraint information of the first parameter, inserting a transformation operator in the process of loading the native parameter as the first parameter.

[0129] Figure 10 This is a schematic diagram illustrating yet another example of a computational graph optimization method provided in at least one embodiment of the present disclosure. For example, Figure 10 Executed in the loading subgraph Figure 2 A specific example of step S30.

[0130] For example, with Figure 10 Taking the loading subgraph as an example, after all the native optimization operators have been executed, a new second optimization operator corresponding to the loading subgraph can be added. This second optimization operator corresponding to the loading subgraph (Restore Graph) can be called RestoreGraphLayoutInferPass. For example, this second optimization operator could be... Figure 3 The second optimization operator 4 in the process.

[0131] For example, such as Figure 10 As shown, the second operators 1, 2, and 3 are used to load the original parameters as the first parameter 1, 2, and 3, respectively. That is, they load the original constraint information of the original parameters as the constraint information of the first parameter 1, 2, or 3. For example, since variable operators 1, 2, and 3 record constraint information related to the first parameter 1, 2, and 3, respectively, the second operators 1, 2, and 3 can obtain the constraint information of the first parameter 1, 2, and 3, respectively. For example, the loaded subgraph also includes node operators for deriving minimum dependency pruning to prune the loaded subgraph to the desired portion.

[0132] For example, such as Figure 10 As shown, taking the process of loading the native parameter as the first parameter 1 as an example, the native constraint information is determined based on the constraint configuration table; since the constraint information of the first parameter 1 is recorded in the variable operator 1, it can be determined whether there is a conflict between the native constraint information and the constraint information of the first parameter 1 before loading the native parameter as the first parameter 1.

[0133] For example, such as Figure 10As shown, in the process of loading the native parameter as the first parameter 1, if there is no conflict between the native constraint information and the constraint information of the first parameter 1, then it follows... Figure 10 The dashed line directly loads the original parameter as the first parameter 1, which means loading the original constraint information as the constraint information of the first parameter 1; if there is a conflict between the original constraint information and the constraint information of the first parameter 1, a transformation operator is inserted in the process of loading the original parameter as the first parameter 1.

[0134] For example, such as Figure 10 As shown, when loading the native parameter as the first parameter 1, there may be cross-device transmission. The native parameter can be first sent to an external device using a send operator to perform the required processing, and then the processed native parameter can be received using a receive operator. For example, a conversion operator can be inserted between the receive operator and the second operator 1 to load the native parameter as the first parameter 1.

[0135] It should be noted that, Figures 6-10 The first or second sub-graph shown is merely an example. The specific structure, type, and function of the first or second sub-graph can be selected according to actual needs, and the embodiments disclosed herein do not impose any limitations on this.

[0136] The computation graph optimization method provided in at least one embodiment of this disclosure can determine the constraint information of the first parameter before performing minimal dependency pruning on the computation graph, without depending on the specific structure of each subgraph after pruning. It can also be compatible with the constraint information in a specific chip and support the flexible combination of operators, thereby greatly reducing the design difficulty and computational complexity of the computation graph and improving computational efficiency.

[0137] In some examples, in response to the need to insert a transformation operator in a third subgraph among multiple subgraphs, the computation graph optimization method provided in at least one embodiment of this disclosure may further include: using a derivation algorithm to decide how to insert the corresponding transformation operator in the third subgraph.

[0138] For example, the derivation algorithm can traverse the computation graph and, when encountering conflicts in the constraint information of upstream and downstream operators, deduce which reorder op should be inserted to resolve the conflict through the loss function graphCost, while ensuring the optimal performance of the computation graph.

[0139] For example, with Figure 5 Taking the computational graph shown as an example, the loss function `graphCost` decomposes the problem of how to insert transformation operators into two sub-problems: 1) operator A; 2) a subgraph composed of operators B and C. For example, operator A has only one solution to its constraints, namely "input: L0, output: L0", but the solutions to the constraints of the subgraph composed of operators B and C are still unknown. To obtain the solution to the constraints with the least performance overhead, such as... Figure 5 The performance of the computational graph shown can be expressed as:

[0140] opCost(A, "Input: L0, Output: L0")+graphCost(B+C, Subgraph Constraints)

[0141] Here, `opCost` represents the loss function of the operator, and `graphCost` represents the loss function of the subgraph. For example, the value of `opCost(A, "Input: L0, Output: L0")` can be obtained by querying the loaded constraint configuration table (e.g., Table 1), that is, the performance of operator A under constraint 1 (Input: L0, Output: L0); the value of `graphCost(B+C, Subgraph Constraint)` needs to be solved by calling the `graphCost` function again (the process of a function calling itself is called a recursive call). Therefore, by solving the `graphCost` function, the result of the entire addition expression can be obtained.

[0142] For example, recursively solving the subgraph composed of operators B and C can be further decomposed into two subproblems: 1) operator B; 2) the subgraph composed only of operator C. For example, the constraint configuration table (e.g., Table 1) shows that operator B has two solutions: 1) input: L1, output: L1; 2) input: L2, output: L2. Therefore, the solutions to the subproblems of the subgraph composed of operators B and C are obtained:

[0143] min(

[0144] opCost(transformation operator R, "input: L0, output: L1")+opCost(B, "input: L1, output:

[0145] L1”)+graphCost(C, subgraph constraint1),

[0146] opCost(transformation operator R, "input: L0, output: L2")+opCost(B, "input: L2, output:

[0147] L2”)+graphCost(C, SubgraphConstraint2) )

[0149] The `min` function selects the optimal solution from two possible outcomes. Since operator B conflicts with the constraint information of its upstream operator A, its performance needs to be calculated by adding the cost of the corresponding transformation operator R. The value obtained by the `opCost` function can be obtained by querying the constraint configuration table (e.g., Table 1). For example, the second parameter differs in the two calls to the `graphCost` function. In subgraph constraint 1, the output of upstream operator B of subgraph operator C is L1; in subgraph constraint 2, the output of upstream operator B is L2.

[0150] For example, further, the subgraph composed of operator C can be solved recursively. For example, the constraint configuration table (e.g., Table 1) shows that operator C has two solutions: 1) Input: L2, Output: L2; 2) Input: L3, Output: L3; However, whether it is necessary to insert the transformation operator R depends on whether the output of the upstream operator B of operator C is L1 or L2 in the input subgraph constraint 1 or subgraph constraint 2 in the previous step:

[0151] 1) When the output of the upstream operator B is L1:

[0152] min(

[0153] opCost(transformation operator R, "input: L1, output: L2") + opCost(C, "input: L2, output: L2")

[0154] L2”),

[0155] opCost(transformation operator R, "input: L1, output: L3") + opCost(C, "input: L3, output:

[0156] L3”) )

[0158] 2) When the output of the upstream operator B is L2:

[0159] min(

[0160] opCost(C,“Input:L2,Output:L2”),

[0161] opCost(transformation operator R, "input: L2, output: L3") + opCost(C, "input: L3, output:

[0162] L3”)

[0163] For example, after multiple recursive solutions as described above, we can prioritize selecting those that make... Figure 5 The solution to the constraint information that minimizes the performance loss of the computation graph is taken as the solution of the computation graph, thus enabling decisions on where and what transformation operator R to insert in the computation graph. Furthermore, a cached map can be used to save the solutions to solved subproblems to avoid duplication of effort.

[0164] It should be noted that the above derivation algorithm is only an example, and the conversion operator can also be selected in other ways according to actual needs. The embodiments disclosed herein do not limit this.

[0165] Figure 11 A schematic block diagram of a computational graph optimization apparatus provided for at least one embodiment of the present disclosure.

[0166] For example, such as Figure 11 As shown, the optimization device 200 for the computational graph includes an optimization module 210 and a trimming module 220.

[0167] For example, optimization module 210 is configured to determine the constraint information of the first parameter in the computation graph. That is, optimization module 210 can be configured to perform, for example... Figure 2 Step S10 is shown.

[0168] For example, the pruning module 220 is configured to perform minimum dependency pruning on the computation graph to obtain multiple subgraphs. That is, the pruning module 220 can be configured to perform, for example... Figure 2 Step S20 is shown.

[0169] For example, optimization module 210 is further configured to determine, based on the constraint information of the first parameter, whether transformation operators need to be inserted in multiple subgraphs. That is, optimization module 210 can also be configured to perform, for example... Figure 2 Step S30 is shown.

[0170] In some examples, such as Figure 11 As shown, the computational graph optimization device 200 may further include a registration module 230 configured to register a first optimization operator. For example, the optimization module 210 is further configured to use the first optimization operator to determine constraint information for a first parameter.

[0171] For example, registration module 230 is further configured to register at least one second optimization operator. For example, optimization module 210 is further configured to determine, based on the constraint information of the first parameter, whether a transformation operator needs to be inserted into multiple subgraphs using at least one second optimization operator.

[0172] In some examples, the optimization module 210 is also configured to record the constraint information of the first parameter in the variable operator. For example, the variable operator can be accessed synchronously by multiple subgraphs.

[0173] In some examples, in response to the need to insert transformation operators in multiple subgraphs, optimization module 210 is also configured to insert transformation operators in multiple subgraphs using at least one second optimization operator, resulting in multiple subgraphs for execution.

[0174] In some examples, the computation graph includes multiple operators, and the optimization module 210 is further configured to: load a constraint configuration table of the computation graph, wherein the constraint configuration table records the constraint information of each operator among the multiple operators; determine the constraint information of at least one first operator among the multiple operators based on the constraint configuration table, wherein at least one first operator uses a first parameter; and determine the constraint information of the first parameter based on the constraint information of at least one first operator.

[0175] In some examples, multiple subgraphs include a first subgraph, which includes multiple operators. The optimization module 210 is further configured to: determine whether there is a conflict between the multiple operators of the first subgraph based on the constraint information and constraint configuration table of the first parameter; and in response to the conflict between the multiple operators of the first subgraph, insert a transformation operator between the conflicting operators.

[0176] For example, the first subgraph includes at least one first operator that uses a first parameter among multiple operators, and the first subgraph includes a training subgraph or a validation subgraph. The first operator in the training subgraph is used to train the first parameter and the input data of the computation graph, and the first operator in the validation subgraph is used to validate the training result output by the training subgraph.

[0177] In some examples, multiple subgraphs include a second subgraph, which includes multiple operators. The optimization module 210 is further configured to: determine whether there is a conflict between the multiple operators in the second subgraph based on the constraint information of the first parameter and the constraint configuration table, wherein the constraint configuration table records the constraint information of each operator in the multiple operators; and insert a transformation operator in the second subgraph in response to the conflict between the multiple operators in the second subgraph.

[0178] For example, the second subgraph includes an initialization subgraph, and the optimization module 210 is further configured to: determine the constraint information of the first value based on the constraint configuration table; and before assigning the first value to the first parameter, determine whether there is a conflict between the constraint information of the first value and the constraint information of the first parameter. For example, in response to a conflict between multiple operators of the second subgraph, the optimization module 210 is further configured to: in response to a conflict between the constraint information of the first value and the constraint information of the first parameter, insert a transformation operator in the process of assigning the first value to the first parameter.

[0179] For example, the second subgraph includes a saving subgraph, which is used to save the first parameter as a native parameter. The constraint information of the native parameter is the native constraint information. The optimization module 210 is also configured to: determine the native constraint information based on the constraint configuration table; and determine whether there is a conflict between the native constraint information and the constraint information of the first parameter before saving the first parameter. For example, in response to a conflict between multiple operators in the second subgraph, the optimization module 210 is also configured to: insert a transformation operator in the process of saving the first parameter in response to a conflict between the native constraint information and the constraint information of the first parameter.

[0180] For example, the second subgraph includes a loading subgraph, which is used to load the native parameters as the first parameters. The constraint information of the native parameters is the native constraint information. The optimization module 210 is further configured to: determine the native constraint information based on the constraint configuration table; and determine whether there is a conflict between the native constraint information and the constraint information of the first parameter before loading the native parameters as the first parameters. For example, in response to a conflict between multiple operators in the second subgraph, the optimization module 210 is further configured to: insert a transformation operator in the process of loading the native parameters as the first parameters in response to a conflict between the native constraint information and the constraint information of the first parameter.

[0181] Due to the above description, for example Figure 2 The optimization method for the computational graph shown has already described in detail the operation of the optimization device 200 for the aforementioned computational graph. Therefore, for the sake of brevity, it will not be repeated here. For relevant details, please refer to the above description. Figures 2 to 10 The description.

[0182] It should be noted that, Figure 11 The modules described above in the optimization apparatus 200 of the illustrated computational graph can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. For example, these modules may correspond to dedicated integrated circuits, pure software code, or modules combining software and hardware. As an example, see [reference needed]. Figure 11 The device described may be a PC computer, tablet device, personal digital assistant, smartphone, web application or other device capable of executing program instructions, but is not limited thereto.

[0183] Furthermore, although the computational graph optimization apparatus 200 described above is divided into modules for performing corresponding processes, it is clear to those skilled in the art that the processes performed by each module can also be performed without any specific module division in the apparatus or without clear boundaries between the modules. In addition, the above references... Figure 11 The optimization device 200 for the computational graph described is not limited to the modules described above, but may also include other modules (e.g., reading modules, control modules, etc.) as needed, or the above modules may be combined.

[0184] At least one embodiment of this disclosure also provides an electronic device including a processor and a memory; the memory includes one or more computer program modules; the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include optimization methods for implementing the computational graph provided by the embodiments of this disclosure described above. For example, the processor may be a single-core processor or a multi-core processor.

[0185] Figure 12 This is a schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0186] For example, such as Figure 12 As shown, the electronic device 300 includes a processor 310 and a memory 320. For example, the memory 320 is used to store non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 310 is used to execute the non-transitory computer-readable instructions, which, when executed by the processor 310, can perform one or more steps of the optimization method according to the computational graph described above. The memory 320 and the processor 310 can be interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0187] For example, processor 310 can be a central processing unit (CPU), graphics processing unit (GPU), general-purpose graphics processing unit (GPGPU), digital signal processor (DSP), or other processing unit with computational graph optimization capabilities and / or program execution capabilities, such as a field-programmable gate array (FPGA); for example, the central processing unit (CPU) can be an x86, RISC-V, or ARM architecture. Processor 310 can be a general-purpose processor or a special-purpose processor, capable of controlling other components in electronic device 300 to perform desired functions.

[0188] For example, memory 320 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable optical disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and processor 310 may run one or more computer program modules to implement various functions of electronic device 300. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0189] It should be noted that, in the embodiments of this disclosure, the specific functions and technical effects of the electronic device 300 can be referred to the description above of the optimization method of the computational graph provided in at least one embodiment of this disclosure, and will not be repeated here.

[0190] Figure 13 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0191] For example, such as Figure 13 As shown, the electronic device 400 is, for example, suitable for implementing the computational graph optimization method provided in the embodiments of this disclosure. It should be noted that... Figure 13 The illustrated electronic device 400 is merely an example and does not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0192] For example, such as Figure 13 As shown, electronic device 400 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 41, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 42 or a program loaded from storage device 48 into random access memory (RAM) 43. RAM 43 also stores various programs and data required for the operation of electronic device 400. Processing unit 41, ROM 42, and RAM 43 are interconnected via bus 44. Input / output (I / O) interface 45 is also connected to bus 44. Typically, the following devices can be connected to I / O interface 45: input devices 46 including, for example, touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 47 including, for example, liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 48 including, for example, magnetic tape, hard disk, etc.; and communication devices 49. Communication device 49 allows electronic device 400 to communicate wirelessly or wiredly with other electronic devices to exchange data.

[0193] Although Figure 13 An electronic device 400 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 400 may alternatively implement or have more or fewer devices.

[0194] For a detailed description of the electronic device 400 and its technical effects, please refer to the above description of the optimization method for the computational graph; it will not be repeated here.

[0195] Figure 14 This is a schematic diagram of a storage medium provided for at least one embodiment of the present disclosure.

[0196] For example, such as Figure 14 As shown, storage medium 500 stores non-transitory computer-readable instructions 510. For example, when the non-transitory computer-readable instructions 510 are executed by a computer, one or more steps in the optimization method according to the computational graph described above are performed.

[0197] For example, this storage medium 500 can be applied to Figure 14In the illustrated electronic device 300, for example, the storage medium 500 can be the memory 320 within the electronic device 300. For example, a description of the storage medium 500 can be found here. Figure 14 The corresponding description of the memory 320 in the illustrated electronic device 300 will not be repeated here.

[0198] The following points need to be clarified regarding this disclosure:

[0199] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0200] (2) Where there is no conflict, features of the same embodiment and different embodiments of this disclosure can be combined with each other.

[0201] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A computational graph optimization method applied to a chip, wherein, The chip includes a general-purpose graphics processor, the computation graph is applied to deep learning computation tasks, and the operators or parameters running on the chip have constraint information, including data format and layout in memory. The optimization method for the computation graph includes: Determine the constraint information of the first parameter in the computation graph, wherein the first parameter includes the weight parameters and bias parameters in the neural network; The computation graph is pruned with minimal dependencies to obtain multiple subgraphs, which are then executed during the execution process to achieve the deep learning computation task; and Based on the constraint information of the first parameter, determine whether it is necessary to insert a transformation operator in the plurality of subgraphs; The step of determining whether a transformation operator needs to be inserted into the plurality of subgraphs based on the constraint information of the first parameter includes: In response to a mismatch in constraint information of the corresponding first parameter in the plurality of subgraphs, the transformation operator is inserted into the plurality of subgraphs.

2. The method for optimizing a computational graph according to claim 1 further includes: Record the constraint information of the first parameter into the variable operator. The variable operator is accessed synchronously by the multiple subgraphs.

3. The method for optimizing a computational graph according to claim 1 further includes: Register the first optimization operator. The constraint information for determining the first parameter in the computation graph includes: The first optimization operator is used to determine the constraint information of the first parameter.

4. The method for optimizing a computational graph according to claim 1 further includes: Register at least one second optimization operator; The process of determining whether the transformation operator needs to be inserted into the plurality of subgraphs based on the constraint information of the first parameter includes: Based on the constraint information of the first parameter, the at least one second optimization operator is used to determine whether the transformation operator needs to be inserted into the plurality of subgraphs.

5. The method for optimizing a computational graph according to claim 4, wherein, In response to the need to insert the transformation operator into the plurality of subgraphs, the method further includes: The transformation operator is inserted into the plurality of subgraphs using the at least one second optimization operator to obtain a plurality of subgraphs for execution.

6. The method for optimizing a computational graph according to claim 1, wherein, The computation graph includes multiple operators. Determining the constraint information of the first parameter in the computation graph includes: Load the constraint configuration table of the computation graph, wherein the constraint configuration table records the constraint information of each of the plurality of operators; Based on the constraint configuration table, constraint information of at least one first operator among the plurality of operators is determined, wherein the at least one first operator uses the first parameter; The constraint information of the first parameter is determined based on the constraint information of the at least one first operator.

7. The method for optimizing a computational graph according to claim 6, wherein, The plurality of subgraphs includes a first subgraph, and the first subgraph includes a plurality of operators. Based on the constraint information of the first parameter, determining whether the transformation operator needs to be inserted into the plurality of subgraphs includes: Based on the constraint information of the first parameter and the constraint configuration table, it is determined whether there is a conflict among the multiple operators of the first subgraph; In response to a conflict between multiple operators in the first subgraph, the transformation operator is inserted between the conflicting operators.

8. The method for optimizing a computational graph according to claim 6, wherein, The first subgraph includes at least one first operator that uses the first parameter among its plurality of operators. The first subgraph includes a training subgraph or a validation subgraph. The first operator in the training subgraph is used to train the first parameter and the input data of the computation graph. The first operator in the validation subgraph is used to validate the training result output by the training subgraph.

9. The method for optimizing a computational graph according to claim 1, wherein, The plurality of subgraphs includes a second subgraph, and the second subgraph includes a plurality of operators. Based on the constraint information of the first parameter, determining whether the transformation operator needs to be inserted into the plurality of subgraphs includes: Based on the constraint information and constraint configuration table of the first parameter, it is determined whether there is a conflict among the multiple operators of the second subgraph, wherein the constraint configuration table records the constraint information of each of the multiple operators; In response to a conflict between multiple operators in the second subgraph, the transformation operator is inserted into the second subgraph.

10. The method for optimizing a computational graph according to claim 9, wherein, The second subgraph includes an initialization subgraph. Based on the constraint information of the first parameter and the constraint configuration table, it is determined whether there is a conflict among multiple operators in the second subgraph, including: The constraint information for the first value is determined based on the constraint configuration table; Before assigning the first value to the first parameter, it is determined whether there is a conflict between the constraint information of the first value and the constraint information of the first parameter.

11. The method for optimizing a computational graph according to claim 10, wherein, In response to a conflict among multiple operators in the second subgraph, the transformation operator is inserted into the second subgraph, including: In response to a conflict between the constraint information of the first value and the constraint information of the first parameter, the transformation operator is inserted in the process of assigning the first value to the first parameter.

12. The method for optimizing a computational graph according to claim 9, wherein, The second subgraph includes a saved subgraph, which is used to save the first parameter as a native parameter, wherein the constraint information of the native parameter is native constraint information. Based on the constraint information of the first parameter and the constraint configuration table, it is determined whether there is a conflict among multiple operators in the second subgraph, including: The native constraint information is determined based on the constraint configuration table; Before saving the first parameter, it is determined whether there is a conflict between the original constraint information and the constraint information of the first parameter.

13. The method for optimizing a computational graph according to claim 12, wherein, In response to a conflict among multiple operators in the second subgraph, the transformation operator is inserted into the second subgraph, including: In response to a conflict between the original constraint information and the constraint information of the first parameter, the transformation operator is inserted in the process of saving the first parameter.

14. The method for optimizing a computational graph according to claim 9, wherein, The second subgraph includes a loading subgraph, which is used to load the native parameters as the first parameters, wherein the constraint information of the native parameters is the native constraint information. Based on the constraint information of the first parameter and the constraint configuration table, it is determined whether there is a conflict among multiple operators in the second subgraph, including: The native constraint information is determined based on the constraint configuration table; Before loading the native parameter as the first parameter, it is determined whether there is a conflict between the native constraint information and the constraint information of the first parameter.

15. The method for optimizing a computational graph according to claim 14, wherein, In response to a conflict among multiple operators in the second subgraph, the transformation operator is inserted into the second subgraph, including: In response to a conflict between the native constraint information and the constraint information of the first parameter, the transformation operator is inserted during the process of loading the native parameter as the first parameter.

16. A computational graph optimization device, applied to a chip, wherein, The chip includes a general-purpose graphics processor, the computation graph is applied to deep learning computation tasks, and the operators or parameters running on the chip have constraint information, including data format and layout in memory. The optimization device for the computational graph includes: The optimization module is configured to determine constraint information of a first parameter in the computation graph, wherein the first parameter includes weight parameters and bias parameters in the neural network; and The pruning module is configured to perform minimum dependency pruning on the computation graph to obtain multiple subgraphs, so as to realize the deep learning computation task by executing the multiple subgraphs during the execution process phase; The optimization module is further configured to determine, based on the constraint information of the first parameter, whether it is necessary to insert a transformation operator in the plurality of subgraphs; The optimization module is further configured to insert the transformation operator into the multiple subgraphs in response to a mismatch in the constraint information of the corresponding first parameter in the multiple subgraphs.

17. The optimization apparatus for a computational graph according to claim 16 further includes a registration module configured to register a first optimization operator. The optimization module is further configured to use the first optimization operator to determine the constraint information of the first parameter.

18. The computational graph optimization apparatus according to claim 17, wherein, The registered module is also configured to register at least one second optimization operator; The optimization module is further configured to determine, based on the constraint information of the first parameter, whether the transformation operator needs to be inserted into the plurality of subgraphs using the at least one second optimization operator.

19. An electronic device comprising: processor; Memory, including one or more computer program modules; The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules are used to implement the optimization method of the computation graph according to any one of claims 1-15.

20. A storage medium storing non-transitory computer-readable instructions that, when executed by a computer, implement the optimization method of the computation graph according to any one of claims 1-15.

Images