A neural network compilation optimization method and related apparatus

Abstract

The embodiment of the application discloses a neural network compilation optimization method, comprising: dividing a computation graph of a neural network into a plurality of subgraphs; querying a topology feature library to determine whether an optimization strategy of a known subgraph that is consistent with the topology of at least one divided subgraph exists; if the optimization strategy exists, extracting the optimization strategy of the known subgraph that is consistent with the topology of the at least one divided subgraph from the topology feature library to optimize the at least one subgraph. The embodiment of the application stores the compilation optimization strategy of the known subgraph of the neural network in the feature library. When the compilation of the neural network is performed, the computation graph of the neural network is divided into a plurality of subgraphs, the optimization strategy corresponding to at least one divided subgraph is found from the topology feature library, and the optimization strategy is directly applied to the subgraph, so that repeated optimization calculation of the subgraph is not needed, and the compilation process of the neural network model is accelerated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of neural network compilation, and in particular, to a neural network compilation optimization method and related device. BACKGROUND

[0002] Currently, artificial neural network is one of the common computing models in the field of artificial intelligence. Unlike source program codes described by high-level programming languages such as C / C++, a neural network is represented by a special model describing a neural network algorithm. Further, through the compilation of the neural network model, i.e., the conversion of the neural network algorithm into a general computing graph, the optimization of the computing graph, and the mapping of the optimized computing graph into instructions and machine codes executable by a backend hardware platform, the model of the neural network is converted into target codes for execution on a computing platform. The neural network compilation optimization is a complex and time-consuming computing process. A typical neural network compilation optimization process includes graph conversion, subgraph segmentation, constant folding, equivalent subgraph transformation, operator fusion, L1 / L2 memory data reuse, DDR memory allocation and reuse, etc. There are often many identical structures in a neural network model, and repeated compilation of these identical structures will cause a serious decrease in the efficiency of online compilation or compilation of super-large models. In addition, if a user only modifies a small part of the neural network model, the above complex and lengthy compilation process has to be repeated, which affects the user experience. SUMMARY

[0003] Embodiments of the present application provide a neural network compilation optimization method and related device, which are used to solve at least one of the above problems in the art.

[0004] According to a first aspect of the present application, a neural network compilation optimization method is provided, which includes: dividing a computing graph of a neural network into a plurality of subgraphs; querying a topology feature library to determine whether there is an optimization strategy of a known subgraph that is consistent with the topology of at least one divided subgraph; and if so, extracting an optimization strategy of a known subgraph that is consistent with the topology of the at least one divided subgraph from the topology feature library to optimize the at least one subgraph.

[0005] According to the first aspect of the present application, the topology feature library includes topology feature vectors of known subgraphs and corresponding optimization strategies, and the step of querying includes querying whether there is an optimization strategy of a known subgraph that is consistent with the topology feature vector of the at least one divided subgraph.

[0006] According to the first aspect of the present application, the method further comprises: if there is no optimization strategy of a known subgraph consistent with the topological feature vector of the at least one segmented subgraph, performing a compilation optimization calculation on the at least one segmented subgraph; generating an optimization strategy of the at least one segmented subgraph after the compilation optimization of the at least one segmented subgraph; and adding the topological feature vector of the at least one segmented subgraph and the optimization strategy of the at least one segmented subgraph into the topological feature library in association.

[0007] According to the first aspect of the present application, performing a compilation optimization calculation on the at least one segmented subgraph comprises determining an equivalent subgraph of the at least one segmented subgraph by optimization calculation, and replacing the at least one segmented subgraph with the equivalent subgraph; and the adding step comprises storing the topological feature vector of the at least one segmented subgraph and the topological feature vector of the equivalent subgraph in the topological feature library in association.

[0008] According to the first aspect of the present application, performing a compilation optimization calculation on the at least one segmented subgraph comprises determining that a plurality of parts in the at least one segmented subgraph can be merged to reduce the calculation amount of the subgraph or the calculation speed of the subgraph, and merging the plurality of parts; and the adding step comprises adding the topological feature vector of the at least one segmented subgraph and the topological feature vector of the merged subgraph into the topological feature library in association.

[0009] According to the first aspect of the present application, performing a compilation optimization calculation on the at least one segmented subgraph comprises determining an equivalent constant of the at least one segmented subgraph, and replacing the at least one segmented subgraph with the equivalent constant; and the adding step comprises adding the topological feature vector of the at least one segmented subgraph and the equivalent constant into the topological feature library in association.

[0010] According to the first aspect of the present application, the step of segmenting comprises segmenting the computation graph into a plurality of coarse-grained subgraphs by using a clustering technique.

[0011] According to the first aspect of the present application, the method further comprises: if there is no optimization strategy of a known subgraph consistent with the topological feature vector of the at least one segmented coarse-grained subgraph, segmenting the at least one segmented coarse-grained subgraph into a plurality of fine-grained subgraphs, querying the topological feature library to determine whether there is an optimization strategy of a known subgraph consistent with the topological feature vector of the at least one fine-grained subgraph; and if there is an optimization strategy of a known subgraph consistent with the topological feature vector of the at least one fine-grained subgraph, extracting the optimization strategy of the known subgraph consistent with the topological feature vector of the at least one fine-grained subgraph from the topological feature library to optimize the at least one fine-grained subgraph.

[0012] According to a first aspect of the present application, wherein if there is no known subgraph optimization strategy consistent with the topology feature vector of the at least one fine-grained subgraph, then: performing a compilation optimization calculation on the at least one fine-grained subgraph; generating an optimization strategy for the at least one fine-grained subgraph; adding the topology feature vector of the at least one fine-grained subgraph and its optimization strategy to the topology feature library.

[0013] According to a second aspect of the present application, an apparatus for neural network compilation optimization is provided, comprising: a processor and a memory, the processor being configured to execute program instructions stored in the memory to enable the apparatus to implement any of the above methods.

[0014] According to a third aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores program code which, when executed by a computer, implements any of the above methods.

[0015] According to a fourth aspect of the present application, a computer program product is provided, wherein the program code contained in the computer program product, when executed by a computer, implements any of the above methods.

[0016] According to a fifth aspect of the present application, an apparatus for neural network compilation optimization is provided, comprising: a partitioning unit configured to partition a computation graph of a neural network into a plurality of subgraphs; a querying unit configured to query a topology feature library to determine whether there is a known subgraph optimization strategy consistent with the topology of the at least one partitioned subgraph; and an optimization unit configured to, if there is, extract the optimization strategy of the known topology consistent with the topology of the at least one partitioned subgraph from the topology feature library to optimize the at least one subgraph.

[0017] According to the fifth aspect of the present application, the topology feature library comprises topology feature vectors of known subgraphs and corresponding optimization strategies, and the querying unit is further configured to query whether there is a known subgraph optimization strategy consistent with the topology feature vector of the at least one partitioned subgraph.

[0018] According to the fifth aspect of the present application, wherein if there is no known subgraph optimization strategy consistent with the topology feature vector of the at least one partitioned subgraph, then the optimization unit is further configured to: perform a compilation optimization calculation on the at least one partitioned subgraph; generate an optimization strategy for the at least one partitioned subgraph after the compilation optimization of the at least one partitioned subgraph; and add the topology feature vector of the at least one partitioned subgraph and its optimization strategy into the topology feature library in association.

[0019] According to the fifth aspect of the present application, wherein, when performing the compiling optimization calculation on the at least one segmented subgraph, the optimization unit is configured to: determine an equivalent subgraph of the at least one segmented subgraph by optimization calculation, replace the at least one segmented subgraph with the equivalent subgraph; and the optimization unit is configured to: store the topological feature vector of the at least one segmented subgraph and the topological feature vector of the equivalent subgraph in the topological feature library in association when performing the adding.

[0020] According to the fifth aspect of the present application, wherein, when performing the compiling optimization calculation on the at least one segmented subgraph, the optimization unit is configured to: determine that a plurality of parts in the subgraph can be merged to reduce the calculation amount of the subgraph or improve the calculation speed of the subgraph by optimization calculation, and merge the plurality of parts; and the optimization unit is configured to: add the topological feature vector of the subgraph and the topological feature vector of the merged subgraph to the topological feature library in association when performing the adding.

[0021] According to the fifth aspect of the present application, wherein, when performing the compiling optimization calculation on the at least one segmented subgraph, the optimization unit is configured to: determine an equivalent constant of the subgraph by optimization calculation, and replace the subgraph with the constant; and the optimization unit is configured to: add the topological feature vector of the subgraph and the constant to the topological feature library in association when performing the adding.

[0022] According to the fifth aspect of the present application, wherein the segmentation unit is further configured to: segment the computation graph into coarse-grained subgraphs using a clustering technique.

[0023] According to the fifth aspect of the present application, wherein, if there is no optimization strategy of a known subgraph consistent with the topological feature vector of the at least one segmented coarse-grained subgraph, the segmentation unit is configured to: segment the at least one segmented coarse-grained subgraph into a plurality of fine-grained subgraphs; the query unit is configured to: query whether there is an optimization strategy of a known subgraph consistent with the topological feature vector of the at least one fine-grained subgraph in the topological feature library; and if there is an optimization strategy of a known subgraph consistent with the topological feature vector of the at least one fine-grained subgraph, the optimization unit is configured to: extract the optimization strategy of the known subgraph consistent with the topological feature vector of the at least one fine-grained subgraph from the topological feature library to optimize the at least one fine-grained subgraph.

[0024] According to the fifth aspect of the present application, wherein, if there is no optimization strategy of a known subgraph consistent with the topological feature vector of the at least one fine-grained subgraph, the optimization unit is configured to: perform a compiling optimization calculation on the at least one fine-grained subgraph; generate an optimization strategy of the at least one fine-grained subgraph; and add the topological feature vector of the at least one fine-grained subgraph and the optimization strategy thereof to the topological feature library.

[0025] Embodiments of the present application accelerate the execution of the compilation process by introducing a topology feature library. Specifically, the compilation optimization strategies of known subgraphs of a neural network are stored in the feature library. When performing the compilation of the neural network, the computation graph of the neural network is divided into a plurality of subgraphs, an optimization strategy corresponding to at least one of the divided subgraphs is found from the topology feature library, and the optimization strategy is directly applied to the subgraph, without the need to repeatedly perform optimization calculation on the subgraph, thereby accelerating the compilation process of the neural network model. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a structural diagram of a neural network;

[0027] Figure 2 is a flowchart of a neural network compilation optimization process of an embodiment of the present application;

[0028] Figure 3 is a schematic diagram of the computation graph of the neural network in the embodiment of Figure 2

[0029] Figure 4 is a schematic diagram of node attribute information in the embodiment of Figure 2

[0030] Figure 5 is a schematic diagram of an equivalent subgraph in the neural network in the embodiment of Figure 2

[0031] Figure 6 is a flowchart of a neural network compilation optimization method of another embodiment of the present application;

[0032] Figure 7 is a schematic diagram of fine-grained division of the computation graph in the embodiment of Figure 6

[0033] Figure 8 is a schematic diagram of a neural network compilation server for implementing the neural network compilation optimization method of the embodiments of the present application;

[0034] Figure 9 is a schematic diagram of a computing device for implementing the neural network compilation optimization method of the embodiments of the present application.

[0035] Figure 10 is a schematic diagram of an apparatus for implementing the neural network compilation optimization method of the embodiments of the present application. DETAILED DESCRIPTION

[0036] ​​​​The technical solutions provided by the present application will be further described below with reference to the drawings and examples. It should be understood that the systems and scenarios provided in the examples of the present application are mainly to explain some possible implementations of the technical solutions of the present application, and should not be interpreted as the only limitation of the technical solutions of the present application.

[0037] The terms "first", "second", and "third" and the like in the examples of the present application and the accompanying drawings are used to distinguish similar objects, and do not necessarily mean a specific order or sequence. In addition, the terms "include" and "have" and any variations thereof are intended to mean non-exclusive inclusion. The method, system, product or device is not necessarily limited to only those steps or units listed literally, but can include other steps or units that are not listed literally or inherent to these processes, methods, products or devices.

[0038] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0039] It should be understood that in the present application, the size of the serial number of each step does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the examples of the present application.

[0040] The meanings of several technical terms related to the present application are introduced below.

[0041] The neural network mentioned in the present application, i.e. artificial neural network (ANN), can be various artificial neural networks known in the art, such as deep neural network (DNN), recurrent neural network (RNN), or convolutional neural network (CNN).

[0042] A neural network is generally represented by a special model describing the neural network algorithm. The compilation of a neural network refers to the conversion of a neural network algorithm into a general computation graph, the optimization and reconstruction of the computation graph, and the mapping of the optimized computation graph into instructions and machine code executable by a backend hardware platform, thereby completing the compilation of the neural network algorithm for the hardware platform.

[0043] A computation graph is a graph that represents the computation of a neural network model, for example, by a plurality of nodes and directed edges between the nodes. The nodes include at least one of a variable node, an operator node, and a sample node. A directed edge between two nodes is used to represent a dependency between the two nodes. The directed edge between the nodes has an attribute, for example, a weight, which represents a value that is multiplied by a signal (or value) input to the next node through the directed edge.

[0044] The neural network compiler can be various types of computing platforms or neural network compilers for neural network computation, for example, a deep neural network compiler (DNNC) that can compile a neural network algorithm into an instruction stream for a DPU (Deep Learning Processor Unit, DLPU) platform. The neural network compiler can also be other various types of compilers known in the art.

[0045] Embodiments of the present application relate to a neural network compilation optimization method. The method of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0046] Figure 1 A schematic diagram of a neural network structure is provided. As shown in Figure 1 , the neural network structure includes a series of sequentially operated layers, such as a convolution layer (Conv), a batch normalization (BatchNorm, BN) layer, a scale layer, an eltwise layer, a Relu (Rectified Linear Unit) layer, etc. The above layers are merely examples and can also include other various types of layers. In addition, the neural network structure provided by the present application is merely for illustration, and the neural network in the present application is not limited to the structure as shown in Figure 1 , and can be various types of neural network structures in the art. Figure 1

[0047] As shown in Figure 1 , in the neural network, the computation output of a previous layer is the input of a next layer. For example, for Figure 1 ​For the boxes containing consecutive Conv, BN and Scale, during the calculation process, the input data required by the Conv layer is first loaded from the off-chip memory to the on-chip cache, the Conv layer performs calculations based on the loaded data, and then stores the calculation results in the off-chip memory; then, the BN layer loads the output results of the Conv layer from the off-chip memory as input data and other necessary parameters, performs calculations, and then stores the results in the off-chip memory; then the Scale layer loads the output results of the previous BN layer from the off-chip memory, performs calculations, and then stores the results in the off-chip memory, and so on, until all the calculation layers are traversed. The calculation process of the above-mentioned neural network is only exemplary. The calculation process (including data access process) of the above-mentioned boxes containing consecutive Conv, BN and Scale can also adopt other operations known in the art.

[0048] The following combination Figures 2-5 The process of the neural network compilation optimization method according to one embodiment of the present application is specifically described.

[0049] like Figure 2 As shown, in step S110, a computational graph of a neural network is generated. Developers can use various deep learning frameworks in the art, such as MxNet, TensorFlow, etc., to design the required neural network model. Furthermore, various neural network model parsers and neural network model builders known in the art can be used to convert the neural network model into a general computational graph corresponding to the neural network processor. For example, the neural network model parser can parse the input neural network model, such as analyzing the grammatical structure or syntax of the input neural network model to generate model information; further, the neural network model builder can generate a computational graph based on the model information, such as a graph including multiple computing nodes.

[0050] Next, in step S120, the computation graph of the neural network is divided into several subgraphs. A clustering graph cut algorithm based on the adjacency matrix can be used to divide the computation graph into several subgraphs. Figure 3 As shown, a clustering algorithm is applied to the adjacency matrix to generate clustering results, thereby determining the boundaries of the segmented subgraphs. Figure 3 The subgraph structures of the segmented neural network, such as subgraph structure A and subgraph structure B, and the corresponding adjacency matrix ( Figure 3 right half).

[0051] The above subgraph segmentation method is only an example, and various other subgraph segmentation techniques known in the art can be used. The above clustering algorithm can be, but is not limited to, K-Means, Graph Community Detection, etc., which will not be described here.

[0052] Next, as shown in step S130, topological information of at least one segmented subgraph is generated, for example, a hash digest of attribute information of each node after traversal in a certain order.

[0053] Traversal of the node can be determining a head node of the subgraph, traversing each node in breadth, and generating a node processing queue. Attribute information of each node is generated from the node queue in sequence, for example Figure 4 The attribute information (Attr Info of Op) of the node shown includes but is not limited to operation type (Op Type), input number (Input Num), shape of each input (Input[0] Shape, Input[1] Shape, etc.), and input node operation type (for example, Input[0] Op Type, Input[1] Op Type, etc.), output number (Output Num), shape of output (Output[0] Shape, etc.), and operation type of output node (for example, Output[0] Op Type), etc. The above attribute information of the node can be encoded into information that can be stored. Here, the encoding can use a Hash algorithm (for example, MD5) for encoding.

[0054] In addition to the above topological information of the subgraph, a topological feature vector can also be used to represent the topological information of the subgraph. In the subgraph of the neural network, there are some subgraphs with different node orders but equivalent structures. For example, Figure 5 In the subgraph of the above, the uppermost Conv layer has three directions of output respectively: Output:0; Output:1; Output:2. Figure 5 The two subgraphs shown belong to subgraphs with different topological orders but equivalent structures.

[0055] For subgraphs with different node ordering but equivalent structure, the order of traversing the nodes is different, thus, different node attribute information summaries can be generated. For subgraphs with equivalent structure, the same optimization strategy can be generally adopted. Preferably, topology information that uniquely represents subgraphs with equivalent structure is needed, avoiding different topology representations for some subgraphs with equivalent structure in the topology feature library. For example, a graph neural network GraphSAGE and a feedforward NN network are adopted to generate the topology feature vector of a subgraph, which can uniquely represent the topology of subgraphs with equivalent structure. Specifically, the adjacency matrix of a subgraph and the topology attribute information of each node of the subgraph (for example, the hash summary of the node attribute described above) are input into the graph neural network GraphSAGE. GraphSAGE is an algorithm known in the art for generating the embedding of each node. Then, the embeddings of the nodes of the subgraph are connected and input into the feedforward neural network, thereby generating the topology feature vector of the subgraph. The above-mentioned way of generating the topology feature vector of a subgraph is exemplary, and various other ways known in the art can also be used to generate the topology feature vector of a subgraph to uniquely represent the topology of subgraphs with equivalent structure.

[0056] In the present embodiment, a topology feature library is provided, in which topology information of known subgraphs and their corresponding optimization strategies are stored. Further, in the topology feature library, the optimization strategy of a subgraph is stored in association with the topology information of the subgraph. At initialization, the topology information of some known subgraphs and the corresponding optimization strategies are stored in the topology feature library. In the topology feature library, the optimization strategy of a subgraph is stored in association with the topology information representing the subgraph. For example, an entry can be used to represent the mapping relationship between the topology information of a subgraph and the optimization strategy. Here, the topology information can use the attribute information of each node of the subgraph described above, or use the topology feature vector of the subgraph described above. For example, the attribute information of each node of the subgraph described above is stored in the topology feature library to represent the topology of the subgraph. Or the topology feature vector of the subgraph described above can be stored in the topology feature library to represent the topology of the subgraph. These are only exemplary, and other topology information reflecting the topology of the subgraph of the neural network known in the art can also be used.

[0057] Next, as shown in step S140, the topology feature library is queried to determine whether there is an optimization strategy of a known subgraph that is consistent with the topology of the at least one segmented subgraph. If so, the corresponding optimization strategy is extracted from the topology feature library, and the optimization strategy is directly applied to the at least one segmented subgraph without further compilation optimization calculation on the at least one segmented subgraph. Specifically, it is determined whether there is an optimization strategy of a known subgraph that is consistent with the topology information (e.g., topology feature vector) of the at least one segmented subgraph in the topology feature library. For example, the topology feature vector of the subgraph generated in step S130 is compared with the topology feature vectors of the known subgraphs in the topology feature library to determine the known subgraph whose topology information (e.g., topology feature vector) is consistent with the topology information (e.g., topology feature vector) of the at least one segmented subgraph in the topology feature library.

[0058] The compilation optimization strategy can be various compilation optimization strategies for the structure of a neural network known in the art. For example, it can be common expression elimination or constant folding of a certain subgraph, and can also be an equivalent subgraph transformation, i.e., a graph replacement operation that replaces the subgraph with another computation graph. It can also be a graph merging operation that merges several parts of the computation graph. It can also be operator fusion that fuses the operators of certain nodes, etc. The optimization strategy is not limited to the above-mentioned enumeration, and can also be various other types of optimization strategies known in the art.

[0059] If there is no optimization strategy of a known subgraph that is consistent with the topology of the at least one segmented subgraph in the topology feature library, then as shown in step S150, a compilation optimization calculation is performed on the at least one segmented subgraph.

[0060] The compilation optimization calculation here refers to optimization reconstruction on the subgraph to reduce the computation amount or improve the computation speed of the subgraph. For example, an equivalent subgraph of the subgraph can be generated through optimization calculation to reduce the computation amount or improve the computation speed, and then the subgraph is replaced by the equivalent subgraph, i.e., graph replacement. It can also be that multiple parts of the subgraph can be merged to reduce the computation amount or improve the computation speed through optimization calculation, and then the multiple parts of the subgraph are merged, i.e., graph merging. It can also be that at least one part of the subgraph is an equivalent constant through optimization calculation, and at least one part of the subgraph is replaced by the constant, i.e., constant folding. It can also be that the operators of a plurality of consecutive layers in the subgraph can be fused to reduce the bandwidth limitation of accessing external memory through optimization calculation, and the operators of the plurality of consecutive layers are fused.

[0061] The above-mentioned compilation optimization calculation is only exemplary, and any compilation optimization calculation that reconstructs the subgraph through optimization calculation to reduce the computation amount or improve the computation speed is within the scope of the present application.

[0062] After the optimization, as shown in step S160, the optimization strategy corresponding to the at least one new subgraph is generated, and then, as shown in step S170, the at least one new subgraph and the optimization strategy thereof are added to the topology feature library. For example, if the above-mentioned graph replacement operation is performed, the topology of the at least one new subgraph and the topology of the replacement graph are added to the topology feature library in association. If the graph merging operation is performed, the topology of the at least one new subgraph and the topology of the optimized graph are added to the topology feature library in association. If the constant folding is performed, the topology of the at least one new subgraph and the equivalent constant can be added to the topology feature library in association. These are exemplary and are not limited to the above-mentioned optimization strategies.

[0063] In the compilation optimization calculation in step S150, the at least one subgraph can be divided into a plurality of smaller subgraphs, the compilation optimization calculation is performed on the at least one smaller subgraph, after the optimization, as shown in step S160, the optimization strategy of the at least one smaller subgraph after the optimization is generated, and the smaller subgraph after the optimization and the corresponding optimization strategy are respectively stored in the topology feature library. The optimization strategy of the original subgraph before the division can also be generated, and the original subgraph and the corresponding optimization strategy are stored in the topology feature library.

[0064] As described before, when initialized, the topology information and the corresponding optimization strategy of a plurality of known subgraphs are stored in the topology feature library. In the compilation optimization of the calculation graph of the neural network in the future, if a new subgraph is encountered, the optimization calculation is performed on the new subgraph, after the optimization, the new subgraph and the optimization strategy thereof can be added to the topology feature library so that when the subgraph is encountered again in the future, the corresponding optimization strategy can be directly retrieved from the topology feature library to apply the optimization strategy to the subgraph without repeating the optimization calculation on the subgraph.

[0065] Based on the above technical solutions of the embodiments of the present application, the neural network topology feature library is set, in which the known subgraphs of the neural network and the optimization strategies thereof are saved. When compiled, the corresponding entry is created for the subgraph of the neural network that is encountered for the first time, and the corresponding optimization strategy is stored. When the same subgraph of the neural network is repeatedly compiled in the future, it is only necessary to query the topology feature database, and for the known subgraph of the neural network, the optimization calculation is not needed, but the corresponding compilation optimization strategy is directly obtained, the repeated optimization calculation on the same subgraph of the neural network is avoided, and the compilation speed is improved.

[0066] The above-mentioned technical solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings. Figure 6 The neural network compilation optimization method of another embodiment of the present application is described in detail.

[0067] Figure 6 The neural network compilation optimization method of the embodiment of the present application is different from the neural network compilation optimization method of the embodiment of the present application Figure 2The difference between the embodiments is that the computation graph is first divided into several coarse-grained subgraphs, and then the topological feature library is queried to determine whether there is an optimization strategy for a known subgraph that is consistent with the topology of at least one coarse-grained subgraph; if not, the coarse-grained subgraph is further divided into fine-grained subgraphs, and then the topological feature library is queried again to determine whether there is an optimization strategy for a known subgraph that is consistent with the topology of at least one fine-grained subgraph. If not, optimization calculations are performed on at least one new fine-grained subgraph or on at least one new coarse-grained subgraph before division. After optimization, the at least one new coarse-grained subgraph or the at least one new fine-grained subgraph and the corresponding optimization strategy are added to the topological feature library. The specific process is as follows:

[0068] See also Figure 6 As shown in step S210, after the computation graph of the neural network is generated, the computation graph is segmented into coarse-grained subgraphs as shown in step S220. The coarse-grained subgraph segmentation mentioned here can be the same as Figure 2 The same technique as used in the embodiment of the computation graph segmentation is employed. For example, a clustering graph cut algorithm based on an adjacency matrix is ​​employed to segment the computation graph into a number of coarse-grained subgraphs. Various other subgraph segmentation techniques known in the art may be employed, including but not limited to K-Means and graph community detection.

[0069] Then, as shown in step S230, Figure 2 Similar to step S130, the topological information of the coarse-grained subgraph is generated, such as the topological feature vector of the subgraph. Figure 2 The embodiment describes the use of a graph neural network GraphSAGE and a feedforward NN network to generate a topological feature vector of a subgraph.

[0070] Then, as shown in step S240, the topology feature library is queried to determine whether there is an optimization strategy for a known subgraph that is topologically consistent with the at least one segmented coarse-grained subgraph. If so, as shown in steps S282 and S284, the corresponding optimization strategy is extracted from the topology feature library and applied to the at least one segmented coarse-grained subgraph. If not, as shown in step S250, the coarse-grained subgraph is further segmented into a plurality of fine-grained subgraphs.

[0071] like Figure 7As shown, the coarse-grained subgraph can be cut into fine-grained subgraphs in the manner of maximum flow and minimum cut. The specific steps are as follows: the weights on the edges of the subgraph are set. For example, the weights on the edges of the subgraph are set according to the optimization items (such as buffer data reuse of operators, data flow template matching, etc.) of the software stack for executing neural network compilation; the edges of the subgraph are cut in the order of the weights from small to large. The above-mentioned manner of setting the weights of the edges is exemplary, and other criteria known in the art can also be used to set the weights of the edges.

[0072] As shown, Figure 7 first, the edge with the smallest weight on the subgraph is cut, for example, the edge with a weight of 1 in Figure 7 ; second, the edge with a larger weight value is cut, for example, the edge with a weight of 2; thereby obtaining a subgraph including only two nodes as shown on the right side of Figure 7 . Figure 7 The fine-grained partitioning of Figure 7 is only exemplary and is not necessarily partitioned into a subgraph including only two nodes as shown on the right side of Figure 7 . The granularity of fine-grained partitioning can be set as needed.

[0073] After the fine-grained subgraph is generated by the above-mentioned partitioning, it is further queried whether there is an optimization strategy of a known subgraph consistent with the topology of at least one fine-grained subgraph from the topology feature library as shown in step S260. If yes, the corresponding optimization strategy is extracted from the topology feature library as shown in steps S282 and S284, and the optimization strategy is applied to the at least one fine-grained subgraph.

[0074] If no, the at least one fine-grained subgraph is subjected to compilation optimization calculation as shown in step S270, and after optimization, the corresponding optimization strategy is generated, and then the topology information of the at least one new fine-grained subgraph and the corresponding optimization strategy are added to the topology feature library as shown in step S280. The at least one new coarse-grained subgraph before partitioning can also be subjected to compilation optimization calculation, and after optimization, the corresponding optimization strategy is generated, and then the topology information of the at least one new coarse-grained subgraph before partitioning and the corresponding optimization strategy are added to the topology feature library.

[0075] Similar to the embodiments of Figure 2 , during initialization, the topology information and the corresponding optimization strategy of a plurality of known subgraphs are stored in the topology feature library. During the compilation optimization of various neural network computation graphs thereafter, if a new subgraph is encountered, the new subgraph is subjected to optimization calculation, and then the new subgraph and the optimization strategy thereof can be added to the topology feature library so that when the subgraph is encountered again in the future, the corresponding optimization strategy can be directly retrieved from the topology feature library to apply the optimization strategy to the subgraph, thereby avoiding repeated optimization calculation of the known neural network subgraph and improving the compilation speed.

[0076] The above-described embodiments of the present application can be implemented in part or in whole through software, firmware, hardware, or any combination thereof. When implemented in software, the present application can be implemented using one or more computer program products. The computer program product(s) can include a computer readable storage medium (or media) having computer readable program code stored thereon. The computer program product(s) can be for use in a computer, for example, a special or general purpose computer(s) having hardware means such as a microprocessor, a microcontroller, a processor unit, or other processing means. The processor unit can include a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof. The computer readable storage medium can be a tangible computer readable storage medium. The computer readable storage medium can be a memory, a computer readable storage medium, a memory on a processor, or a reprogrammable non-volatile computer readable storage medium. Examples of the computer readable storage medium include an electronic, magnetic, optical, electromagnetic, infrared, semiconductor system, a carrier wave, or a semiconductor system including a computer program product. The computer readable program code can include any data or instructions executable by a computer. The computer readable program code can include machine code, such as produced by a compiler, or code that can be executed by an interpreter. The described hardware implementations can include a processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described in the application. With the above implementations in mind, it should be understood that the application can be implemented on one computer or multiple computers or other processing devices. For example, the various elements of the application can be located on a single computer, distributed among co-located computers or devices, or distributed across multiple computers or devices located at different sites, different states, different countries, or even different continents.

[0077] The computer program code or computer program instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, e.g., from a website, a computer, a server, or a datacenter to another website, computer, server, or datacenter via a wired (e.g., coaxial cable, fiber optic cable, etc.) or wireless (e.g., infrared, radio, microwave, etc.) means. The computer readable storage medium can be any available media that can be accessed by a computer. The computer readable storage medium can be a memory, a computer readable storage medium, a memory on a processor, or a reprogrammable non-volatile computer readable storage medium. The computer readable storage medium can be a tangible computer readable storage medium. The tangible computer readable storage medium can include a floppy diskette, a hard disk, a magnetic tape, a compact disk, a DVD, a Blu-ray disk, a memory, a memory on a processor, or a solid state drive. The computer readable program code can include any data or instructions executable by a computer. The computer readable program code can include machine code, such as produced by a compiler, or code that can be executed by an interpreter. The described hardware implementations can include a processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described in the application. With the above implementations in mind, it should be understood that the application can be implemented on one computer or multiple computers or other processing devices. For example, the various elements of the application can be located on a single computer, distributed among co-located computers or devices, or distributed across multiple computers or devices located at different sites, different states, different countries, or even different continents.

[0078] The flow and block diagrams in the accompanying drawings illustrate possible architectures, functionality, and operations for a system and method in accordance with various embodiments of the present application. Each block in the flow and block diagrams can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may be executed in the reverse order, depending upon the functionality involved. Each block and combination of blocks in the block and flow diagrams can be implemented by a dedicated hardware-based system that performs the specified function or functions, or by a combination of dedicated hardware and computer instructions.

[0079] The following description refers to the accompanying drawings, which illustrate preferred embodiments of the application. Figure 8 A neural network compilation server 100 that implements the methods of the present embodiments.

[0080] As Figure 8As shown, the neural network compilation server 100 can be various types of compute graph compilation servers known in the art, or other various types of servers known in the art that implement neural network compilation optimizations.

[0081] The neural network compilation server 100 includes a processor 110, a neural network processor 120, a storage 130, and a cryptographic chip 140.

[0082] The processor 110 can be a general purpose processor or a processor specially designed for a specific technical field. For example, the processor can be a central processing unit (CPU), a digital signal processor (DSP), a micro control unit (MCU), or various types of processors. For example, the processor 110 can be a single core or multi-core CPU processor of various types such as Intel or ARM, for example, an Intel Xeon CPU.

[0083] The neural network processor 120 can be various types of neural network dedicated processors, such as processors specially designed for artificial intelligence (AI) applications, including but not limited to neural network processing units (NPUs), GPUs, tensor processing units (TPUs), and the like.

[0084] The term "neural network dedicated processor" used in the present application can also be referred to as "neural network processor" or "NN processor" for short. The neural network dedicated processor can be implemented as a deep learning dedicated processor or a deep learning processor.

[0085] The storage device 130 can store a topology feature library 132 of the embodiments of the present application. The topology feature library (i.e., knowledge base) 132 is used to record the topology information of the neural network and its corresponding compilation optimization strategy. The topology feature library (i.e., knowledge base) 132 can employ a high-speed in-memory database, such as a Mango database, or can also employ other types of various databases. The storage device 130 can include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage device. The ROM can store static data or instructions required by the processor. The permanent storage device can be a read-write storage device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device employs a mass storage device (e.g., a magnetic disk or an optical disk, a flash memory) as a permanent storage device. The architecture of the neural network compilation server 100 described above is exemplary, and other various types of hardware architectures can also be included to implement the method of the embodiments of the present application.

[0086] The storage device 130 also stores computer program codes or computer program instructions, which, when processed by the neural network compilation server 100, can cause the neural network compilation server 100 to perform the neural network compilation optimization method described above.

[0087] In executing the method of the embodiments of the present application described above, the processor 110 is configured to perform the main steps of the method of the embodiments of the present application. In executing, for example, step S130, the step of performing the Hash algorithm (e.g., MD5) to encode the node attribute information is performed by a dedicated encryption chip 140 (e.g., a hard acceleration chip 140 in compliance with the national standard). The step of employing the GraphSAGE and the feedforward NN network in the method of the embodiments described above to generate the topology feature vector of the subgraph is performed by the neural network processor 120.

[0088] Figure 9 A structural schematic diagram of a computing device 200 according to the embodiments of the present application is shown, which can be used to implement the neural network compilation optimization method described above.

[0089] Referring to Figure 9 , the computing device 200 includes a processor 210 and a memory 220. The computing device of the present application can be various types of computing devices, including but not limited to various cloud computing devices, edge computing devices of various networks (e.g., wired or wireless communication networks, smart power networks, Internet of Things, etc.), user terminal devices, and the like.

[0090] The processor 210 can be a multi-core processor, or can include multiple processors. In some embodiments, the processor 210 can include a general-purpose main processor and one or more special-purpose co-processors, such as a graphics processor (GPU), a digital signal processor (DSP), a neural network processing unit (NPU), a tensor processing unit (TPU), etc. The memory 220 can include various types of storage units, such as a system memory, a read-only memory (ROM), and a permanent storage device. The ROM can store static data or instructions required by the processor 220. The permanent storage device can be a read-write storage device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (e.g., a magnetic disk or an optical disk, a flash memory) as the permanent storage device. In addition, in some embodiments, the permanent storage device can be a removable storage device (e.g., a floppy disk, an optical drive). The system memory can be a read-write storage device or a volatile read-write storage device, such as a dynamic random access memory. The system memory can store some or all of the instructions and data required by the processor during runtime. In addition, the memory 220 can include a combination of any computer-readable storage media, including various types of semiconductor storage chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks and / or optical disks. In some embodiments, the memory 220 can include a read-only and / or writeable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., an SD card, a minSD card, a Micro-SD card, etc.), a magnetic floppy disk, etc.

[0091] The computer program code or computer program instructions stored on the memory 220 can cause the processor 210 to perform the neural network compilation optimization method described above when the computer program code or computer program instructions are processed by the processor 210.

[0092] In addition, the method of the present application can also be implemented as a computer program or a computer program product, which includes computer program code instructions for performing the above steps defined in the above-mentioned method of the present application.

[0093] Alternatively, the application can also be implemented as a non-transitory machine readable storage medium (or computer readable storage medium, or machine readable storage medium) having stored thereon executable code (or computer program, or computer instruction code) that, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the various steps described in the above method according to the application.

[0094] Figure 10 A neural network compilation optimization apparatus 300 according to an embodiment of the application is shown, which can be used to implement the above-mentioned neural network compilation optimization method. The neural network compilation optimization apparatus 300 comprises a segmentation unit 310, a query unit 320 and an optimization unit 330.

[0095] In the execution of the method according to an embodiment of the application, the segmentation unit 310 is configured to segment a computation graph of a neural network into a plurality of subgraphs; the query unit 320 is configured to query whether there is an optimization strategy of a known subgraph in the topology feature library that is consistent with the topology of the at least one segmented subgraph; and the optimization unit 330 is configured to, if there is, extract an optimization strategy of a known topology from the topology feature library that is consistent with the topology of the at least one segmented subgraph to optimize the at least one subgraph.

[0096] Further, the topology feature library comprises topology feature vectors of known subgraphs and corresponding optimization strategies, and the query unit 320 is further configured to query whether there is an optimization strategy of a known subgraph that is consistent with the topology feature vector of the at least one segmented subgraph.

[0097] Further, if there is no optimization strategy of a known subgraph that is consistent with the topology feature vector of the at least one segmented subgraph, the optimization unit 330 is further configured to perform compilation optimization calculation on the at least one segmented subgraph; after the compilation optimization of the at least one segmented subgraph, generate an optimization strategy of the at least one segmented subgraph; and add the topology feature vector of the at least one segmented subgraph and its optimization strategy into the topology feature library in association.

[0098] Further, the optimization unit 330 is configured to, when performing compilation optimization calculation on the at least one segmented subgraph, determine an equivalent subgraph of the at least one segmented subgraph through optimization calculation, and replace the at least one segmented subgraph with the equivalent subgraph; and the optimization unit 330 is configured to, when performing the adding, store the topology feature vector of the at least one segmented subgraph and the topology feature vector of the equivalent subgraph in the topology feature library in association.

[0099] Further, the optimization unit 330 is configured to, when performing the compiling optimization calculation on the at least one segmented subgraph, determine, by the optimization calculation, that a plurality of parts in a subgraph can be merged to reduce the calculation amount or improve the calculation speed of the subgraph, and merge the plurality of parts; and the optimization unit 330 is configured to, when performing the adding, add the topology feature vector of the subgraph and the topology feature vector of the merged subgraph into the topology feature library in association.

[0100] Further, the optimization unit 330 is configured to, when performing the compiling optimization calculation on the at least one segmented subgraph, determine, by the optimization calculation, an equivalent constant of the subgraph, and replace the subgraph with the constant; and the optimization unit 330 is configured to, when performing the adding, add the topology feature vector of the subgraph and the constant into the topology feature library in association.

[0101] Further, the segmentation unit 310 is further configured to segment the computation graph into coarse-grained subgraphs by using a clustering technique.

[0102] Further, if there is no known optimization strategy of a subgraph consistent with the topology feature vector of the at least one segmented coarse-grained subgraph, the segmentation unit 310 is configured to segment the at least one segmented coarse-grained subgraph into a plurality of fine-grained subgraphs; the query unit 320 is configured to query the topology feature library to determine whether there is a known optimization strategy of a subgraph consistent with the topology feature vector of the at least one fine-grained subgraph; and if there is a known optimization strategy of a subgraph consistent with the topology feature vector of the at least one fine-grained subgraph, the optimization unit 330 is configured to extract the known optimization strategy of the subgraph consistent with the topology feature vector of the at least one fine-grained subgraph from the topology feature library to optimize the at least one fine-grained subgraph.

[0103] Further, if there is no known optimization strategy of a subgraph consistent with the topology feature vector of the at least one fine-grained subgraph, the optimization unit 330 is configured to perform a compiling optimization calculation on the at least one fine-grained subgraph, generate an optimization strategy of the at least one fine-grained subgraph, and add the topology feature vector of the at least one fine-grained subgraph and the optimization strategy thereof into the topology feature library.

[0104] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for neural network compilation optimization, the method comprising: The method comprises: segmenting a computation graph of a neural network into a plurality of coarse-grained subgraphs; querying a topology feature library to determine whether there is an optimization strategy of a known subgraph that is consistent with a topology of at least one segmented coarse-grained subgraph; if there is, extracting the optimization strategy of the known subgraph that is consistent with the topology of the at least one segmented coarse-grained subgraph from the topology feature library to optimize the at least one coarse-grained subgraph; if there is no optimization strategy of a known subgraph that is consistent with a topology feature vector of the at least one segmented coarse-grained subgraph, segmenting the at least one segmented coarse-grained subgraph into a plurality of fine-grained subgraphs, querying the topology feature library to determine whether there is an optimization strategy of a known subgraph that is consistent with a topology feature vector of at least one fine-grained subgraph; if there is an optimization strategy of a known subgraph that is consistent with a topology feature vector of at least one fine-grained subgraph, extracting the optimization strategy of the known subgraph that is consistent with the topology feature vector of the at least one fine-grained subgraph from the topology feature library to optimize the at least one fine-grained subgraph.

2. The method of claim 1, wherein, The topology feature library comprises topology feature vectors of known subgraphs and corresponding optimization strategies, and the querying step comprises querying whether there is an optimization strategy of a known subgraph that is consistent with a topology feature vector of the at least one segmented coarse-grained subgraph.

3. The method of claim 2, wherein, The method further comprises: if there is no optimization strategy of a known subgraph that is consistent with a topology feature vector of the at least one segmented fine-grained subgraph, performing a compilation optimization calculation on the at least one segmented coarse-grained subgraph; after the compilation optimization of the at least one segmented coarse-grained subgraph, generating an optimization strategy of the at least one segmented coarse-grained subgraph; and adding the topology feature vector of the at least one segmented coarse-grained subgraph and the optimization strategy thereof into the topology feature library in association.

4. The method of claim 3, wherein, Performing a compilation optimization calculation on the at least one segmented coarse-grained subgraph comprises determining, by the optimization calculation, an equivalent subgraph of the at least one segmented coarse-grained subgraph, and replacing the at least one segmented coarse-grained subgraph with the equivalent subgraph; and the adding step comprises storing the topology feature vector of the at least one segmented coarse-grained subgraph and the topology feature vector of the equivalent subgraph in the topology feature library in association.

5. The method of claim 3, wherein, Performing a compilation optimization calculation on the at least one segmented coarse-grained subgraph comprises determining that a plurality of parts in the at least one segmented coarse-grained subgraph can be merged to reduce the computation amount of the coarse-grained subgraph or to improve the computation speed of the coarse-grained subgraph, and merging the plurality of parts, and the adding step comprises adding the topology feature vector of the at least one segmented coarse-grained subgraph and the topology feature vector of the merged coarse-grained subgraph into the topology feature library in association.

6. The method of claim 3, wherein, Performing a compilation optimization calculation on the at least one segmented coarse-grained subgraph comprises determining an equivalent constant of the at least one segmented coarse-grained subgraph, and replacing the at least one segmented coarse-grained subgraph with the equivalent constant, and the adding step comprises adding the topology feature vector of the at least one segmented coarse-grained subgraph and the equivalent constant into the topology feature library in association.

7. The method according to any one of claims 1-3, characterized in that, The segmentation of the computation graph into a plurality of coarse-grained subgraphs comprises segmenting the computation graph into a plurality of coarse-grained subgraphs by using a clustering technique.

8. The method of claim 1, wherein, If there is no known subgraph optimization strategy consistent with the topology feature vector of the at least one fine-grained subgraph, the compiling optimization calculation is performed on the at least one fine-grained subgraph; the optimization strategy of the at least one fine-grained subgraph is generated; and the topology feature vector of the at least one fine-grained subgraph and the optimization strategy thereof are added to the topology feature library.

9. An apparatus for neural network compilation optimization, comprising: The apparatus comprises: The processor is configured to execute program instructions stored in the memory to enable the apparatus to implement the method of any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program codes which, when executed by a computer, implement the method of any one of claims 1 to 8.

11. A computer program product, characterised in that, The computer program product comprises program codes which, when executed by a computer, implement the method of any one of claims 1 to 8.

12. An apparatus for neural network compilation optimization, the apparatus comprising: The apparatus comprises: The partition unit is configured to partition a computation graph of a neural network into a plurality of coarse-grained subgraphs; The query unit is configured to query whether there is a known subgraph optimization strategy consistent with the topology of at least one partitioned coarse-grained subgraph in a topology feature library; The optimization unit is configured to, if there is, extract the optimization strategy of the known topology consistent with the topology of the at least one partitioned coarse-grained subgraph from the topology feature library to optimize the at least one coarse-grained subgraph; The partition unit is further configured to, if there is no known subgraph optimization strategy consistent with the topology feature vector of the at least one partitioned coarse-grained subgraph, partition the at least one partitioned coarse-grained subgraph into a plurality of fine-grained subgraphs; The query unit is further configured to query whether there is a known subgraph optimization strategy consistent with the topology feature vector of the at least one fine-grained subgraph in the topology feature library; The optimization unit is further configured to, if there is a known subgraph optimization strategy consistent with the topology feature vector of the at least one fine-grained subgraph, extract the optimization strategy of the known subgraph consistent with the topology feature vector of the at least one fine-grained subgraph from the topology feature library to optimize the at least one fine-grained subgraph.

13. The apparatus of claim 12, wherein, The topology feature library comprises topology feature vectors of known subgraphs and corresponding optimization strategies, and the query unit is further configured to generate the topology feature vector of the at least one partitioned coarse-grained subgraph, and query whether there is a known subgraph optimization strategy consistent with the topology feature vector of the at least one partitioned coarse-grained subgraph.

14. The apparatus of claim 13, wherein, If there is no known subgraph optimization strategy consistent with the topology feature vector of the at least one partitioned coarse-grained subgraph, the optimization unit is further configured to perform a compiling optimization calculation on the at least one partitioned coarse-grained subgraph; generate the optimization strategy of the at least one partitioned coarse-grained subgraph after the compiling optimization of the at least one partitioned coarse-grained subgraph; and add the topology feature vector of the at least one partitioned coarse-grained subgraph and the optimization strategy thereof into the topology feature library in association.

15. The apparatus of claim 14, wherein, The optimization unit is configured to, when performing the compiling optimization calculation on the at least one partitioned coarse-grained subgraph, determine an equivalent subgraph of the at least one partitioned coarse-grained subgraph by optimization calculation, and replace the at least one partitioned coarse-grained subgraph with the equivalent subgraph. The optimization unit is configured to, when performing the adding, store the topological feature vector of the at least one partitioned coarse-grained subgraph and the topological feature vector of the equivalent subgraph in the topological feature library in association.

16. The apparatus of claim 14, wherein, The optimization unit is configured to, when performing the compiling optimization calculation on the at least one partitioned coarse-grained subgraph, determine that a plurality of parts in the coarse-grained subgraph can be merged to reduce the calculation amount of the coarse-grained subgraph or improve the calculation speed of the coarse-grained subgraph, and merge the plurality of parts, The optimization unit is configured to, when performing the adding, add the topological feature vector of the coarse-grained subgraph and the topological feature vector of the merged coarse-grained subgraph to the topological feature library in association.

17. The apparatus of claim 14, wherein, The optimization unit is configured to, when performing the compiling optimization calculation on the at least one partitioned coarse-grained subgraph, perform the compiling optimization calculation on the coarse-grained subgraph includes determining an equivalent constant of the coarse-grained subgraph, and replacing the coarse-grained subgraph with the constant. The optimization unit is configured to, when performing the adding, add the topological feature vector of the coarse-grained subgraph and the constant to the topological feature library in association.

18. The apparatus of any one of claims 12-14, wherein, The partition unit is further configured to partition the computation graph into coarse-grained subgraphs using a clustering technique.

19. The apparatus of claim 12, wherein, If there is no optimization strategy of a known subgraph consistent with the topological feature vector of the at least one fine-grained subgraph, the optimization unit is configured to perform the compiling optimization calculation on the at least one fine-grained subgraph, generate an optimization strategy of the at least one fine-grained subgraph, and add the topological feature vector of the at least one fine-grained subgraph and the optimization strategy thereof to the topological feature library.

Images