Neural network model compilation method and device, electronic device, and storage medium

Abstract

A method and apparatus for compiling a neural network model, an electronic device, and a storage medium are disclosed, wherein the method for compiling a neural network model includes the steps of: obtaining a neural network model to be compiled; determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the initial computation graph including at least one first computation subgraph; determining second computation subgraphs from each first computation subgraph including preset operator nodes and dividing the second computation subgraphs into a first number of target computation subgraphs based on input tensors of the second computation subgraphs; establishing dependencies among the first number of target computation subgraphs based on preset dependency rules to obtain first dependency graphs corresponding to the second computation subgraphs; and generating a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graphs corresponding to each second computation subgraph. This method helps to improve the effectiveness of parallelism of the instruction sequence, thereby improving the efficiency of instruction scheduling.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This disclosure claims priority to a Chinese patent application filed on December 20, 2022, bearing application number CN202211659819.X and entitled "Method and apparatus for compiling neural network models, electronic devices, and storage media," the entire contents of which are incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to artificial intelligence technology, and in particular to a method and apparatus for compiling a neural network model, an electronic device, and a storage medium. [Background technology]

[0003] With the rapid development of artificial intelligence (AI) technology, there is a growing demand for faster training and inference processing speeds for neural network models in applications such as autonomous driving. To shorten the training and inference time, AI chips have certain needs for neural network compilers. Typically, during the compilation process of a neural network model, the input tensors of the operator nodes in the computation graph of the neural network model need to be divided and distributed to multiple computation units for execution. Summary of the Invention [Problem to be solved by the invention]

[0004] Embodiments of the present disclosure provide a method and apparatus for compiling a neural network model, an electronic device, and a storage medium. [Means for solving the problem]

[0005] According to one aspect of an embodiment of the present disclosure, there is provided a method for compiling a neural network model, the method including: obtaining a neural network model to be compiled; determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the initial computation graph including at least one first computation subgraph, each of the first computation subgraphs including at least one operator node and a connection relationship between each of the operator nodes; determining second computation subgraphs including preset operator nodes from each of the first computation subgraphs, and dividing the second computation subgraphs into a first number of target computation subgraphs based on an input tensor of the second computation subgraph; establishing dependencies between the first number of target computation subgraphs based on preset dependency rules, and obtaining a first dependency graph corresponding to the second computation subgraph, the dependency relationships being used to represent a computation order between each of the target computation subgraphs; and generating a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graphs respectively corresponding to each of the second computation subgraphs.

[0006] According to another aspect of the embodiment of the present disclosure, there is provided a first processing module for determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the initial computation graph including at least one first computation subgraph, each of the first computation subgraphs including at least one operator node and a connection relationship between each of the operator nodes; and a first processing module for determining a second computation subgraph including preset operator nodes from each of the first computation subgraphs, based on an input tensor of the second computation subgraph. a third processing module used to establish dependencies between the first number of target computational subgraphs based on a preset dependency rule to obtain a first dependency graph corresponding to the second computational subgraphs, where the dependencies characterize a computation order between the target computational subgraphs; and a fourth processing module used to generate a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graphs corresponding to each of the second computational subgraphs.

[0007] According to a further aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program for executing the method for compiling a neural network model according to any of the above embodiments of the present disclosure.

[0008] According to a further aspect of an embodiment of the present disclosure, there is provided an electronic device including a processor and a memory for storing instructions executable by the processor, the processor being adapted to read and execute the executable instructions from the memory to implement the method for compiling a neural network model described in any of the above embodiments of the present disclosure.

[0009] According to a further aspect of an embodiment of the present disclosure, there is provided a computer program product, the computer program product comprising instructions that, when executed by a processor, perform the method for compiling a neural network model as set forth in any of the above embodiments of the present disclosure. [Effects of the Invention]

[0010] According to the neural network model compilation method and device, electronic device, and storage medium provided by the above embodiments of the present disclosure, after dividing a second computation subgraph including a preset operator node in an initial computation graph corresponding to a neural network model to be compiled, a dependency relationship is established between target computation subgraphs obtained by the division to control the computation order of the target computation subgraph, thereby enabling the computation of the target computation subgraph to achieve both parallelism and resource occupancy, and the generated instruction sequence can perform instruction scheduling with an effective parallelism, thereby allowing the resource occupancy of instruction scheduling to reserve usable memory resources for subsequent processing, which helps to reduce the occurrence of a situation where the parallelism of the target computation subgraph is excessive, resulting in pre-occupation of resources and a lack of resources for subsequent processing of the target computation subgraph, and helps to improve the efficiency of instruction scheduling.

[0011] The technical solution of the present disclosure will be further described in detail below through the drawings and examples. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is an exemplary application scenario of the neural network model compilation method provided by the present disclosure. [Figure 2] 1 is a flowchart of a method for compiling a neural network model provided by one exemplary embodiment of the present disclosure. [Figure 3] FIG. 2 is a schematic diagram of a first computation subgraph provided by one exemplary embodiment of the present disclosure. [Figure 4] FIG. 10 is a schematic diagram of a division principle of a second computation subgraph provided by an exemplary embodiment of the present disclosure; [Figure 5] 10 is a flowchart of a method for compiling a neural network model provided by another exemplary embodiment of the present disclosure. [Figure 6] FIG. 2 is a schematic diagram of a first dependency graph provided by one exemplary embodiment of the present disclosure. [Figure 7] 10 is a flowchart of a method for compiling a neural network model provided by a further exemplary embodiment of the present disclosure. [Figure 8] FIG. 10 is a schematic diagram of a first dependency graph provided by another exemplary embodiment of the present disclosure. [Figure 9] FIG. 2 is a schematic diagram of a scheduling sliding window provided by one exemplary embodiment of the present disclosure. [Figure 10] 10 is a flowchart of a method for compiling a neural network model provided by a further exemplary embodiment of the present disclosure. [Figure 11] FIG. 1 is a schematic diagram of a division principle corresponding to a matrix multiplication operation type provided by one exemplary embodiment of the present disclosure. [Figure 12] 1 is a schematic structural diagram of a neural network model compilation device provided by one exemplary embodiment of the present disclosure; FIG. [Figure 13] 1 is a schematic structural diagram of a neural network model compilation device provided by one exemplary embodiment of the present disclosure; FIG. [Figure 14] 1 is a schematic structural diagram of an application example of an electronic device of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] In order to explain the present disclosure, exemplary embodiments of the present disclosure will be described in detail below with reference to the drawings. Obviously, it should be understood that the described embodiments are only some embodiments of the present disclosure, not all embodiments, and the present disclosure is not limited to the exemplary embodiments.

[0014] It should be noted that the relative arrangement of parts and steps, formulas and numerical values ​​described in these examples do not limit the scope of the present disclosure unless specifically stated otherwise.

[0015] Summary of the Disclosure In the process of realizing the present disclosure, the inventors discovered that with the rapid development of artificial intelligence (AI) technology, there are increasing requirements for the training or inference processing speed of neural network models in scenarios such as autonomous driving. In order to shorten the training or inference time, AI chips have certain needs for neural network compilers. Typically, in the compilation process of a neural network model, the input tensors of the operator nodes in the computation graph of the neural network model need to be divided and distributed to multiple computation units for execution. If the number of computation subgraphs after division is large, during instruction scheduling, the instruction sequences of all computation subgraphs may be scheduled in parallel, which will result in excessive parallelism in the data loading stage of each computation subgraph, causing resources to be occupied in advance and resulting in a shortage of processing resources for subsequent stages of each computation subgraph, thereby affecting the efficiency of instruction scheduling.

[0016] Illustrative Overview FIG. 1 is an exemplary application scenario of the neural network model compilation method provided by the present disclosure.

[0017] In an autonomous driving scenario, the neural network model compilation method (executed by a neural network model compilation device) of the present disclosure can be used to determine an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the initial computation graph can include at least one first computation subgraph, and each first computation subgraph can include at least one operator node and a connection relationship between each operator node, a second computation subgraph including preset operator nodes in each first computation subgraph can be determined based on the initial computation graph, and a second computation subgraph can be determined based on an input tensor of the second computation subgraph. The subgraph can be divided into a first number of target computation subgraphs, and dependencies between the first number of target computation subgraphs are established based on preset dependency rules to obtain a first dependency graph corresponding to a second computation subgraph, where the dependencies can be used to characterize the computation order between each target computation subgraph. Furthermore, based on the first dependency graphs corresponding to each second computation subgraph, a target instruction sequence corresponding to the neural network model to be compiled can be generated and used in the autonomous driving AI chip to perform model inference for corresponding functions, such as object detection and semantic segmentation, and the obtained inference results can be used for autonomous driving decision-making and planning. Since it is used to establish dependencies between target computation subgraphs and control the computation order of the target computation subgraphs, the computation of the target computation subgraphs can achieve both parallelism and resource occupancy, and the generated instruction sequence can perform instruction scheduling with an effective parallelism, so that the resource occupancy of the instruction scheduling can reserve available memory resources for subsequent processing, which helps to reduce the occurrence of a situation where the parallelism of the target computation subgraphs is too high and resources are occupied in advance, resulting in a shortage of resources for subsequent processing of the target computation subgraphs, and helps to improve the efficiency of instruction scheduling.

[0018] The neural network model compilation method provided by the embodiments of the present disclosure is not limited to application to autonomous driving scenarios, but can also be applied to any other scenarios that require input tensor division, such as image recognition-based access control system scenarios and medical image processing scenarios that require image processing, and can be specifically configured according to actual needs.

[0019] Exemplary Methods 2 is a flowchart of a neural network model compilation method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, specifically, servers or terminals, and includes the following steps, as shown in FIG.

[0020] In step 201, a neural network model to be compiled is obtained.

[0021] Here, the neural network model to be compiled (which may be abbreviated as "model" in the embodiments of the present disclosure) may be any neural network model obtained by pre-training, such as an object detection model or a semantic segmentation model, and is not limited to this in the embodiments of the present disclosure.

[0022] In one alternative example, step 201 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a first acquisition module executed by the processor.

[0023] In step 202, based on the neural network model to be compiled, an initial computation graph corresponding to the neural network model to be compiled is determined, where the initial computation graph includes at least one first computation subgraph, and each first computation subgraph includes at least one operator node and a connection relationship between each operator node.

[0024] Here, the operator nodes may include data load nodes, data computation nodes, and data storage nodes, where the data load nodes are used to load data used in the data computation nodes, such as tensors and weights, the data computation nodes may include various computation operators in the model, such as matrix multiplication, convolution, corresponding element addition, pooling, and corresponding element multiplication, and the data storage nodes are nodes for storing the computation results of the data computation nodes. The first computation subgraph is a graphical representation describing the computation order of a set of operator nodes composed of the data load nodes, data computation nodes, and data storage nodes, and the initial computation graph may include an overall graphical representation of the first computation subgraph in the neural network model to be compiled. For example, one first computation subgraph may correspond to any computation operator (e.g., convolution) in the neural network model to be compiled, and include a data load node corresponding to the computation operator, a data computation node corresponding to the computation operator, and a data storage node corresponding to the computation operator.

[0025] 3 is a schematic diagram of a first computation subgraph provided by one exemplary embodiment of the present disclosure, where Load Tensor A represents loading tensor A and belongs to a data load node, Load Tensor B represents loading tensor B and belongs to a data load node, Add represents adding two tensors element by element and belongs to a data computation node, Relu represents an activation function that activates the element by element addition result to obtain tensor C (Tensor C) and also belongs to a data computation node, and Store Tensor C represents storing tensor C and belongs to a data storage node.

[0026] In one alternative example, step 202 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a first processing module executed by the processor.

[0027] In step 203, a second computation subgraph including a preset operator node is determined from each first computation subgraph, and the second computation subgraph is divided into a first number of target computation subgraphs based on the input tensor of the second computation subgraph.

[0028] Here, the preset operator nodes may be operator nodes that can divide tensors based on computational characteristics and can be set according to actual needs. For example, the preset operator nodes can include an element-wise addition operator node, a matrix multiplication operator node, a convolution operator node, a pooling operator node, a corresponding element multiplication operator node, etc. A first computation subgraph including the preset operator nodes is defined as a second computation subgraph. The first number can be set according to actual needs. For example, for element-wise addition operator nodes and element-wise multiplication operator nodes, the first number can be determined based on the availability of computation units available for the computation in the AI ​​chip. For example, the first number can be set to 2, 4, 8, etc. The target computation subgraph matches the overall logic of the first computation subgraph, but the size of the computed tensor is different.

[0029] 4 is a schematic diagram of the division principle of a second computation subgraph provided by one exemplary embodiment of the present disclosure. This example still takes an element-wise addition operator node as an example. When the first number is 4, the second computation subgraph is divided into four target computation subgraphs. Here, tensor A is divided according to the dashed lines into four subtensors A1, A2, A3, and A4, and tensor B is divided into four subtensors B1, B2, B3, and B4. According to the calculation characteristics of element-wise addition, tensor A and tensor B have the same size, and they need to be divided according to the same division scheme, so that the divided subtensors Ai and Bi belong to the same positions in A and B, respectively, where i=1, 2, 3, 4, and so that the divided subtensors Ai and Bi have the same size. The tensor C obtained by the calculation of tensor A and tensor B is divided into four subtensors, C1, C2, C3, and C4. The division of tensor C also needs to be divided according to the division method of tensor A and tensor B based on the calculation characteristics of element-wise addition. Each target calculation subgraph completes the element-wise addition calculation of two subtensors (e.g., A1 and B1) at the corresponding position, and obtains the result tensor (C1) at the corresponding position after activation.

[0030] In one possible example, different operator nodes may have different conditions that need to be met when splitting tensor A, tensor B, and tensor C due to differences in their computational characteristics. For example, the matrix multiplication operator node has different computational characteristics from the above-mentioned element-wise addition operator node. Matrix multiplication requires the elements of each row in tensor A to be multiplied by the elements of each column in tensor B to achieve the sum. Therefore, tensor A can be split according to rows, and tensor B can be split according to columns, and the number of rows in the subtensors after splitting tensor A is the same as the number of columns in the subtensor of tensor B. Tensor C can be split based on the corresponding positions in tensor C of the operation results of the subtensors after splitting tensors A and B. This can be specifically set according to actual needs.

[0031] In one alternative example, step 203 may be performed by the processor calling corresponding instructions stored in memory, or may be performed by a second processing module executed by the processor.

[0032] In step 204, a dependency relationship between a first number of target computation subgraphs is established based on a preset dependency rule, and a first dependency graph corresponding to a second computation subgraph is obtained, and the dependency relationship is used to characterize the computation order between each target computation subgraph.

[0033] Here, the preset dependency rules can be set according to actual needs and are not specifically limited as long as they establish dependencies between each target computation subgraph. The dependencies between each target computation subgraph characterize the computation order between each target computation subgraph, thereby helping instruction scheduling avoid the pitfalls of greedy methods due to excessive pseudo-parallelism. For example, the goal of instruction scheduling in the related art is to achieve higher parallelism under limited resource conditions. Without establishing a computation order between each target computation subgraph, during instruction scheduling, loading tensors A1, A2, A3, A4, B1, B2, B3, and B4 may not have data dependencies and can be executed in advance. Performing these operations prematurely occupies high-speed memory resources, causing the scheduler to spend a lot of time identifying these invalid attempts, significantly reducing scheduling speed. Furthermore, the premature occupancy of memory resources is likely to result in a shortage of memory resources for subsequent processing. By establishing dependencies between target computation subgraphs, embodiments of the present disclosure can help reduce the premature execution of invalid operations, thereby improving scheduling efficiency.

[0034] For example, the calculation order between each target computation subgraph can be determined randomly, or the calculation order between each target computation subgraph can be determined based on other rules, such as limiting the parallel processing of only partial target computation subgraphs at the same time, for example, allowing the parallel processing of two target computation subgraphs, and allowing a new target computation subgraph to start loading data after one target computation subgraph is completed, thereby avoiding multiple target computation subgraphs loading data in parallel and occupying resources. Specific rules are not limited.

[0035] In one alternative example, step 204 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a third processing module executed by the processor.

[0036] In step 205, a target instruction sequence corresponding to the neural network model to be compiled is generated based on the first dependency graphs respectively corresponding to each second computation sub-graph.

[0037] Here, the target instruction sequence is a compiled binary code sequence corresponding to the neural network model to be compiled, and when the target instruction sequence is deployed to the AI ​​chip, it can realize the corresponding inference function of the neural network model to be compiled, such as object detection, semantic segmentation, etc. The AI ​​chip can perform corresponding instruction scheduling and execution based on the target instruction sequence to realize the corresponding function.

[0038] In one alternative example, step 205 may be performed by the processor calling corresponding instructions stored in memory, or may be performed by a fourth processing module executed by the processor.

[0039] The neural network model compilation method provided in this embodiment divides a second computation subgraph including a preset operator node in an initial computation graph corresponding to the neural network model to be compiled, and then establishes dependencies between target computation subgraphs obtained by the division and controls the computation order of the target computation subgraph, so that the computation of the target computation subgraph can achieve both parallelism and resource occupancy, and the generated instruction sequence can perform instruction scheduling with an effective parallelism, thereby allowing the resource occupancy of instruction scheduling to reserve usable memory resources for subsequent processing, which helps to reduce the occurrence of a situation where the parallelism of the target computation subgraph is too high and resources are occupied in advance, resulting in a shortage of resources for subsequent processing of the target computation subgraph, and helps to improve the efficiency of instruction scheduling.

[0040] FIG. 5 is a flowchart of a method for compiling a neural network model provided by another exemplary embodiment of the present disclosure.

[0041] In one alternative embodiment, step 204 may specifically include the following steps:

[0042] In step 2041, a computation priority level corresponding to each target computation subgraph is determined based on a preset rule.

[0043] Here, the preset rules can be set according to actual needs, for example, the calculation priority level corresponding to each target calculation subgraph can be determined randomly, or can be determined according to the order of the subtensors of the divided tensor, for example, the calculation priority level of the target calculation subgraph corresponding to A1 and B1 above is the highest, the calculation priority level of the target calculation subgraph corresponding to A2 and B2 is the next highest, and so on. Specific examples are not limited.

[0044] In one alternative example, step 2041 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a first decision unit executed by a processor.

[0045] In step 2042, based on the computation priority level of each target computation subgraph, a first dependency edge is established between the same corresponding operator nodes in any two adjacent target computation subgraphs, and the first dependency edge is directed from the operator node of the first target computation subgraph with a higher computation priority level to the operator node of the second target computation subgraph with a lower computation priority level in the two target computation subgraphs.

[0046] Here, the first dependency edge characterizes the dependency of the computation order between the same corresponding operator nodes in two adjacent target computation subgraphs, and the instruction of the operator node pointed to by the first dependency edge can be executed only after the instruction corresponding to the start node of the first dependency edge is executed.

[0047] In one alternative example, step 2042 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a first processing unit executed by a processor.

[0048] In step 2043, for any target computation subgraph, a second dependency edge is established between each operator node in the target computation subgraph based on the preset computation logic between each operator node in the target computation subgraph.

[0049] Here, for any target computation subgraph, referring to FIG. 4, there can be a certain computation logic between each operator node, and the second dependency edge is used to characterize the dependency of the computation order between each operator node in the same target computation subgraph, and the principle is similar to that of the first dependency edge.

[0050] In one alternative example, step 2043 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a second processing unit executed by the processor.

[0051] In step 2044, a first dependency graph is determined based on each target computation subgraph, the first dependency edges between the same corresponding operator nodes in any two adjacent target computation subgraphs, and the second dependency edges between each operator node in each target computation subgraph.

[0052] Here, the first dependency graph may include dependencies between adjacent target computation subgraphs in each target computation subgraph into which the second computation subgraph corresponding to the first dependency graph is divided, and dependencies within each target computation subgraph.

[0053] 6 is a schematic diagram of a first dependency graph provided by one exemplary embodiment of the present disclosure. This example still takes the aforementioned element-wise addition operator node as an example, where dashed arrows represent first dependency edges between two adjacent target computation subgraphs, and solid arrows represent second dependency edges within the target computation subgraphs.

[0054] In one alternative example, step 2044 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a third processing unit executed by the processor.

[0055] This embodiment characterizes the dependency of the calculation order between each operator node by establishing a first dependency edge between each target computation subgraph and a second dependency edge within the target computation subgraph, which helps to avoid the occurrence of scheduling situations with a high degree of pseudo-parallelism, thereby greatly improving the efficiency of instruction scheduling during instruction scheduling.

[0056] FIG. 7 is a flowchart of a method for compiling a neural network model provided by a further exemplary embodiment of the present disclosure.

[0057] In one alternative embodiment, step 2044 of determining a first dependency graph based on each target computation subgraph, the first dependency edges between corresponding same operator nodes in any two adjacent target computation subgraphs, and the second dependency edges between each operator node in each target computation subgraph includes the following steps:

[0058] In step 20441, a first weight corresponding to each first dependency edge between each target computation subgraph and a second weight corresponding to each second dependency edge within each target computation subgraph are determined based on a preset weight rule.

[0059] Here, the preset weight rule can be set according to actual needs, for example, a large weight can be set to the first dependent edge, for example, the first weight can be set to 100, and a small weight can be set to the second dependent edge, for example, the second weight can be set to 1, and specifically, can be set according to actual needs.

[0060] In one alternative example, step 20441 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a third processing unit executed by the processor.

[0061] In step 20442, a node weight of each operator node is determined based on a first weight corresponding to each first dependency edge and a second weight corresponding to each second dependency edge, and the node weight is used to characterize a calculation priority level of the corresponding operator node.

[0062] Here, for any second computation subgraph, based on the directions of all first and second dependency edges, the node weight of the final pointing end operator node can be set to a reference value (e.g., 0), and based on the first or second dependency edge pointing to the end operator node, the node weight of the starting operator node to which the first or second dependency edge connects can be determined, and so on until the node weights of all operator nodes are obtained.

[0063] In one alternative example, step 20442 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a third processing unit executed by the processor.

[0064] In step 20443, a first dependency graph is determined based on each target computation subgraph, each first dependency edge, each first weight corresponding to each first dependency edge, each second dependency edge, each second weight corresponding to each second dependency edge, and each node weight.

[0065] 8 is a schematic diagram of a first dependency graph provided by another exemplary embodiment of the present disclosure, where starting from the operator node Store Tensor C4 at the bottom right, its node weight (gray number) can be set to a reference value of 0, the first weight corresponding to the first dependency edge is set to 100, and the second weight corresponding to the second dependency edge is set to 1, and the node weights of other operator nodes can be obtained by adding the first weights of the first dependency edges that depend on them and the node weights of the operator nodes to which the first dependency edges point, or by adding the second weights of the second dependency edges that depend on them and the node weights of the operator nodes to which the second dependency edges point. For example, the node weight of operator node Store Tensor C3 is the sum of the node weight of Store Tensor C4 and the first weight of the first dependency edge between them, so the node weight of Store Tensor C3 is 100. Also, for example, the node weight of operator node Add in target computation subgraph 3 is the sum (102) of the node weight (101) of its dependent operator node Relu and the second weight (1) of the second dependency edge between them, so the node weight of Add is also the sum (102) of the node weight (2) of operator node Add in target computation subgraph 4 that depends on it and the first weight (100) of the first dependency edge between them. By analogy, the node weights of each operator node can be obtained as shown in the figure. In actual applications, other first and second weights can be adopted, or node weights can be determined based on other node weight rules, as long as the computation order between each operator node can be represented according to needs, and specifically, can be set according to actual needs.

[0066] In one possible example, the first weight of each first dependency edge can also be set to a different or partially different value, for example, the first weight of each first dependency edge between target computation subgraph 1 and target computation subgraph 2 can be different from the first weight of each first dependency edge between target computation subgraph 2 and target computation subgraph 3, and specifically can be set according to actual needs.

[0067] In one possible example, the second weight of each second dependency edge can also be set to a different or partially different value, for example, the second weight of the second dependency edge of the target calculation subgraph 1 can be different from the second weight of the second dependency edge of the target calculation subgraph 2, and specifically can be set according to actual needs.

[0068] In one alternative example, when determining the node weight of another operator node other than the operator node set to the reference value, for any other operator node, the result of adding a first weight of a first dependency edge that depends on the other operator node and the node weight of the operator node to which the first dependency edge is directed can be a first result, and the result of adding a second weight of a second dependency edge that depends on the other operator node and the node weight of the operator node to which the second dependency edge is directed can be a second result, and if the first result and the second result are different, the maximum value of the first result and the second result can be the node weight of the other operator node. For example, when the second weight of the second dependency edge of the target computation subgraph 4 in FIG. 8 is set to 1 and the second weight of the second dependency edge of the target computation subgraph 3 is set to 2, the node weight of the operator node Add in the target computation subgraph 3 is the sum of the node weight (102) of the operator node Relu that depends on it and the second weight (2) of the second dependency edge between them, which is 104, and the sum of the node weight (2) of the operator node Add in the target computation subgraph 4 that depends on the operator node Add and the first weight (100) of the first dependency edge between them is 102, and since 102 is smaller than 104, the node weight of the operator node Add in the target computation subgraph 3 can be determined to be 104.

[0069] In one alternative example, step 20443 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a third processing unit executed by the processor.

[0070] In this embodiment, to address the unfairness problem that may exist in the dependency graph of FIG. 6, the node weights corresponding to the operator nodes are determined based on the first weight corresponding to the first dependency edge and the second weight corresponding to the second dependency edge, thereby helping to control the parallel execution status of adjacent target computation subgraphs. Here, the unfairness problem refers to, for example, if TensorA is relatively small, TensorB is relatively large, and the calculation speed of Add is slow, when executing Add in the first target computation subgraph, the scheduler will face the problem of whether to load Tensors A2, A3, and A4 in idle time, and even whether to load Tensors B2, B3, and B4. Obviously, these Tensors will be used after a long time has passed. For example, A4 and B4 will only be used when calculating the fourth Add, so there is still a possibility that resources will be pre-occupied. Referring to FIG. 8, by setting the first weight of the first dependency edge between target computation subgraphs to a large value, the node weights of the operator nodes of adjacent target computation subgraphs are at different levels (e.g., 200 level and 300 level), thereby controlling the number of levels that can be executed in parallel or controlling the node weight threshold to limit the number of target computation subgraphs that can be executed in parallel, which helps to avoid the above-mentioned possible unfairness problem and further improves scheduling efficiency.

[0071] In one alternative embodiment, the step 205 of generating a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graphs respectively corresponding to each second computation sub-graph includes the following steps:

[0072] In step 2051, for each second computation subgraph, a scheduling sliding window corresponding to the second computation subgraph is determined based on the first dependency graph corresponding to the second computation subgraph, and the scheduling sliding window is for accommodating a second number of target computation subgraphs.

[0073] Here, the second number can be set according to actual needs, for example, 2, 3, etc. The scheduling sliding window is used to control the number of target computation subgraphs that can be scheduled in parallel at the same time. For example, if there are four target computation subgraphs, if the second number is set to 2, only two target computation subgraphs can be processed in parallel at the same time. After the instruction scheduling corresponding to the first target computation subgraph in the scheduling sliding window is completed, the sliding window slides the completed target computation subgraph out of the window, and at the same time slides one subsequent unprocessed target computation subgraph into the window. Based on this, it is possible to ensure a certain degree of parallelism and avoid the occurrence of a situation of pseudo-over-parallelism, thereby further improving the efficiency of instruction scheduling.

[0074] 9 is a schematic diagram of a scheduling sliding window provided by an exemplary embodiment of the present disclosure. The scheduling sliding window (solid-line rectangular frame) at a certain time includes target computation subgraph 1 and target computation subgraph 2. When target computation subgraph 1 is completed, the window slides to the position of the dashed-line rectangular frame. At this time, the scheduling sliding window includes target computation subgraph 2 and target computation subgraph 3. The sliding of the window can be controlled by the node weight threshold. For example, the initial node weight threshold is set to a value between 100 and 200, for example, 150, and the sliding of the window can be achieved by decreasing the node weight threshold by 100 each time. This can be specifically set according to actual needs.

[0075] In one alternative example, step 2051 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a second decision unit executed by the processor.

[0076] In step 2052, a first instruction sequence corresponding to the second computation sub-graph is generated based on the first dependency graph corresponding to the second computation sub-graph and the scheduling sliding window.

[0077] Here, the first instruction sequence is a binary instruction sequence that can be executed by hardware (e.g., an AI chip) corresponding to the second computation subgraph. The specific generation can be realized by compiling with a corresponding compiler, and the description thereof is omitted here.

[0078] In one alternative example, step 2052 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a fourth processing unit executed by the processor.

[0079] In step 2053, a target instruction sequence is determined based on the first instruction sequences corresponding to each second computation subgraph.

[0080] Here, since the initial computation graph corresponding to the neural network model to be compiled usually includes multiple first computation subgraphs, in addition to each of the second computation subgraphs mentioned above, there are other first computation subgraphs that do not include preset operator nodes, and the first instruction sequences corresponding to each of the second computation subgraphs can determine the target instruction sequence together with the instruction sequences of the other first computation subgraphs according to the logic position of each second computation subgraph in the entire computation graph.

[0081] In one alternative example, step 2053 may be performed by the processor calling up corresponding instructions stored in a memory, or may be performed by a third decision unit executed by the processor.

[0082] This embodiment helps to reduce the occurrence of pseudo-over-parallelism situations while guaranteeing a certain degree of parallelism by controlling the number of target computation subgraphs that can be scheduled in parallel at the same time through a scheduling sliding window, thereby further improving the efficiency of instruction scheduling.

[0083] FIG. 10 is a flowchart of a method for compiling a neural network model provided by a further exemplary embodiment of the present disclosure.

[0084] In one alternative embodiment, after step 202 of determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the method further includes step 301 of generating, for third computation subgraphs other than each second computation subgraph among each first computation subgraph, a second instruction sequence corresponding to each third computation subgraph based on each third computation subgraph.

[0085] Here, since the second computation subgraph is the first computation subgraph that includes the preset operator node, there may be other first computation subgraphs that do not include the preset operator node. This part of the first computation subgraph is taken as the third computation subgraph, and a corresponding second instruction sequence is generated based on the third computation subgraph. Specifically, it can be generated based on the computation logic between each operator node in the third computation subgraph, and the description is omitted here.

[0086] In one alternative example, step 301 may be performed by the processor calling corresponding instructions stored in memory, or may be performed by a fifth processing module executed by the processor.

[0087] Step 2053 of determining a target instruction sequence based on a first instruction sequence corresponding to each second computation subgraph includes step 20531 of determining a target instruction sequence based on a first instruction sequence corresponding to each second computation subgraph and a second instruction sequence corresponding to each third computation subgraph.

[0088] Specifically, instruction scheduling can be performed on each first instruction sequence and each second instruction sequence based on the computation logic between each second computation subgraph and each third computation subgraph to determine a target instruction sequence.

[0089] In one alternative example, step 2053 may be performed by the processor calling up corresponding instructions stored in a memory, or may be performed by a third decision unit executed by the processor.

[0090] In this embodiment, by determining a target instruction sequence based on each first instruction sequence and each second instruction sequence, when the target instruction sequence is scheduled and executed on a hardware platform, under the condition that it has a certain degree of parallelism, the situation of excessive pseudo-parallelism can be reduced, which helps to significantly improve the efficiency of instruction scheduling.

[0091] In one possible example, the compilation flow of the neural network model may include three parts: computation graph construction, computation graph optimization, and instruction generation, where the computation graph construction is constructing a general-purpose first intermediate representation based on an input model file, the first intermediate representation being in a graphical form and corresponding to the above-mentioned initial computation graph in the embodiment of the present disclosure, the computation graph optimization is performing graph optimization on the first intermediate representation to obtain a second intermediate representation in a graphical form, corresponding to the flow of generating a dependency graph based on the initial computation graph in the embodiment of the present disclosure, and the instruction generation is performing scheduling optimization on the second intermediate representation to obtain a fine-grained third intermediate representation and compiling the third representation into instruction codes to be executed on a hardware platform, corresponding to the flow of generating a target instruction sequence based on the first dependency graph corresponding to each second computation subgraph in the embodiment of the present disclosure. The specific compiler architecture is not limited.

[0092] In one alternative embodiment, the step 202 of determining, based on the neural network model to be compiled, an initial computation graph corresponding to the neural network model to be compiled includes the following steps:

[0093] In step 2021, the neural network model to be compiled is analyzed to obtain the analysis results.

[0094] The analysis of the neural network model to be compiled may be performed based on predefined analysis rules or by using any available analyzer, and is not limited to these. For example, the syntax and semantics of the model file of the neural network model to be compiled may be analyzed based on the analysis rules, and the syntax and semantics may be used as the analysis result. The model file may include a large number of computer programming languages ​​and may be applied to target identification, image recognition, image classification, semantic segmentation, speech recognition, and the like. The computer coding language may be a dedicated coding language or a general-purpose coding language. For example, in the embodiments of the present disclosure, the computer programming language may be a neural network-specific programming language, which may be specifically configured according to actual needs. The neural network-specific programming language may be characterized as program code that characterizes the neural network based on a description of a computation graph, and the program code may include program statements that describe the computation graph.

[0095] In one alternative example, step 2021 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by an analysis unit executed by a processor.

[0096] In step 2022, an initial computation graph corresponding to the neural network model to be compiled is constructed based on the analysis results.

[0097] Here, since the program code of the neural network model to be compiled includes program statements describing a computation graph, a computation graph structure corresponding to the initial computation graph can be constructed based on the syntax and semantics obtained by analyzing the program statements of the neural network model to be compiled, and the initial computation graph can be obtained based on the computation graph structure. The initial computation graph may be a nested structure diagram including a complex template, and is consistent with the semantics in the program code of the neural network model to be compiled. An expansion process can be performed on the computation graph obtained based on the computation graph structure to obtain a computation graph formed by connecting basic units in the program to each other as the initial computation graph. The initial computation graph can include the complete execution process of the program and can be translated into an executable instruction sequence. The embodiments of the present disclosure perform an optimization process according to the method of the embodiments of the present disclosure based on the initial computation graph to obtain a computation graph structure (i.e., an entire computation graph including each first dependency graph) that is performance-optimized on the target computing platform, thereby achieving a certain degree of parallelism and helping to avoid or reduce the occurrence of situations of excessive pseudo-parallelism.

[0098] In one alternative example, step 2022 may be performed by a processor calling corresponding instructions stored in a memory, or may be performed by a construction unit executed by a processor.

[0099] In one alternative embodiment, step 203 of determining a second computation subgraph including preset operator nodes from each first computation subgraph and dividing the second computation subgraph into a first number of target computation subgraphs based on input tensors of the second computation subgraphs includes the following steps:

[0100] In step 2031, a second computation subgraph is determined based on the preset matching rules.

[0101] Here, the preset matching rule can be determined based on the specific situation of the preset operator node. For example, a preset operator node identifier set can be set, and for each first calculation subgraph, the operator identifier of the first calculation subgraph is obtained and matched with the preset operator node identifier set to determine whether the first calculation subgraph includes a preset operator node. The specific matching rule is not limited.

[0102] In one alternative example, step 2031 may be performed by the processor calling up corresponding instructions stored in a memory, or may be performed by a fourth decision unit executed by the processor.

[0103] In step 2032, an operation type corresponding to the second computation sub-graph is determined based on the preset operator nodes.

[0104] Here, the operation type may include the aforementioned operation types such as element-wise addition, element-wise multiplication, matrix multiplication, etc.

[0105] In one alternative example, step 2032 may be performed by the processor calling up corresponding instructions stored in a memory, or may be performed by a fifth decision unit executed by the processor.

[0106] In step 2033, a target splitting rule is determined based on the operation type corresponding to the second computation sub-graph.

[0107] Here, for different operation types, partitioning rules corresponding to the operation types can be set based on the specific circumstances of the hardware processing unit. For example, for element-wise addition and element-wise multiplication operations, as long as the partitioning methods of the two tensors are consistent, they can be partitioned according to the needs of any hardware processing unit. For example, for a matrix multiplication operation, the partitioning methods of the two tensors are different, one is partitioned according to rows and the other is partitioned according to columns, so that the results obtained after the operation of each target computation subgraph after partitioning can be the same as those obtained without partitioning.

[0108] In one alternative example, step 2033 may be performed by the processor calling up corresponding instructions stored in a memory, or may be performed by a sixth decision unit executed by the processor.

[0109] In step 2034, the second computation subgraph is divided into a first number of target computation subgraphs based on the target division rule and the first input tensor and the second input tensor of the second computation subgraph.

[0110] Here, for different operation types, the target splitting rules may be the same or different, and since the tensors after splitting are smaller, the data loading time of the target calculation subgraph can be shortened, and the subsequent data calculation stage can be entered quickly, thereby improving resource utilization while maintaining a certain degree of parallelism.

[0111] 11 is a schematic diagram of the division principle corresponding to the matrix multiplication operation type provided by one exemplary embodiment of the present disclosure, taking matrix multiplication as an example. Matmul represents matrix multiplication, where tensor A is a 2x4 tensor and tensor B is a 4x2 tensor. Tensor A is divided into two 1x4 tensors (A1 and A2), and tensor B is divided into two 4x1 tensors (B1 and B2). Four target calculation subgraphs (including target calculation subgraphs corresponding to four pairs of A1 and B1, A1 and B2, A2 and B1, and A2 and B2) are obtained. Merging the results obtained by the four target calculation subgraphs results in a matrix multiplication result C of A and B, where C1 is obtained by matrix multiplication of A1 and B1, C2 is obtained by matrix multiplication of A1 and B2, C3 is obtained by matrix multiplication of A2 and B1, and C4 is obtained by matrix multiplication of A2 and B2.

[0112] In one alternative example, step 2034 may be performed by the processor calling corresponding instructions stored in memory, or may be performed by a fifth processing unit executed by the processor.

[0113] Each of the above embodiments or alternative examples of the present disclosure may be implemented alone or in any combination as long as they do not conflict with each other, and may be specifically set according to actual needs and are not limited by the present disclosure.

[0114] Any of the neural network model compilation methods provided by the embodiments of the present disclosure can be executed by any suitable device having data processing capabilities (e.g., including, but not limited to, a terminal device and a server). Alternatively, any of the neural network model compilation methods provided by the embodiments of the present disclosure can be executed by a processor, for example, by calling corresponding instructions stored in a memory to execute any of the neural network model compilation methods described in the embodiments of the present disclosure. Further description is omitted below.

[0115] As can be understood by those skilled in the art, all or part of the steps of the above method embodiments can be completed by hardware associated with program instructions, and the above program can be stored in a computer-readable storage medium, which, when executed, performs the steps of the above method embodiments, and the above storage medium includes various media that can store program code, such as a ROM, a RAM, a magnetic disk, or an optical disk.

[0116] Exemplary Apparatus 12 is a schematic structural diagram of a neural network model compilation device provided by an exemplary embodiment of the present disclosure. The device of this embodiment can implement the corresponding method embodiment of the present disclosure. The device shown in FIG. 12 includes a first acquisition module 501, a first processing module 502, a second processing module 503, a third processing module 504, and a fourth processing module 505.

[0117] The first acquisition module 501 is used to acquire a neural network model to be compiled.

[0118] The first processing module 502 is used to determine, based on the neural network model to be compiled, an initial computation graph corresponding to the neural network model to be compiled, where the initial computation graph includes at least one first computation sub-graph, and each first computation sub-graph includes at least one operator node and a connection relationship between each operator node.

[0119] The second processing module 503 is used to determine, from each first computation subgraph, a second computation subgraph including a preset operator node, and divide the second computation subgraph into a first number of target computation subgraphs based on the input tensor of the second computation subgraph.

[0120] The third processing module 504 is used to establish dependencies between a first number of target computation subgraphs based on preset dependency rules and obtain a first dependency graph corresponding to the second computation subgraph, where the dependencies characterize the computation order between each target computation subgraph.

[0121] The fourth processing module 505 is used to generate a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graphs respectively corresponding to each second computation sub-graph.

[0122] FIG. 13 is a schematic structural diagram of a neural network model compilation device provided according to one exemplary embodiment of the present disclosure.

[0123] In one alternative embodiment, the third processing module 504 includes a first determining unit 5041 , a first processing unit 5042 , a second processing unit 5043 and a third processing unit 5044 .

[0124] The first determining unit 5041 is used to determine a computation priority level respectively corresponding to each target computation sub-graph based on a preset rule.

[0125] The first processing unit 5042 is used to establish a first dependency edge between corresponding identical operator nodes in any two adjacent target computation subgraphs based on the computation priority level of each target computation subgraph, and the first dependency edge is directed from the operator node of the first target computation subgraph with a higher computation priority level to the operator node of the second target computation subgraph with a lower computation priority level among the two target computation subgraphs.

[0126] The second processing unit 5043 is used to establish, for any target computation subgraph, a second dependency edge between each operator node in the target computation subgraph based on preset computation logic between each operator node in the target computation subgraph.

[0127] The third processing unit 5044 is used to determine a first dependency graph based on each target computation subgraph, the first dependency edges between corresponding same operator nodes in any two adjacent target computation subgraphs, and the second dependency edges between each operator node in each target computation subgraph.

[0128] In one alternative embodiment, the third processing unit 5044 specifically comprises: Based on the preset weight rule, a first weight corresponding to each first dependency edge between each target computation subgraph and a second weight corresponding to each second dependency edge within each target computation subgraph are determined; a node weight of each operator node is determined based on the first weight corresponding to each first dependency edge and the second weight corresponding to each second dependency edge; and a first dependency graph is determined based on each target computation subgraph, each first dependency edge, the first weight corresponding to each first dependency edge, each second dependency edge, the second weight corresponding to each second dependency edge, and each node weight, wherein the node weight is for representing the computation priority level of the corresponding operator node.

[0129] In one alternative embodiment, the fourth processing module 505 includes a second determining unit 5051, a fourth processing unit 5052 and a third determining unit 5053.

[0130] The second determination unit 5051 is used to determine, for each second computation subgraph, a scheduling sliding window corresponding to the second computation subgraph based on the first dependency graph corresponding to the second computation subgraph, where the scheduling sliding window accommodates a second number of target computation subgraphs.

[0131] The fourth processing unit 5052 is used to generate a first instruction sequence corresponding to the second computation sub-graph based on the first dependency graph corresponding to the second computation sub-graph and the scheduling sliding window.

[0132] The third determination unit 5053 is used for determining a target instruction sequence based on the first instruction sequences respectively corresponding to each second computation sub-graph.

[0133] In one alternative embodiment, the apparatus of the disclosed embodiment further includes a fifth processing module 506 used to generate, for third computational subgraphs other than each second computational subgraph in each first computational subgraph, a second instruction sequence corresponding to each third computational subgraph based on each third computational subgraph.

[0134] The third determination unit 5053 is specifically used for determining a target instruction sequence based on the first instruction sequence corresponding to each second computation sub-graph respectively and the second instruction sequence corresponding to each third computation sub-graph respectively.

[0135] In one alternative embodiment, the first processing module 502 includes an analyzing unit 5021 and a constructing unit 5022 .

[0136] The analysis unit 5021 is used to perform analysis on the neural network model to be compiled and obtain the analysis results.

[0137] The construction unit 5022 is used to construct an initial computation graph corresponding to the neural network model to be compiled based on the analysis results.

[0138] In one alternative embodiment, the second processing module 503 includes a fourth determining unit 5031 , a fifth determining unit 5032 , a sixth determining unit 5033 and a fifth processing unit 5034 .

[0139] The fourth determining unit 5031 is used for determining the second computation subgraph based on the preset matching rule.

[0140] The fifth determining unit 5032 is used for determining the operation type corresponding to the second computation sub-graph according to the preset operator node.

[0141] The sixth determining unit 5033 is used for determining a target division rule based on the operation type corresponding to the second computation sub-graph.

[0142] The fifth processing unit 5034 is used for dividing the second computation subgraph into a first number of target computation subgraphs based on the target division rule and the first input tensor and the second input tensor of the second computation subgraph.

[0143] The modules and units in the above embodiments of the present disclosure can also be divided into finer granularities according to actual needs, and specifically can be configured according to actual needs.

[0144] The beneficial technical effects corresponding to the exemplary embodiments of the present apparatus may refer to the beneficial technical effects corresponding to the exemplary method part above, and will not be described here.

[0145] Exemplary Electronic Devices 14 is a schematic structural diagram of an application embodiment of the electronic device of the present disclosure. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0146] The processor 11 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components within the electronic device 10 to perform desired functions.

[0147] The memory 12 may include one or more computer program products, including various forms of computer-readable storage media, such as, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or high-speed cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored in the computer-readable storage medium, and the processor 11 may execute the one or more computer program instructions to implement the methods of each embodiment of the present disclosure described above and / or other desired functions.

[0148] In one example, electronic device 10 may further include input devices 13 and output devices 14, with these components connected to one another via a bus system and / or other form of connection (not shown).

[0149] The input device 13 may further include, for example, a keyboard, a mouse, and the like.

[0150] The output device 14 can output various types of information to the outside, and can include, for example, a display, a speaker, a printer, a communication network, and a remote output device connected thereto.

[0151] Of course, for the sake of simplicity, Fig. 14 shows only some of the components related to the present disclosure in the electronic device 10, and omits components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may further include any other appropriate components according to specific application situations.

[0152] Exemplary Computer Program Products and Computer-Readable Storage Media In addition to the methods and apparatus described above, embodiments of the present disclosure may also be a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods of various embodiments of the present disclosure described in the "Example Methods" section above.

[0153] The computer program product may have program code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and common procedural programming languages ​​such as "C" or similar programming languages, for carrying out operations of embodiments of the present disclosure. The program code may run entirely on the user computing device, partially on the user device, as a standalone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0154] Additionally, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the method steps of various embodiments of the present disclosure described in the "Exemplary Methods" section above of this specification.

[0155] The computer-readable storage medium may be any combination of one or more computer-readable media. The computer-readable medium may be a readable signal medium or a readable storage medium. The computer-readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0156] Although the basic principles of the present disclosure have been described above with reference to specific embodiments, the benefits, advantages, effects, etc. mentioned in the present disclosure are merely illustrative and not limiting, and it is not believed that these benefits, advantages, effects, etc. must be included in each embodiment of the present disclosure. Furthermore, the specific details disclosed above merely serve to illustrate and facilitate understanding, and are not limiting, and the details do not limit the scope of the present disclosure to those specific details.

[0157] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure also intends to include these modifications and variations.

Claims

1. obtaining a neural network model to be compiled; determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the initial computation graph including at least one first computation subgraph, each of the first computation subgraphs including at least one operator node and a connection relationship between each of the operator nodes; determining a second computation subgraph including a preset operator node from each of the first computation subgraphs, and dividing the second computation subgraph into a first number of target computation subgraphs based on an input tensor of the second computation subgraph; establishing a dependency relationship between the first number of the target computation subgraphs based on a preset dependency rule to obtain a first dependency graph corresponding to the second computation subgraph, wherein the dependency relationship is used to represent a computation order between each of the target computation subgraphs; generating a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graphs corresponding to each of the second computation subgraphs.

2. Establishing dependencies between the first number of the target computational subgraphs based on a preset dependency rule and obtaining a first dependency graph corresponding to the second computational subgraph includes: determining a computation priority level corresponding to each of the target computation sub-graphs based on a preset rule; establishing a first dependency edge between the same corresponding operator nodes in any two adjacent target computational subgraphs based on the computational priority level of each of the target computational subgraphs, wherein the first dependency edge is directed from an operator node in a first target computational subgraph with a higher computational priority level to an operator node in a second target computational subgraph with a lower computational priority level among the two target computational subgraphs; For any of the target computational subgraphs, establishing a second dependency edge between each operator node in the target computational subgraph based on a preset computation logic between each operator node in the target computational subgraph; and determining the first dependency graph based on each of the target computational subgraphs, the first dependency edges between corresponding same operator nodes in any two adjacent target computational subgraphs, and the second dependency edges between each operator node in each of the target computational subgraphs.

3. The step of determining the first dependency graph based on each of the target computational subgraphs, the first dependency edges between the same corresponding operator nodes in any two adjacent target computational subgraphs, and the second dependency edges between each operator node in each of the target computational subgraphs includes: determining, based on a preset weight rule, first weights corresponding to the first dependency edges between the target computation subgraphs and second weights corresponding to the second dependency edges within the target computation subgraphs; determining a node weight for each of the operator nodes based on the first weights corresponding to each of the first dependency edges and the second weights corresponding to each of the second dependency edges, wherein the node weights are used to characterize a computational priority level of the operator node corresponding to the node weight; determining the first dependency graph based on each of the target computational subgraphs, each of the first dependency edges, the first weights corresponding to each of the first dependency edges, each of the second dependency edges, the second weights corresponding to each of the second dependency edges, and each of the node weights.

4. generating a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graphs corresponding to the second computation subgraphs, for each second computational subgraph, determining a scheduling sliding window corresponding to the second computational subgraph based on the first dependency graph corresponding to the second computational subgraph, the scheduling sliding window being used to accommodate a second number of the target computational subgraphs; generating a first instruction sequence corresponding to the second computational subgraph based on the first dependency graph corresponding to the second computational subgraph and the scheduling sliding window; and determining the target instruction sequence based on a first instruction sequence corresponding to each of the second computational subgraphs.

5. After the step of determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, further comprising a step of generating, for each third computation subgraph other than each second computation subgraph in each of the first computation subgraphs, a second instruction sequence corresponding to each third computation subgraph based on each third computation subgraph; determining the target instruction sequence based on first instruction sequences respectively corresponding to each of the second computation subgraphs, 5. The method of claim 4, further comprising determining the target instruction sequence based on the first instruction sequence corresponding to each of the second computational subgraphs and the second instruction sequence corresponding to each of the third computational subgraphs.

6. The step of determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled includes: performing an analysis on the neural network model to be compiled and obtaining an analysis result; and constructing the initial computation graph corresponding to the neural network model to be compiled based on the analysis results.

7. The step of determining a second computation subgraph including a preset operator node from each of the first computation subgraphs, and dividing the second computation subgraph into a first number of target computation subgraphs based on an input tensor of the second computation subgraph, includes: determining the second computational subgraph based on a preset matching rule; determining an operation type corresponding to the second computation sub-graph based on the preset operator node; determining a target splitting rule based on the operation type corresponding to the second computational sub-graph; and dividing the second computational subgraph into the first number of target computational subgraphs based on the target division rule and the first input tensor and the second input tensor of the second computational subgraph.

8. a first acquisition module used to acquire a neural network model to be compiled; a first processing module used to determine an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the initial computation graph including at least one first computation subgraph, each of the first computation subgraphs including at least one operator node and a connection relationship between each of the operator nodes; a second processing module used to determine a second computational subgraph including preset operator nodes from each of the first computational subgraphs, and to divide the second computational subgraphs into a first number of target computational subgraphs based on an input tensor of the second computational subgraph; a third processing module used to establish dependencies between the first number of the target computation subgraphs based on a preset dependency rule to obtain a first dependency graph corresponding to the second computation subgraph, wherein the dependencies characterize a computation order between each of the target computation subgraphs; and a fourth processing module used to generate a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graphs corresponding to each of the second computation subgraphs.

9. A computer-readable storage medium storing a computer program for executing the neural network model compilation method according to any one of claims 1 to 7.

10. a processor; a memory for storing executable instructions for said processor; The processor is used to read and execute the executable instructions from the memory to implement the neural network model compilation method according to any one of claims 1 to 7.

Images