Compiling method and device of neural network model, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure disclose a method and device for compiling a neural network model, an electronic device and a storage medium, wherein the method comprises: obtaining a neural network model to be compiled; determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the initial computation graph comprising at least one first computation subgraph; determining a second computation subgraph comprising a preset operator node in each first computation subgraph, and splitting the second computation subgraph into a first number of target computation subgraphs according to input tensors of the second computation subgraph; establishing a dependency relationship between the first number of target computation subgraphs according to a preset dependency rule, and obtaining a first dependency graph corresponding to the second computation subgraph; and generating a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graph corresponding to each second computation subgraph. The embodiments of the present disclosure can avoid the greedy trap caused by the excessively high false parallel degree of instruction scheduling, and effectively improve the efficiency of instruction scheduling.

Description

Technical Field

[0001] This disclosure relates to artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for compiling a neural network model. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, the requirements for the training or inference processing speed of neural network models in scenarios such as autonomous driving are constantly increasing. In order to reduce training or inference time, AI chips have certain requirements for neural network compilers. Usually, during the compilation process of the neural network model, the input tensors of the operator nodes in the computation graph of the neural network model need to be divided and distributed to multiple computing units for execution. However, the existing technology is prone to excessive pseudo-parallelism in the operation of the divided input tensors, which leads to greedy traps. This causes the instruction scheduling corresponding to the operator to get stuck in the quagmire of identifying each parallel operator when generating the target instruction sequence that the chip can execute, thus reducing the scheduling efficiency of instructions. Summary of the Invention

[0003] To address the aforementioned technical problems such as low instruction scheduling efficiency, this disclosure is proposed. Embodiments of this disclosure provide a method, apparatus, electronic device, and storage medium for compiling a neural network model.

[0004] According to one aspect of the present disclosure, a method for compiling a neural network model is provided, comprising: obtaining a neural network model to be compiled; determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the initial computation graph including at least one first computation subgraph, each first computation subgraph including at least one operator node and connection relationships between the operator nodes; determining a second computation subgraph including preset operator nodes in each of the first computation subgraphs, and dividing the second computation subgraph into a first number of target computation subgraphs according to the input tensor of the second computation subgraph; establishing a dependency relationship between the first number of target computation subgraphs according to a preset dependency rule, and obtaining a first dependency graph corresponding to the second computation subgraph, the dependency relationship being used to characterize the computation order between the target computation subgraphs; and generating a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graph corresponding to each of the second computation subgraphs.

[0005] According to another aspect of the present disclosure, a neural network model compilation apparatus is provided, comprising: a first acquisition module for acquiring a neural network model to be compiled; a first processing module for determining an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the initial computation graph including at least one first computation subgraph, each first computation subgraph including at least one operator node and connection relationships between the operator nodes; a second processing module for determining a second computation subgraph including a preset operator node in each of the first computation subgraphs, and dividing the second computation subgraph into a first number of target computation subgraphs according to the input tensor of the second computation subgraph; a third processing module for establishing a dependency relationship between the first number of target computation subgraphs according to a preset dependency rule, and obtaining a first dependency graph corresponding to the second computation subgraph, the dependency relationship being used to characterize the computation order between the target computation subgraphs; and a fourth processing module for generating a target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graph corresponding to each of the second computation subgraphs.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for executing the compilation method of the neural network model described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the compilation method of the neural network model described in any of the above embodiments of the present disclosure.

[0008] Based on the neural network model compilation method, apparatus, electronic device and storage medium provided in the above embodiments of this disclosure, by dividing the second computation subgraph including preset operator nodes in the initial computation graph corresponding to the neural network model to be compiled, the dependency relationship between the target computation subgraphs obtained by the division is established to control the computation order of the target computation subgraphs, thereby avoiding the greedy trap caused by excessive pseudo-parallelism in instruction scheduling and effectively improving instruction scheduling efficiency.

[0009] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the compilation method for the neural network model provided in this disclosure;

[0012] Figure 2 This is a flowchart illustrating a method for compiling a neural network model provided in an exemplary embodiment of this disclosure;

[0013] Figure 3 This is a schematic diagram of a first computational subgraph provided in an exemplary embodiment of this disclosure;

[0014] Figure 4 This is a schematic diagram illustrating the segmentation principle of the second computational subgraph provided in an exemplary embodiment of this disclosure;

[0015] Figure 5 This is a flowchart illustrating a method for compiling a neural network model provided in another exemplary embodiment of this disclosure;

[0016] Figure 6 This is a schematic diagram of a first dependency graph provided in an exemplary embodiment of this disclosure;

[0017] Figure 7 This is a flowchart illustrating a method for compiling a neural network model provided in yet another exemplary embodiment of this disclosure;

[0018] Figure 8 This is a schematic diagram of a first dependency graph provided in another exemplary embodiment of this disclosure;

[0019] Figure 9 This is a schematic diagram of a scheduling sliding window provided in an exemplary embodiment of this disclosure;

[0020] Figure 10 This is a flowchart illustrating a method for compiling a neural network model provided in yet another exemplary embodiment of this disclosure;

[0021] Figure 11 This is a schematic diagram of the segmentation principle corresponding to the matrix multiplication operation type provided in an exemplary embodiment of this disclosure;

[0022] Figure 12 This is a schematic diagram of the structure of a compilation device for a neural network model provided in an exemplary embodiment of the present disclosure;

[0023] Figure 13This is a schematic diagram of the structure of a compilation device for a neural network model provided in an exemplary embodiment of the present disclosure;

[0024] Figure 14 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed herein. Detailed Implementation

[0025] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0026] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0027] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0028] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0029] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0030] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0031] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0032] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0033] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0034] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0035] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0036] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0037] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0038] This disclosure outlines

[0039] In the process of realizing this disclosure, the inventors discovered that with the rapid development of artificial intelligence (AI) technology, the requirements for the training or inference processing speed of neural network models in scenarios such as autonomous driving are constantly increasing. In order to reduce training or inference time, AI chips have certain requirements for neural network compilers. Usually, during the compilation of the neural network model, the input tensors of the operator nodes in the computation graph of the neural network model need to be divided and distributed to multiple computing units for execution. However, the existing technology is prone to excessive pseudo-parallelism in the operation of the divided input tensors, which leads to greedy traps. This causes the instruction scheduling corresponding to the operator to get stuck in the quagmire of identifying each parallel operator when generating the target instruction sequence that the chip can execute, thus reducing the scheduling efficiency of the instructions.

[0040] Exemplary Overview

[0041] Figure 1 This is an exemplary application scenario of the compilation method for the neural network model provided in this disclosure.

[0042] In autonomous driving scenarios, the neural network model compilation method disclosed herein (executed in a neural network model compilation device) can determine an initial computation graph corresponding to the neural network model to be compiled. The initial computation graph includes at least one first computation subgraph, each first computation subgraph including at least one operator node and the connection relationships between these operators. Based on the initial computation graph, second computation subgraphs including preset operator nodes are determined for each of the first computation subgraphs. The second computation subgraphs are then divided into a first number of target computation subgraphs according to their input tensors. Dependencies between the first number of target computation subgraphs are established according to preset dependency rules, resulting in a first dependency graph corresponding to each second computation subgraph. These dependencies characterize the computation order between the target computation subgraphs. Furthermore, based on the first dependency graphs corresponding to each second computation subgraph, a target instruction sequence corresponding to the neural network model to be compiled is generated. This sequence is used by the autonomous driving AI chip to perform model inference for corresponding functions, such as object detection and semantic segmentation. The inference results are then used for autonomous driving decision-making and planning. Because the dependency relationships between the target computation subgraphs are established to control their computation order, the greedy trap caused by excessive pseudo-parallelism in instruction scheduling can be avoided, effectively improving instruction scheduling efficiency.

[0043] The neural network model compilation method provided in this disclosure is not limited to autonomous driving scenarios, but can also be applied to any other scenarios that require input tensor segmentation, such as access control systems based on image recognition, medical image processing scenarios, and other scenarios that require image processing. The specific method can be set according to actual needs.

[0044] Exemplary methods

[0045] Figure 2 This is a flowchart illustrating a method for compiling a neural network model according to an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, specifically servers or terminals, such as... Figure 2 As shown, it includes the following steps:

[0046] Step 201: Obtain the neural network model to be compiled.

[0047] The neural network model to be compiled (which may be referred to as the model in this embodiment) can be any pre-trained neural network model, such as an object detection model, a semantic segmentation model, etc., and this disclosure does not limit it.

[0048] Step 202: Based on the neural network model to be compiled, determine the initial computation graph corresponding to the neural network model to be compiled. The initial computation graph includes at least one first computation subgraph, and each first computation subgraph includes at least one operator node and the connection relationship between each operator node.

[0049] The operator nodes can include data loading nodes, data computation nodes, and data storage nodes. Data loading nodes load tensors, weights, and other data for the data computation nodes. Data computation nodes can include operators for various calculations in the model, such as those for matrix multiplication, convolution, element-wise addition, pooling, and element-wise multiplication. Data storage nodes store the computation results from the data computation nodes. The first computation subgraph is a graphical representation of the computation order of a set of operator nodes consisting of data loading nodes, data computation nodes, and data storage nodes. The initial computation graph includes the overall graphical representation of all first computation subgraphs in the neural network model to be compiled. For example, for any computation operator (such as convolution) in the neural network model to be compiled, there can be a corresponding first computation subgraph, including the data loading node, the data computation node, and the data storage node corresponding to that operator.

[0050] For example, Figure 3 This is a schematic diagram of a first computation subgraph provided in an exemplary embodiment of this disclosure. Load Tensor A represents loading tensor A and belongs to the data loading node; Load Tensor B represents loading tensor B and belongs to the data loading node; Add represents element-wise addition of two tensors and belongs to the data computation node; ReLU represents an activation function that activates the result of element-wise addition to obtain tensor C, and also belongs to the data computation node; Store Tensor C represents storing tensor C and belongs to the data storage node.

[0051] Step 203: Determine the second computation subgraphs that include preset operator nodes in each first computation subgraph, and divide the second computation subgraphs into a first number of target computation subgraphs according to the input tensor of the second computation subgraphs.

[0052] The preset operator nodes are those that can segment tensors based on computational characteristics. They can be set according to actual needs; for example, preset operator nodes may include element-wise addition operators, matrix multiplication operators, convolution operators, pooling operators, corresponding element-wise multiplication operators, and so on. The first computational subgraph including the preset operator nodes is used as the second computational subgraph. The first quantity can be set according to actual needs. For example, for element-wise addition and element-wise multiplication operators, the first quantity can be determined based on the number of computational units available in the AI ​​chip for that computation; for example, the first quantity can be set to 2, 4, 8, etc. The target computational subgraph has the same overall logic as the first computational subgraph, only the size of the computed tensor differs.

[0053] For example, Figure 4 This is a schematic diagram illustrating the segmentation principle of the second computational subgraph provided in an exemplary embodiment of this disclosure. This example still uses element-wise addition as an example of calculating sub-nodes. With a first quantity of 4, the second computational subgraph is segmented into 4 target computational subgraphs. Tensor A is segmented along the dashed lines into four sub-tensors: A1, A2, A3, and A4. Tensor B is segmented into four sub-tensors: B1, B2, B3, and B4. Based on the computational characteristics of element-wise addition, tensor A and tensor B have the same size, and they need to be segmented in the same way to ensure that the segmented sub-tensors Ai and Bi belong to the same positions in A and B respectively, i = 1, 2, 3, 4, thus making the segmented sub-tensors Ai and Bi the same size. Tensor C, obtained from the computation of tensors A and B, is divided into four sub-tensors: C1, C2, C3, and C4. The division of tensor C also needs to be performed according to the division method of tensors A and B, based on the element-wise addition computational characteristics. Each target computational subgraph is responsible for performing the element-wise addition of the two sub-tensors (e.g., A1 and B1) at the corresponding position, and after activation, obtains the result tensor (C1) at the corresponding position.

[0054] In one optional example, the conditions that need to be met for the partitioning of tensors A, B, and C may differ depending on the computational characteristics of different operator nodes. For example, the matrix multiplication operator node has different computational characteristics than the element-wise addition operator node mentioned above. Matrix multiplication requires the sum of the products of each row element in tensor A and each column element in tensor B. Therefore, tensor A needs to be partitioned by row, and tensor B needs to be partitioned by column. The number of rows in the sub-tensors partitioned by tensor A is the same as the number of columns in the sub-tensors of tensor B. Tensor C needs to be partitioned according to the corresponding positions in tensor C of the sub-tensor operation results after partitioning tensor A and tensor B. The specific settings can be configured according to actual needs.

[0055] Step 204: Based on the preset dependency rules, establish the dependency relationships between the first number of target computation subgraphs to obtain the first dependency graph corresponding to the second computation subgraph. The dependency relationships are used to characterize the computation order between each target computation subgraph.

[0056] The preset dependency rules can be set according to actual needs, as long as the dependency relationships between each target computation subgraph can be established; no specific limitations are imposed. The dependency relationships between each target computation subgraph represent the computation order between them, thereby avoiding the greedy trap caused by excessive pseudo-parallelism in instruction scheduling. For example, the goal of traditional instruction scheduling is to seek higher parallelism under limited resource conditions. If the computation order between each target computation subgraph is not established, during instruction scheduling, loading tensors A1, A2, A3, A4, B1, B2, B3, and B4 has no data dependencies and can be executed prematurely. Executing these operations prematurely will consume high-speed storage resources, and the scheduler will also consume a lot of time to identify these invalid attempts, greatly reducing the scheduling speed. Furthermore, because storage resources are occupied in advance, it is easy to cause insufficient storage resources for subsequent processing. This disclosure, by establishing the dependency relationships between target computation subgraphs, can avoid the premature execution of invalid operations, thereby improving scheduling efficiency.

[0057] For example, the computation order between each target computation subgraph can be randomly determined, or it can be determined according to other rules, such as limiting the parallel processing of only some target computation subgraphs at the same time, or allowing two target computation subgraphs to be processed in parallel. Only when one target computation subgraph is completed is a new target computation subgraph allowed to start loading data, thereby avoiding the resource consumption of multiple target computation subgraphs loading data in parallel. The specific rules are not limited.

[0058] Step 205: Based on the first dependency graph corresponding to each second computation subgraph, generate the target instruction sequence corresponding to the neural network model to be compiled.

[0059] The target instruction sequence is the compiled binary code sequence corresponding to the neural network model to be compiled. Deploying the target instruction sequence to the AI ​​chip allows it to implement the corresponding inference functions of the neural network model, such as object detection and semantic segmentation. The AI ​​chip performs corresponding instruction scheduling and execution based on the target instruction sequence to achieve the corresponding functions.

[0060] The neural network model compilation method provided in this embodiment establishes the dependency relationship between the target computation subgraphs obtained by dividing the second computation subgraph, which includes preset operator nodes, in the initial computation graph corresponding to the neural network model to be compiled. This relationship is used to control the computation order of the target computation subgraphs, thereby avoiding the greedy trap caused by excessive pseudo-parallelism in instruction scheduling and effectively improving instruction scheduling efficiency.

[0061] Figure 5 This is a flowchart illustrating a method for compiling a neural network model provided in another exemplary embodiment of this disclosure.

[0062] In an optional embodiment, step 204 may specifically include the following steps:

[0063] Step 2041: Based on preset rules, determine the computation priority level corresponding to each target computation subgraph.

[0064] The preset rules can be set according to actual needs. For example, the computation priority level of each target computation subgraph can be randomly determined, or it can be determined according to the order of the sub-tensors of the split tensor. For example, the target computation subgraphs corresponding to A1 and B1 have the highest computation priority, followed by the target computation subgraphs corresponding to A2 and B2, and so on. There are no specific limitations.

[0065] Step 2042: Based on the computation priority level of each target computation subgraph, establish a first dependency edge between corresponding operator nodes in any two adjacent target computation subgraphs. The first dependency edge points from the operator node in the first target computation subgraph with the higher computation priority level to the operator node in the second target computation subgraph with the lower computation priority level.

[0066] The first dependency edge represents the computational order dependency between corresponding operator nodes in two adjacent target computation subgraphs. The instruction of the operator node pointed to by the first dependency edge can only be executed after the instruction corresponding to the starting node of the first dependency edge has been executed.

[0067] Step 2043: For any target computation subgraph, establish a second dependency edge between each operator node in the target computation subgraph according to the preset computation logic between each operator node in the target computation subgraph.

[0068] For any target computation subgraph, see [link to relevant documentation]. Figure 4 Each operator node has a certain computational logic. The second dependency edge is used to represent the computational order dependency relationship between each operator node in the same target computational subgraph. The principle is similar to that of the first dependency edge.

[0069] Step 2044: Determine the first dependency graph based on each target computation subgraph, the first dependency edge between corresponding identical operator nodes in any two adjacent target computation subgraphs, and the second dependency edge between each operator node in each target computation subgraph.

[0070] The first dependency graph includes the dependency relationships between adjacent target computation subgraphs and within each target computation subgraph, which are divided by the corresponding second computation subgraph.

[0071] For example, Figure 6 This is a schematic diagram of a first dependency graph provided in an exemplary embodiment of this disclosure. This example still uses the aforementioned element-wise addition operator sub-node as an example, where dashed arrows represent first dependency edges between two adjacent target computation subgraphs, and solid arrows represent second dependency edges within the target computation subgraph.

[0072] This embodiment establishes a first dependency edge between each target computation subgraph and a second dependency edge within each target computation subgraph to characterize the computation order dependency relationship between each operator node. This can effectively avoid scheduling situations with high pseudo-parallelism, thereby greatly improving instruction scheduling efficiency during instruction scheduling.

[0073] Figure 7 This is a flowchart illustrating a method for compiling a neural network model provided in another exemplary embodiment of this disclosure.

[0074] In an optional embodiment, step 2044, which determines the first dependency graph based on each target computation subgraph, the first dependency edge between corresponding operator nodes in any two adjacent target computation subgraphs, and the second dependency edge between each operator node in each target computation subgraph, includes:

[0075] Step 20441: According to the preset weighting rules, determine the first weights corresponding to the first dependent edges between each target computation subgraph and the second weights corresponding to the second dependent edges within each target computation subgraph.

[0076] The preset weight rules can be set according to actual needs. For example, the first dependent edge can be set to a larger weight, such as 100, and the second dependent edge can be set to a smaller weight, such as 1. The specific settings can be made according to actual needs.

[0077] Step 20442: Based on the first weight corresponding to each first dependent edge and the second weight corresponding to each second dependent edge, determine the node weight of each operator node. The node weight is used to characterize the computation priority level of its corresponding operator node.

[0078] Specifically, for any second computational subgraph, the node weight of the final operator node can be set to a reference value (e.g., 0) based on the directions of all first and second dependent edges. The node weight of the starting operator node connected by the first or second dependent edge pointing to the final operator node can be determined, and so on, until the node weights of all operator nodes are obtained.

[0079] Step 20443: Based on each target calculation subgraph, each first dependency edge, each first dependency edge corresponding to its first weight, each second dependency edge, each second dependency edge corresponding to its second weight, and each node weight, determine the first dependency graph.

[0080] For example, Figure 8 This is a schematic diagram of a first dependency graph provided in another exemplary embodiment of this disclosure. Starting from the lower right operator node Store Tensor C4, its node weight (gray number) is set to a reference value of 0. The first weight corresponding to the first dependency edge is set to 100, and the second weight corresponding to the second dependency edge is set to 1. The node weights of other operator nodes are obtained by superimposing the first weight of the first dependency edge they depend on and the node weight of the operator node pointed to by that first dependency edge, or by superimposing the second weight of the second dependency edge they depend on and the node weight of the operator node pointed to by that second dependency edge. For example, the node weight of operator node Store Tensor C3 is the sum of the node weight of Store Tensor C4 and the first weight of the first dependency edge between them; therefore, the node weight of Store Tensor C3 is 100. Another example is the target operator... Figure 3 The node weight of the operator node Add is the sum (102) of the node weight of the ReLU operator node that depends on it (101) and the second weight (1) of the second dependency edge between them. The node weight of Add is also the weight of the target operator that depends on it. Figure 4 The node weight (2) of the operator node Add is summed with the first weight (100) of the first dependent edge between them (102). This process is repeated to obtain the node weights of each operator node, as shown in the figure. In practical applications, other first and second weights can also be used, or node weights can be determined based on other node weight rules, as long as the calculation order between each operator node can be represented according to the requirements. The specific settings can be configured according to actual needs.

[0081] In an optional example, the first weights of each first dependency edge can also be set to different or partially different values, such as the target computed sub-edge. Figure 1 and target computation Figure 2 The first weight of each first dependent edge between them can be related to the target computed sub-edge. Figure 2 and target computation Figure 3 The first weights of the first dependent edges between them are different, and can be set according to actual needs.

[0082] In an optional example, the second weights of each second dependency edge can also be set to different or partially different values, such as the target computed sub-edge. Figure 1 The second weight of the second dependent edge can be related to the target computer. Figure 2 The second weight of the second dependent edge is different, and can be set according to actual needs.

[0083] In an optional example, when determining the node weights of operators other than those with reference values, for any other operator, the first result is the sum of the first weight of the first dependent edge that depends on that other operator and the node weight of the operator pointed to by the first dependent edge; the second result is the sum of the second weight of the second dependent edge that depends on that other operator and the node weight of the operator pointed to by the second dependent edge. If the first result and the second result are different, the maximum value between the first result and the second result is taken as the node weight of that other operator. For example, ... Figure 8 Medium target calculation sub Figure 4 The second weight of the second dependent edge is set to 1, and the target computation sub-edge is... Figure 3 If the second weight of the second dependent edge is set to 2, then for the target computation sub Figure 3 The node weight of the operator node Add is such that the sum of the node weight (102) of the operator node ReLU that depends on it and the second weight (2) of the second dependency edge between them is 104, while the target computation node that depends on this operator node Add... Figure 4 If the sum of the node weight (2) of the operator node Add and the first weight (100) of the first dependent edge between them is 102, which is less than 104, then the target operator is determined. Figure 3 The node weight of the sub-node Add is 104.

[0084] This embodiment is for Figure 6 The dependency graph may contain unfairness issues (for example, if Tensor A is relatively small and Tensor B is relatively large, the Add computation speed is relatively low. When executing the Add operation of the first target computation subgraph, the scheduler will face the question of whether to schedule and load Tensors A2, A3, A4, or even Tensors B2, B3, B4 during idle time. Obviously, these Tensors will not be used for a long time. For example, A4 and B4 will only be used when computing the fourth Add operation, so there is a possibility that resources will be occupied in advance). By determining the node weight corresponding to the operator node through the first weight corresponding to the first dependency edge and the second weight corresponding to the second dependency edge, the parallel execution of adjacent target computation subgraphs can be effectively controlled. See [link to relevant documentation]. Figure 8 By setting the first weight of the first dependent edge between target computation subgraphs to a larger value, the node weights of the operator nodes of adjacent target computation subgraphs are at different levels (e.g., level 200 and level 300). This allows the number of target computation subgraphs that can be executed in parallel to be limited by controlling the number of levels that can be executed in parallel or by controlling the node weight threshold, thereby avoiding the aforementioned unfairness issues and further improving scheduling efficiency.

[0085] In an optional embodiment, step 205, which generates the target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graphs corresponding to each of the second computational subgraphs, includes:

[0086] Step 2051: For each second computational subgraph, based on the first dependency graph corresponding to the second computational subgraph, determine the scheduling sliding window corresponding to the second computational subgraph. The scheduling sliding window is used to accommodate the second number of target computational subgraphs.

[0087] The second quantity can be set according to actual needs, such as 2, 3, etc. The scheduling sliding window is used to control the number of target computation subgraphs that can be scheduled in parallel at the same time. For example, when there are 4 target computation subgraphs, the second quantity is set to 2, and only two target computation subgraphs can be processed in parallel at the same time. When the instruction scheduling corresponding to the first target computation subgraph in the scheduling sliding window is completed, the sliding window will slide the completed target computation subgraph out of the window, and at the same time slide the subsequent unprocessed target computation subgraph into the window. Based on this, while ensuring a certain degree of parallelism, it can avoid the occurrence of excessive pseudo-parallelism and further improve the instruction scheduling efficiency.

[0088] For example, Figure 9 This is a schematic diagram of a scheduling sliding window provided in an exemplary embodiment of this disclosure. At a certain moment, the scheduling sliding window (solid-lined rectangle) includes the target computation sub-module. Figure 1 and target computation Figure 2 When the target computation sub Figure 1 Once completed, slide the window to the position of the dashed rectangle. At this point, the scheduling sliding window includes the target computation sub-module. Figure 2 and target computation Figure 3 The sliding of the window can be controlled by the node weight threshold. For example, the initial node weight threshold can be set to a value between 100 and 200, such as 150. The window can be slid by decreasing the node weight threshold by 100 each time. The specific settings can be adjusted according to actual needs.

[0089] Step 2052: Based on the first dependency graph and scheduling sliding window corresponding to the second computation subgraph, generate the first instruction sequence corresponding to the second computation subgraph.

[0090] The first instruction sequence is the sequence of binary instructions executable by the hardware (such as an AI chip) corresponding to the second computational subgraph. The specific generation can be achieved through compilation using a suitable compiler, which will not be elaborated upon here.

[0091] Step 2053: Determine the target instruction sequence based on the first instruction sequence corresponding to each of the second computation subgraphs.

[0092] Since the initial computation graph corresponding to the neural network model to be compiled usually includes multiple first computation subgraphs, in addition to the aforementioned second computation subgraphs, there are other first computation subgraphs that do not include preset operator nodes. The first instruction sequence corresponding to each second computation subgraph is determined together with the instruction sequences of other first computation subgraphs according to the logical position of each second computation subgraph in the overall computation graph.

[0093] This embodiment controls the number of target computation subgraphs that can be scheduled in parallel at the same time by using a scheduling sliding window. This can avoid excessive pseudo-parallelism while ensuring a certain degree of parallelism, and further improve instruction scheduling efficiency.

[0094] Figure 10 This is a flowchart illustrating a method for compiling a neural network model provided in yet another exemplary embodiment of this disclosure.

[0095] In an optional embodiment, after determining the initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled in step 202, the method further includes:

[0096] Step 301: For each third computation subgraph other than each second computation subgraph in each first computation subgraph, generate a second instruction sequence corresponding to each third computation subgraph based on each third computation subgraph.

[0097] Since the second computational subgraph is the first computational subgraph that includes preset operator nodes, there may also be a first computational subgraph that does not include preset operator nodes. For this part of the first computational subgraph, it is used as the third computational subgraph. The corresponding second instruction sequence is generated according to the third computational subgraph. Specifically, it can be generated based on the computational logic between each operator node in the third computational subgraph, which will not be elaborated here.

[0098] Step 2053, based on the first instruction sequence corresponding to each of the second computational subgraphs, determines the target instruction sequence, including:

[0099] Step 20531: Determine the target instruction sequence based on the first instruction sequence corresponding to each second computation subgraph and the second instruction sequence corresponding to each third computation subgraph.

[0100] Specifically, the target instruction sequence can be determined by scheduling the first instruction sequence and the second instruction sequence according to the computational logic between each second computational subgraph and each third computational subgraph.

[0101] This embodiment determines the target instruction sequence through each first instruction sequence and each second instruction sequence, so that when the target instruction sequence is scheduled and executed on the hardware platform, it can effectively avoid the situation of excessive pseudo-parallelism while ensuring a certain degree of parallelism, thus greatly improving the instruction scheduling efficiency.

[0102] In an optional example, the compilation process of a neural network model may include three parts: computation graph construction, computation graph optimization, and instruction generation. Computation graph construction involves building a general first intermediate representation based on the input model file. This first intermediate representation is in graph form, corresponding to the initial computation graph described above in this embodiment. Computation graph optimization involves optimizing the first intermediate representation to obtain a second intermediate representation in graph form, corresponding to the process of generating a dependency graph based on the initial computation graph in this embodiment. Instruction generation involves scheduling optimization of the second intermediate representation to obtain a fine-grained third intermediate representation, and compiling the third representation into instruction code executable on the hardware platform. This can correspond to the process of generating a target instruction sequence based on the first dependency graph corresponding to each second computation subgraph in this embodiment. The specific compiler architecture is not limited.

[0103] In an optional embodiment, step 202, determining the initial computation graph corresponding to the neural network model to be compiled, includes:

[0104] Step 2021: parse the neural network model to be compiled and obtain the parsing results.

[0105] The parsing of the neural network model to be compiled can be implemented based on pre-configured parsing rules or using any implementable parser, without specific limitations. For example, the syntax and semantics of the model file of the neural network model to be compiled can be parsed according to the parsing rules, and the syntax and semantics can be used as the parsing result. The model file can include a large number of computer programming languages, which can be applied to object recognition, image recognition, image classification, semantic segmentation, speech recognition, etc. The computer coding language can be a special-purpose coding language or a general-purpose coding language. For example, in this disclosure, the computer programming language can be a neural network-specific programming language, which can be set according to actual needs. The neural network-specific programming language can be represented as program code that represents the neural network based on the description of the computation graph, and the program code can include program statements describing the computation graph.

[0106] Step 2022: Based on the analysis results, construct the initial computation graph corresponding to the neural network model to be compiled.

[0107] Since the program code of the neural network model to be compiled includes program statements describing the computation graph, the computation graph structure corresponding to the initial computation graph can be directly constructed based on the syntax and semantics obtained by parsing the program statements of the neural network model to be compiled. The initial computation graph is then obtained based on this structure. The initial computation graph can be a nested structure graph containing complex templates, consistent with the semantics in the program code of the neural network model to be compiled. The computation graph obtained based on the computation graph structure can also be extended to obtain a computation graph composed of interconnected basic units in the program, which serves as the initial computation graph. The initial computation graph can include the complete execution process of the program and can be translated into an executable instruction sequence. This disclosure optimizes the initial computation graph according to the method of this disclosure to obtain a computation graph structure that maximizes performance on the target computing platform (i.e., the overall computation graph including each first dependency graph), ensuring a certain degree of parallelism while avoiding excessive pseudo-parallelism.

[0108] In an optional embodiment, step 203, determining that each first computational subgraph includes a second computational subgraph with preset operator nodes, and dividing the second computational subgraph into a first number of target computational subgraphs based on the input tensor of the second computational subgraph, includes:

[0109] Step 2031: Determine the second computational subgraph based on the preset matching rules.

[0110] The preset matching rules can be determined based on the specific circumstances of the preset operator nodes. For example, a preset operator node identifier set can be set. For each first computational subgraph, the operator identifier of the first computational subgraph can be obtained and matched with the preset operator node identifier set to determine whether the first computational subgraph includes the preset operator node. No specific restrictions are imposed.

[0111] Step 2032: Determine the operation type corresponding to the second computation subgraph based on the preset operator nodes.

[0112] The operation types can include the aforementioned element-wise addition, element-wise multiplication, matrix multiplication, and other operation types.

[0113] Step 2033: Determine the target segmentation rule based on the operation type corresponding to the second computational subgraph.

[0114] Specifically, for different operation types, the corresponding partitioning rules can be set based on the specific situation of the hardware processing unit. For example, for element-wise addition and element-wise multiplication operations, as long as the partitioning methods of the two tensors are consistent, partitioning can be performed according to the needs of any hardware processing unit. For another example, for matrix multiplication operations, the partitioning methods of the two tensors are different, one partitioning by row and the other partitioning by column, to ensure that the results obtained after the operation of each target computation subgraph after partitioning can form the same result as when it is not partitioned.

[0115] Step 2034: Based on the target segmentation rule and the first and second input tensors of the second computational subgraph, the second computational subgraph is segmented into a first number of target computational subgraphs.

[0116] Among them, the target splitting rules can be the same or different for different operation types. After splitting, the tensor becomes smaller, which reduces the data loading time of the target computation subgraph and allows it to enter the subsequent data computation stage in a timely manner, thereby ensuring resource utilization on the basis of a certain degree of parallelism.

[0117] For example, let's take matrix multiplication as an example. Figure 11 This is a schematic diagram illustrating the segmentation principle corresponding to the matrix multiplication operation type provided in an exemplary embodiment of this disclosure. Matmul represents matrix multiplication. Tensor A is a 2×4 tensor, and tensor B is a 4×2 tensor. Tensor A is segmented into two 1×4 tensors (A1 and A2), and tensor B is segmented into two 4×1 tensors (B1 and B2), resulting in four target computation subgraphs (including four sets of target computation subgraphs corresponding to A1 and B1, A1 and B2, A2 and B1, and A2 and B2, respectively). The results obtained from the four target computation subgraphs are combined and are exactly the same as the result C of matrix multiplication of A and B. Specifically, matrix multiplication of A1 and B1 results in C1, matrix multiplication of A1 and B2 results in C2, matrix multiplication of A2 and B1 results in C3, and matrix multiplication of A2 and B2 results in C4.

[0118] The embodiments or optional examples disclosed herein can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit it.

[0119] The compilation method for any neural network model provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to terminal devices and servers. Alternatively, the compilation method for any neural network model provided in this disclosure can be executed by a processor, such as by a processor executing the compilation method for any neural network model mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0120] Exemplary device

[0121] Figure 12 This is a schematic diagram of the structure of a compilation apparatus for a neural network model provided in an exemplary embodiment of this disclosure. The apparatus of this embodiment can be used to implement corresponding method embodiments of this disclosure, such as... Figure 12 The device shown includes: a first acquisition module 501, a first processing module 502, a second processing module 503, a third processing module 504, and a fourth processing module 505.

[0122] The first acquisition module 501 is used to acquire the neural network model to be compiled.

[0123] The first processing module 502 is used to determine the initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled. The initial computation graph includes at least one first computation subgraph, and each first computation subgraph includes at least one operator node and the connection relationship between each operator node.

[0124] The second processing module 503 is used to determine the second computation subgraphs that include preset operator nodes in each first computation subgraph, and to divide the second computation subgraphs into a first number of target computation subgraphs according to the input tensor of the second computation subgraphs.

[0125] The third processing module 504 is used to establish the dependency relationship between a first number of target computation subgraphs according to the preset dependency rules, and obtain the first dependency graph corresponding to the second computation subgraph. The dependency relationship is used to characterize the computation order between each target computation subgraph.

[0126] The fourth processing module 505 is used to generate the target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graph corresponding to each of the second computation subgraphs.

[0127] Figure 13 This is a schematic diagram of the structure of a compilation device for a neural network model provided in an exemplary embodiment of this disclosure.

[0128] In an optional embodiment, the third processing module 504 includes: a first determining unit 5041, a first processing unit 5042, a second processing unit 5043, and a third processing unit 5044.

[0129] The first determining unit 5041 is used to determine the calculation priority level corresponding to each target calculation subgraph based on preset rules.

[0130] The first processing unit 5042 is used to establish a first dependency edge between corresponding operator nodes in any two adjacent target computation subgraphs according to the computation priority level of each target computation subgraph. The first dependency edge points from the operator node in the first target computation subgraph with a higher computation priority level to the operator node in the second target computation subgraph with a lower computation priority level.

[0131] The second processing unit 5043 is used to establish a second dependency edge between each operator node in the target computation subgraph according to the preset computation logic between each operator node in the target computation subgraph for any target computation subgraph.

[0132] The third processing unit 5044 is used to determine the first dependency graph based on each target computation subgraph, the first dependency edge between corresponding identical operator nodes in any two adjacent target computation subgraphs, and the second dependency edge between each operator node in each target computation subgraph.

[0133] In an optional embodiment, the third processing unit 5044 is specifically used for:

[0134] According to the preset weighting rules, the first weights corresponding to the first dependent edges between each target computation subgraph and the second weights corresponding to the second dependent edges within each target computation subgraph are determined. Based on the first weights corresponding to the first dependent edges and the second weights corresponding to the second dependent edges, the node weights of each operator node are determined, and the node weights are used to characterize the computation priority level of the corresponding operator node. Based on each target computation subgraph, each first dependent edge, the first weights corresponding to each first dependent edge, each second dependent edge, the second weights corresponding to each second dependent edge, and the node weights, the first dependency graph is determined.

[0135] In an optional embodiment, the fourth processing module 505 includes: a second determining unit 5051, a fourth processing unit 5052, and a third determining unit 5053.

[0136] The second determining unit 5051 is used to determine, for each second computational subgraph, a scheduling sliding window corresponding to the second computational subgraph based on the first dependency graph corresponding to the second computational subgraph. The scheduling sliding window is used to accommodate a second number of target computational subgraphs.

[0137] The fourth processing unit 5052 is used to generate a first instruction sequence corresponding to the second computation subgraph based on the first dependency graph and the scheduling sliding window corresponding to the second computation subgraph.

[0138] The third determining unit 5053 is used to determine the target instruction sequence based on the first instruction sequence corresponding to each of the second calculation subgraphs.

[0139] In an optional embodiment, the apparatus of this disclosure further includes: a fifth processing module 506, configured to generate a second instruction sequence corresponding to each third computational subgraph based on each third computational subgraph other than each second computational subgraph in each first computational subgraph.

[0140] The third determining unit 5053 is specifically used to: determine the target instruction sequence based on the first instruction sequence corresponding to each second calculation subgraph and the second instruction sequence corresponding to each third calculation subgraph.

[0141] In an optional embodiment, the first processing module 502 includes a parsing unit 5021 and a construction unit 5022.

[0142] The parsing unit 5021 is used to parse the neural network model to be compiled and obtain the parsing result.

[0143] Construction unit 5022 is used to construct the initial computation graph corresponding to the neural network model to be compiled based on the parsing results.

[0144] In an optional embodiment, the second processing module 503 includes: a fourth determining unit 5031, a fifth determining unit 5032, a sixth determining unit 5033, and a fifth processing unit 5034.

[0145] The fourth determining unit 5031 is used to determine the second calculation subgraph based on a preset matching rule.

[0146] The fifth determining unit 5032 determines the operation type corresponding to the second computation subgraph based on the preset operator nodes.

[0147] The sixth determining unit 5033 is used to determine the target segmentation rule based on the operation type corresponding to the second calculation subgraph.

[0148] The fifth processing unit 5034 is used to divide the second computational subgraph into a first number of target computational subgraphs based on the target segmentation rules and the first and second input tensors of the second computational subgraph.

[0149] The modules and units in the above embodiments of this disclosure can be further divided into finer-grained units according to actual needs, and can be specifically set according to actual needs.

[0150] Exemplary electronic devices

[0151] This disclosure also provides an electronic device, including: a memory for storing computer programs;

[0152] A processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, it implements the compilation method of the neural network model described in any of the above embodiments of the present disclosure.

[0153] Figure 14 This is a schematic diagram of an application embodiment of the electronic device disclosed herein. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0154] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0155] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0156] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0157] For example, the input device 13 may be the microphone or microphone array described above, used to capture the input signal of the sound source.

[0158] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0159] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0160] Of course, for the sake of simplicity, Figure 14 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0161] Exemplary computer program products and computer-readable storage media

[0162] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0163] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0164] Furthermore, embodiments of this disclosure may also be computer-readable storage media having computer program instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0165] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0166] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0167] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0168] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0169] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0170] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0171] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0172] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for compiling a neural network model, comprising: Obtain the neural network model to be compiled; Based on the neural network model to be compiled, an initial computation graph corresponding to the neural network model to be compiled is determined. The initial computation graph includes at least one first computation subgraph, and each first computation subgraph includes at least one operator node and the connection relationship between each operator node. Each of the first computational subgraphs includes a second computational subgraph containing a preset operator node. The second computational subgraph is then divided into a first number of target computational subgraphs based on the input tensor of the second computational subgraph. The tensor size calculated by the target computation subgraph is different from the tensor size calculated by the second computation subgraph; According to a preset dependency rule, a dependency relationship is established between the first number of target computation subgraphs to obtain a first dependency graph corresponding to the second computation subgraph. The dependency relationship is used to characterize the computation order between each target computation subgraph. Based on the first dependency graph corresponding to each of the second computational subgraphs, a target instruction sequence corresponding to the neural network model to be compiled is generated.

2. The method according to claim 1, wherein, The step of establishing the dependency relationships between the first number of target computation subgraphs according to preset dependency rules, and obtaining the first dependency graph corresponding to the second computation subgraph, includes: Based on preset rules, the computation priority level corresponding to each of the target computation subgraphs is determined; Based on the computation priority level of each target computation subgraph, a first dependency edge is established between corresponding operator nodes in any two adjacent target computation subgraphs. The first dependency edge points from the operator node in the first target computation subgraph with a higher computation priority level to the operator node in the second target computation subgraph with a lower computation priority level. For any of the target computation subgraphs, a second dependency edge is established between the operator nodes in the target computation subgraph according to the preset computation logic between each operator node in the target computation subgraph. The first dependency graph is determined based on each of the target computation subgraphs, the first dependency edge between corresponding operator nodes in any two adjacent target computation subgraphs, and the second dependency edge between each operator node in each of the target computation subgraphs.

3. The method according to claim 2, wherein, The step of determining the first dependency graph based on each of the target computation subgraphs, the first dependency edge between corresponding identical operator nodes in any two adjacent target computation subgraphs, and the second dependency edge between each operator node in each of the target computation subgraphs includes: According to the preset weighting rules, the first weights corresponding to each first dependency edge between each target computation subgraph and the second weights corresponding to each second dependency edge inside each target computation subgraph are determined. Based on the first weight corresponding to each of the first dependent edges and the second weight corresponding to each of the second dependent edges, the node weight of each operator node is determined, and the node weight is used to characterize the computation priority level of its corresponding operator node. The first dependency graph is determined based on each target computation subgraph, each first dependency edge, the first weight corresponding to each first dependency edge, each second dependency edge, the second weight corresponding to each second dependency edge, and each node weight.

4. The method according to claim 3, wherein, The step of generating the target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graph corresponding to each of the second computational subgraphs includes: For each second computational subgraph, a scheduling sliding window corresponding to the second computational subgraph is determined based on the first dependency graph corresponding to the second computational subgraph. The scheduling sliding window is used to accommodate a second number of the target computational subgraphs. Based on the first dependency graph corresponding to the second computation subgraph and the scheduling sliding window, a first instruction sequence corresponding to the second computation subgraph is generated; The target instruction sequence is determined based on the first instruction sequence corresponding to each of the second computation subgraphs.

5. The method according to claim 4, wherein, After determining the initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled, the method further includes: For each third computation subgraph in each first computation subgraph other than each second computation subgraph, a second instruction sequence corresponding to each third computation subgraph is generated based on each third computation subgraph; Determining the target instruction sequence based on the first instruction sequence corresponding to each of the second computational subgraphs includes: The target instruction sequence is determined based on the first instruction sequence corresponding to each of the second computation subgraphs and the second instruction sequence corresponding to each of the third computation subgraphs.

6. The method according to claim 1, wherein, The step of determining the initial computation graph corresponding to the neural network model to be compiled, based on the neural network model to be compiled, includes: The neural network model to be compiled is parsed to obtain the parsing results; Based on the analysis results, the initial computation graph corresponding to the neural network model to be compiled is constructed.

7. The method according to any one of claims 1-6, wherein, The step of determining a second computational subgraph including preset operator nodes in each of the first computational subgraphs, and dividing the second computational subgraph into a first number of target computational subgraphs based on the input tensor of the second computational subgraph, includes: The second computational subgraph is determined based on preset matching rules; Based on the preset operator nodes, determine the operation type corresponding to the second computation subgraph; Based on the operation type corresponding to the second computational subgraph, the target segmentation rule is determined; Based on the target segmentation rule and the first and second input tensors of the second computational subgraph, the second computational subgraph is segmented into the first number of target computational subgraphs.

8. A compilation device for a neural network model, comprising: The first acquisition module is used to acquire the neural network model to be compiled. The first processing module is used to determine an initial computation graph corresponding to the neural network model to be compiled based on the neural network model to be compiled. The initial computation graph includes at least one first computation subgraph, and each first computation subgraph includes at least one operator node and the connection relationship between each operator node. The second processing module is used to determine that each of the first computational subgraphs includes a second computational subgraph containing a preset operator node, and to divide the second computational subgraph into a first number of target computational subgraphs according to the input tensor of the second computational subgraph; The tensor size calculated by the target computation subgraph is different from the tensor size calculated by the second computation subgraph; The third processing module is used to establish the dependency relationship between the first number of target computation subgraphs according to the preset dependency rules, and obtain the first dependency graph corresponding to the second computation subgraph. The dependency relationship is used to characterize the computation order between each target computation subgraph. The fourth processing module is used to generate the target instruction sequence corresponding to the neural network model to be compiled based on the first dependency graph corresponding to each of the second computation subgraphs.

9. A computer-readable storage medium storing a computer program for executing a method for compiling a neural network model according to any one of claims 1-7.

10. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the compilation method of the neural network model according to any one of claims 1-7.

Images