A task compilation method, device and compiler

Abstract

Embodiments of the present application provide a task compiling method, device and compiler. The method comprises: the compiler receiving at least one compiling task input by a user, the compiler judging whether the compiling task comprises multiple branch tasks, if it is judged that the compiling task comprises multiple branch tasks, then dividing the hardware resources of an embedded neural network processor (NPU) obtained according to the multiple branch tasks, generating a hardware allocation result, and generating a first compiling instruction according to the hardware allocation result, the compiling parameters input by the user and the core parameters required by each NPU core calculation obtained; the compiler sending the first compiling instruction to a scheduler, so that the scheduler schedules the hardware resources according to the first compiling instruction, thereby realizing reasonable allocation of the hardware resources of the NPU through the compiler and improving the operation speed of the NPU.

Description

[Technical Field]

[0001] This invention relates to the field of artificial intelligence (AI) technology, and more particularly to a task compilation method, apparatus and compiler. [Background Technology]

[0002] With the rapid development of technology, the AI ​​industry is growing rapidly, and the demand for image and video processing is constantly increasing. Embedded neural network processing units (NPUs), as processors specifically designed to accelerate neural network inference, primarily employ application-specific integrated circuit (ASIC) technology to simulate neural networks in hardware. This compensates for the shortcomings of central processing units (CPUs) and graphics processing units (GPUs) in terms of neural network computing architecture design, significantly improving the computing speed of AI chips.

[0003] In related technologies, to further improve the computing speed of AI chips, the design of NPUs inevitably moves towards a multi-core architecture. Currently, the development of multi-core NPUs has not yet formed a complete system; it lacks mature and targeted compilers to analyze front-end models, and cannot rationally allocate back-end hardware configurations according to tasks, resulting in a reduction in computing speed. [Summary of the Invention]

[0004] In view of this, embodiments of the present invention provide a task compilation method, apparatus and compiler, which enables reasonable allocation of NPU hardware resources to improve the NPU's computing speed.

[0005] In a first aspect, embodiments of the present invention provide a task compilation method, the method comprising:

[0006] At least one compilation task that receives user input;

[0007] Determine whether the compilation task includes multiple branch tasks;

[0008] If it is determined that the compilation task includes multiple branch tasks, the hardware resources of the acquired embedded neural network processor (NPU) are divided according to the multiple branch tasks to generate hardware allocation results.

[0009] Based on the hardware allocation results, the compilation parameters input by the user, and the core parameters required for each NPU core to compute, a first compilation instruction is generated;

[0010] The first compilation instruction is sent to the scheduler so that the scheduler can schedule the hardware resources according to the first compilation instruction.

[0011] Optionally, the step of allocating the acquired hardware resources of the embedded neural network processor (NPU) according to multiple branch tasks to generate a hardware allocation result includes:

[0012] Cluster multiple branch tasks to generate multiple task category data;

[0013] The hardware resources are divided according to data from multiple task categories to generate hardware allocation results.

[0014] Optionally, it also includes:

[0015] If it is determined that the compilation task does not include branch tasks, then a second compilation instruction is generated based on the compilation parameters and core parameters;

[0016] The second compilation instruction is sent to the scheduler so that the scheduler can schedule the hardware resources according to the second compilation instruction.

[0017] Optionally, the number of compilation tasks is multiple, and before determining whether the compilation tasks include multiple branch tasks, the method further includes:

[0018] If it is determined that the number of compilation tasks is less than or equal to the number of NPU clusters, then each compilation task is assigned to the corresponding NPU cluster, and the step of determining whether the compilation task includes multiple branch tasks is executed in parallel.

[0019] Optionally, the number of compilation tasks is multiple, and the method further includes:

[0020] If it is determined that the number of compilation tasks is greater than the number of NPU clusters, then the computational requirements of each compilation task are calculated.

[0021] A specific number of compilation tasks are selected from multiple compilation tasks input by the user. The computational requirements of these specific number of compilation tasks are all less than the computational requirements of other compilation tasks input by the user. The specific number is equal to the number of NPU clusters.

[0022] Each compilation task in a specific number of compilation tasks is assigned to the corresponding NPU cluster, and the step of determining whether the compilation task includes multiple branch tasks is executed in parallel.

[0023] Secondly, embodiments of the present invention provide a task compilation apparatus, the apparatus comprising:

[0024] The receiving module is used to receive at least one compilation task from user input;

[0025] The acquisition module is used to acquire NPU hardware resources;

[0026] The judgment module is used to determine whether the compilation task includes multiple branch tasks;

[0027] The generation module is used to divide the hardware resources according to the multiple branch tasks and generate a hardware allocation result if the judgment module determines that the compilation task includes multiple branch tasks; and to generate a first compilation instruction according to the hardware allocation result, compilation parameters and core parameters.

[0028] The sending module is used to send the first compilation instruction to the scheduler, so that the scheduler can schedule the hardware resources according to the first compilation instruction.

[0029] Optionally, the generation module includes:

[0030] The first generation submodule is used to cluster multiple branch tasks and generate multiple task category data;

[0031] The second generation submodule is used to divide the hardware resources according to multiple task category data and generate hardware allocation results.

[0032] Optionally, it also includes:

[0033] The generation module is also used to generate a second compilation instruction based on the compilation parameters and core parameters if the judgment module determines that the compilation task does not include a branch task.

[0034] The sending module is also used to send the second compilation instruction to the scheduler, so that the scheduler can schedule the hardware resources according to the second compilation instruction.

[0035] Thirdly, embodiments of the present invention provide a computer-readable storage medium comprising a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to execute the task compilation method in the first aspect or any possible implementation thereof.

[0036] Fourthly, embodiments of the present invention provide a compiler, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the task compilation method of the first aspect or any possible implementation thereof.

[0037] In the technical solution provided by this invention, the compiler receives at least one compilation task input by the user. The compiler determines whether the compilation task includes multiple branch tasks. If it determines that the compilation task includes multiple branch tasks, it divides the hardware resources of the acquired embedded neural network processor (NPU) according to the multiple branch tasks, generates a hardware allocation result, and generates a first compilation instruction based on the hardware allocation result, the compilation parameters input by the user, and the core parameters required for each NPU core calculation. The compiler sends the first compilation instruction to the scheduler so that the scheduler can schedule the hardware resources according to the first compilation instruction. Thus, the compiler achieves reasonable allocation of NPU hardware resources and improves the NPU's computing speed. [Attached Image Description]

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A flowchart of a task compilation method provided in an embodiment of the present invention;

[0040] Figure 2 A flowchart of another task compilation method provided in an embodiment of the present invention;

[0041] Figure 3 A flowchart of another task compilation method provided in an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of the structure of a task compilation device provided in an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of a compiler provided in an embodiment of the present invention.

Detailed Implementation Methods

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0046] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0047] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0048] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0049] Figure 1 A flowchart of a task core compilation method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes:

[0050] Step 101: The compiler receives at least one compilation task from the user input.

[0051] At least one compilation task includes one or more compilation tasks. The compiler receives one or more compilation tasks from the user through the compiler front end.

[0052] Step 102: The compiler obtains the hardware resources of the NPU.

[0053] As an alternative, when the compilation task is a single compilation task, and the NPU comprises multiple NPU cores, the hardware resources include identifiers for these multiple NPU cores. For example, the identifiers for the multiple NPU cores could be core 1 through core 8.

[0054] As an alternative, when the compilation task consists of multiple compilation tasks, the NPU comprises multiple NPU clusters, and each NPU cluster comprises multiple NPU cores. The hardware resources then include identifiers for the multiple NPU clusters and identifiers for the multiple NPU cores within each NPU cluster. For example, the identifiers for the multiple NPU clusters might be cluster 1 to cluster 3, the identifiers for the multiple NPU cores in cluster 1 might be core 1 to core 4, the identifiers for the multiple NPU cores in cluster 2 might be core 5 to core 8, and the identifiers for the multiple NPU cores in cluster 3 might be core 9 to core 16.

[0055] It should be noted that, for example, step 101 can also be performed after step 102.

[0056] Step 103: The compiler determines whether the compilation task includes multiple branch tasks. If yes, proceed to step 104; otherwise, proceed to step 108.

[0057] The compilation task includes multiple tasks, each of which is a branch task of the compilation task. If the compiler determines that the compilation task includes multiple branch tasks, it means that multiple branch tasks can be executed in parallel. In this case, hardware resources need to be partitioned, and step 104 is executed. If the compiler determines that the compilation task does not include branch tasks, it means that hardware resources do not need to be partitioned, and step 108 is executed.

[0058] As an optional approach, when the compilation task is a single compilation task, the compiler determines whether a compilation task includes multiple branch tasks. If the compiler determines that a compilation task includes multiple branch tasks, it indicates that multiple branch tasks can be executed in parallel. In this case, it is necessary to divide the identifiers of multiple NPU cores, and then proceed to step 104. If the compiler determines that a compilation task does not include branch tasks, it indicates that it is not necessary to divide the identifiers of multiple NPU cores, and then proceed to step 108.

[0059] As an alternative, when there are multiple compilation tasks, the compiler allocates these tasks to the corresponding NPU clusters. In this case, the compiler determines whether the compilation tasks in a single NPU cluster include multiple branch tasks. If the compiler determines that the compilation tasks in a single NPU cluster include multiple branch tasks, it indicates that multiple branch tasks can be executed in parallel. In this case, it is necessary to divide the identifiers of the multiple NPU cores in a single NPU cluster, and then proceed to step 104. If the compiler determines that the compilation tasks in a single NPU cluster do not include branch tasks, it indicates that it is not necessary to divide the identifiers of the multiple NPU cores in a single NPU cluster, and then proceed to step 108.

[0060] Step 104: The compiler divides hardware resources according to multiple branch tasks and generates hardware allocation results.

[0061] As an alternative approach, when there is only one compilation task, the compiler partitions the identifiers of multiple NPU cores based on multiple branch tasks, generating hardware allocation results. For example, the identifiers of the multiple NPU cores include core 1 to core 8, and the multiple branch tasks include branch task 1, branch task 2, and branch task 3. The compiler partitions cores 1 to 8 based on branch task 1, branch task 2, and branch task 3, generating hardware allocation results. These hardware allocation results include allocating the NPU cores corresponding to core 1 and core 2 to branch task 1, allocating the NPU cores corresponding to core 3 and core 4 to branch task 2, and allocating the NPU cores corresponding to cores 5 to 8 to branch task 3. The compiler achieves hardware resource allocation by partitioning the identifiers of multiple NPU cores.

[0062] As an alternative approach, when there are multiple compilation tasks, the compiler partitions the identifiers of multiple NPU clusters based on these tasks, assigning each compilation task to a corresponding NPU cluster. The compiler then partitions the identifiers of multiple NPU cores within a single NPU cluster based on the multiple branch tasks of each compilation task, generating a hardware allocation result. For example, the multiple compilation tasks may include compilation task 1 and compilation task 2; compilation task 1 may include branch task 1 and branch task 2; compilation task 2 may include branch tasks 3 through 6; the identifiers of the multiple NPU clusters may include cluster 1 through cluster 3; the identifiers of multiple NPU cores in cluster 1 may include core 1 through core 4; the identifiers of multiple NPU cores in cluster 2 may include core 5 through core 8; and the identifiers of multiple NPU cores in cluster 3 may include core 9 through core 16. The compiler divides clusters 1 to 3 based on compilation tasks 1 and 2, assigning compilation task 1 to cluster 1 and compilation task 2 to cluster 3. Based on branch tasks 1 and 2 of compilation task 1, the compiler divides cores 1 to 4 of cluster 1, generating hardware allocation results. These results include allocating the NPU cores corresponding to core 1 and core 2 of cluster 1 to branch task 1, and allocating the NPU cores corresponding to core 3 and core 4 of cluster 1 to branch task 2. Similarly, based on branch tasks 3 to 6 of compilation task 2, the compiler divides clusters 3 from core 9 to core 16, generating hardware allocation results. These results include allocating the NPU cores corresponding to core 9 and core 10 of cluster 3 to branch task 3, the NPU cores corresponding to core 11 and core 12 of cluster 3 to branch task 4, the NPU cores corresponding to core 13 and core 14 of cluster 3 to branch task 5, and the NPU cores corresponding to core 15 and core 16 of cluster 3 to branch task 6. The compiler partitions hardware resources by identifying multiple NPU clusters and multiple NPU cores.

[0063] Step 105: The compiler generates the first compilation instruction based on the hardware allocation result, the compilation parameters input by the user, and the core parameters required for each NPU core to perform calculations.

[0064] User-inputted compilation parameters include, but are not limited to, at least one of weight compression parameters, precision parameters, and scheduling parameters. The core parameters required for computation by each NPU core include, but are not limited to, at least one of minimum memory capacity, padding size, kernel parameters, and stride. The compiler generates the first compilation instruction based on the hardware allocation results, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, kernel parameters, and stride.

[0065] Step 106: The compiler sends the first compilation instruction to the scheduler.

[0066] Step 107: The scheduler schedules hardware resources according to the first compilation instruction, and the process ends.

[0067] The scheduler performs tensor segmentation on the feature map and weights of the compilation task based on at least one of the following: hardware allocation results, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride, in order to reduce memory bandwidth. The scheduler then schedules hardware resources based on the segmented compilation tasks. For example, the scheduler performs tensor segmentation on the feature map and weights of the compilation task based on the hardware allocation results, weight compression parameters, minimum memory capacity, padding size, and convolution stride. The scheduler then schedules NPU cores based on the segmented compilation tasks. Each NPU core uses Multiple and Accumulate (MAC) operations to compile the compilation task. The overall computing power of the NPU is related to the number of NPU cores, the frequency of the NPU cores, and the number of operations per clock cycle of the NPU cores.

[0068] Step 108: The compiler generates the second compilation instructions based on the compilation parameters and core parameters.

[0069] The compiler generates a second set of compilation instructions based on at least one of the following: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride.

[0070] Step 109: The compiler sends the second compilation instruction to the scheduler.

[0071] Step 110: The scheduler schedules hardware resources according to the second compilation instruction, and the process ends.

[0072] The compiler performs tensor segmentation on the feature map and weights of the compilation task based on at least one of the following: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride. The scheduler then schedules hardware resources based on the segmented compilation tasks. For example, the scheduler performs tensor segmentation on the feature map and weights of the compilation task based on the weight compression parameters, minimum memory capacity, padding size, and convolution stride, and then schedules NPU cores based on the segmented compilation tasks.

[0073] In this embodiment of the invention, the compiler can employ a Tensor Virtual Machine (TVM) based on the vendor's own Bring Your Own Codegen (BYOC) mechanism. TVM, as an open-source deep learning compiler, provides a universal compilation mode for various AI processors. TVM supports mainstream deep learning front-end frameworks, including TensorFlow, MXNet, PyTorch, and Keras, and can be deployed on a wide range of hardware back-ends, including CPUs, GPUs, NPUs, and various other dedicated accelerators. While TVM compilers have good versatility, they lack specificity. TVM's compilation mode cannot obtain the processor's hardware configuration. TVM uses theoretical algorithms such as machine learning to select the optimal configuration for model compilation, achieving automatic adjustment and scheduling of hardware configuration. Therefore, the practical application effect of TVM is not ideal, and there is still significant room for improvement in hardware utilization and computational efficiency. To compensate for the shortcomings of TVM, the BYOC mechanism can be introduced, allowing hardware vendors to add their own target code generators to TVM. A TVM with a target code generator is a TVM based on the BYOC mechanism.

[0074] In the technical solution of the task core compilation method provided by this invention, the compiler receives at least one compilation task input by the user. The compiler determines whether the compilation task includes multiple branch tasks. If the compilation task includes multiple branch tasks, the compiler divides the hardware resources of the acquired embedded neural network processor (NPU) according to the multiple branch tasks to generate a hardware allocation result. Based on the hardware allocation result, the compilation parameters input by the user, and the core parameters required for each acquired NPU core calculation, the compiler generates a first compilation instruction. The compiler sends the first compilation instruction to the scheduler so that the scheduler can schedule the hardware resources according to the first compilation instruction. Thus, the compiler achieves reasonable allocation of NPU hardware resources and improves the NPU's computing speed.

[0075] Figure 2 A flowchart of another task core compilation method provided in an embodiment of the present invention is shown below. Figure 2 As shown, the method includes:

[0076] Step 201: The compiler receives the compilation task input by the user.

[0077] A compilation task consists of one task, meaning the number of compilation tasks is 1. The compiler receives a compilation task from the user through the compiler front-end.

[0078] Step 202: The compiler obtains the hardware resources of the NPU.

[0079] When the compilation task is a single compilation task, and the NPU includes multiple NPU cores, then the hardware resources include identifiers for these multiple NPU cores. For example, the identifiers for the multiple NPU cores might be core 1 through core 8.

[0080] Step 203: The compiler determines whether the compilation task includes multiple branch tasks. If yes, proceed to step 204; otherwise, proceed to step 211.

[0081] If the compiler determines that the compilation task includes multiple branch tasks, it indicates that multiple branch tasks can be executed in parallel. In this case, hardware resources need to be allocated, and step 204 is executed. If the compiler determines that the compilation task does not include branch tasks, it indicates that hardware resources do not need to be allocated, and step 211 is executed. For example, when the compilation task is a single compilation task, the compiler determines whether the compilation task includes multiple branch tasks. If the compiler determines that the compilation task includes multiple branch tasks, it indicates that multiple branch tasks can be executed in parallel. In this case, the identifiers of multiple NPU cores need to be allocated, and step 204 is executed. If the compiler determines that the compilation task does not include branch tasks, it indicates that the identifiers of multiple NPU cores do not need to be allocated, and the compilation task is completed using the maximum concurrent computing power of all cores, and step 211 is executed.

[0082] Step 204: The compiler clusters multiple branch tasks to generate multiple task category data.

[0083] Task category data includes operator clustering data. The compiler uses the first operator of each branch task as the initial cluster center and iteratively clusters other operators using the K-means algorithm until the sum of the distances from all operators to the cluster centers is minimized. Tasks are then divided according to the operator cluster categories, generating multiple task category data. For example, if a compilation task has multiple branch tasks including branch task 1, branch task 2, and branch task 3, clustering branch task 1, branch task 2, and branch task 3 generates multiple task category data, where these multiple task category data include operator clustering data 1, operator clustering data 2, and operator clustering data 3.

[0084] Step 205: The compiler divides hardware resources according to multiple task category data and generates hardware allocation results.

[0085] Hardware resources include identifiers for multiple NPU cores, such as cores 1 through 8. Compilation tasks include branch task 1, branch task 2, and branch task 3. Multiple task category data include operator clustering data 1, operator clustering data 2, and operator clustering data 3. The compiler partitions cores 1 through 8 based on operator clustering data 1, operator clustering data 2, and operator clustering data 3, generating hardware allocation results. These results include allocating the NPU cores corresponding to core 1 and core 2 to branch task 1, allocating the NPU cores corresponding to core 3 and core 4 to branch task 2, and allocating the NPU cores corresponding to cores 5 through 8 to branch task 3. The compiler partitions hardware resources by identifying the identifiers of multiple NPU cores.

[0086] Step 206: The compiler generates the first compilation instruction based on the hardware allocation result, the compilation parameters input by the user, and the core parameters required for each NPU core to perform calculations.

[0087] User-inputted compilation parameters include, but are not limited to, at least one of weight compression parameters, precision parameters, and scheduling parameters. Core parameters required for computation by each NPU core include, but are not limited to, at least one of minimum memory capacity, padding size, kernel parameters, and stride. The compiler performs binary conversion on at least one of the hardware allocation results, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, kernel parameters, and stride to generate the first compilation instruction.

[0088] Step 207: The compiler sends the first compilation instruction to the scheduler.

[0089] Step 208: The scheduler parses the first compilation instruction to obtain the configuration parameters.

[0090] The configuration parameters include at least one of the following: hardware allocation result, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride. The scheduler performs binary deserialization on the first compiled instruction to obtain at least one of the following: hardware allocation result, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride.

[0091] Step 209: The scheduler determines whether the NPU includes idle cores. If yes, proceed to step 210; otherwise, proceed to step 209.

[0092] If the scheduler determines that the NPU includes idle cores, it means that the idle NPU cores can be scheduled to complete the compilation task, and then proceed to step 210; if the scheduler determines that the NPU does not include idle cores, it means that the NPU cores cannot be scheduled and the compilation task cannot be compiled, then continue to wait for an idle NPU core, and proceed to step 209.

[0093] Step 210: The scheduler schedules hardware resources according to the configuration parameters, and the process ends.

[0094] The scheduler performs tensor segmentation on the feature map and weights of the compilation task based on at least one of the following: hardware allocation results, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride. The scheduler then schedules hardware resources based on the segmented compilation tasks. For example, the scheduler performs tensor segmentation on the feature map and weights of the compilation task based on the hardware allocation results, weight compression parameters, minimum memory capacity, padding size, and convolution stride, and then schedules the NPU cores corresponding to core 1 through core 8 based on the segmented compilation tasks.

[0095] Step 211: The compiler generates the second compilation instructions based on the compilation parameters and core parameters.

[0096] The compiler performs binary conversion on at least one of the following parameters: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride, to generate a second compilation instruction.

[0097] Step 212: The compiler sends the second compilation instruction to the scheduler.

[0098] Step 213: The scheduler parses the second compilation instruction to obtain the configuration parameters.

[0099] The configuration parameters include at least one of the following: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride. The scheduler performs binary deserialization on the second compilation instruction to obtain at least one of the following: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride.

[0100] Step 214: The scheduler determines whether the NPU includes idle cores. If yes, proceed to step 215; otherwise, proceed to step 214.

[0101] If the scheduler determines that the NPU includes idle cores, it means that the idle NPU cores can be scheduled to complete the compilation of the task, and then proceed to step 215; if the scheduler determines that the NPU does not include idle cores, it means that the NPU cores cannot be scheduled and the task cannot be compiled, and it continues to wait for an idle NPU core, then proceed to step 214.

[0102] Step 215: The scheduler schedules hardware resources according to the configuration parameters, and the process ends.

[0103] The scheduler performs tensor segmentation on the feature map and weights of the compilation task based on at least one of the following: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride. The scheduler then schedules hardware resources based on the segmented compilation tasks. For example, the scheduler performs tensor segmentation on the feature map and weights of the compilation task based on the weight compression parameters, minimum memory capacity, padding size, and convolution stride, and then schedules the NPU cores corresponding to cores 1 through 8 based on the segmented compilation tasks.

[0104] In another task core compilation method provided by this invention, the compiler receives a compilation task input by the user and obtains the hardware resources of the NPU. If the compiler determines that the compilation task includes multiple branch tasks, it clusters the multiple branch tasks to generate multiple task category data, and divides the hardware resources according to the multiple task category data to generate a hardware allocation result. The compiler generates a first compilation instruction based on the hardware allocation result, the compilation parameters input by the user, and the core parameters required for each NPU core calculation, and sends the first compilation instruction to the scheduler. The scheduler parses the first compilation instruction to obtain configuration parameters. If the scheduler determines that the NPU includes idle cores, the scheduler schedules the hardware resources according to the configuration parameters. Thus, by adaptively dividing the compilation task through the compiler, the hardware resources of the NPU are reasonably allocated, the utilization rate of the NPU hardware resources is improved, the computing speed of the NPU is increased, and the performance of the neural network model executing tasks on the NPU is improved.

[0105] Figure 3 A flowchart of another task core compilation method provided in an embodiment of the present invention is shown below. Figure 3 As shown, the method includes:

[0106] Step 301: The compiler receives multiple compilation tasks input by the user.

[0107] A compilation task consists of multiple compilation tasks, meaning the number of compilation tasks is greater than or equal to two. The compiler receives multiple compilation tasks from the user through the compiler front-end.

[0108] Step 302: The compiler obtains the hardware resources of the NPU.

[0109] When there are multiple compilation tasks, the NPU includes multiple NPU clusters, and each NPU cluster includes multiple NPU cores. The hardware resources then include identifiers for the multiple NPU clusters and identifiers for the multiple NPU cores within each NPU cluster. For example, the identifiers for the multiple NPU clusters might be cluster 1 to cluster 3, the identifiers for the multiple NPU cores in cluster 1 might be core 1 to core 4, the identifiers for the multiple NPU cores in cluster 2 might be core 5 to core 8, and the identifiers for the multiple NPU cores in cluster 3 might be core 9 to core 16.

[0110] Step 303: The compiler determines whether the number of compilation tasks is less than or equal to the number of NPU clusters. If yes, proceed to step 304; otherwise, proceed to step 318.

[0111] If the compiler determines that the number of compilation tasks is less than or equal to the number of NPU clusters, it indicates that each compilation task can be executed in parallel within the NPU clusters. In this case, it is necessary to allocate identifiers for the NPU clusters, and then proceed to step 304. If the compiler determines that the number of compilation tasks is greater than the number of NPU clusters, it indicates that it is necessary to allocate identifiers for the NPU clusters based on the computational requirements of each compilation task, and then proceed to step 318.

[0112] Step 304: The compiler assigns each compilation task to the corresponding NPU cluster.

[0113] The compiler assigns each compilation task to the corresponding NPU cluster based on the number of NPU cores in a single NPU cluster.

[0114] In this embodiment of the invention, NPU cores in a single NPU cluster can jointly access shared inter-core memory within the single NPU cluster.

[0115] Step 305: The compiler determines whether the compilation task includes multiple branch tasks. If yes, proceed to step 306; otherwise, proceed to step 313.

[0116] If the compiler determines that the compilation task includes multiple branch tasks, it indicates that multiple branch tasks can be executed in parallel. In this case, hardware resources need to be allocated, and step 306 is executed. If the compiler determines that the compilation task does not include branch tasks, it indicates that hardware resources do not need to be allocated, and step 313 is executed. For example, when there are multiple compilation tasks, the compiler allocates multiple compilation tasks to the corresponding NPU clusters. At this time, the compiler determines whether the compilation tasks in a single NPU cluster include multiple branch tasks. If the compiler determines that the compilation tasks in a single NPU cluster include multiple branch tasks, it indicates that multiple branch tasks can be executed in parallel. In this case, the identifiers of multiple NPU cores in a single NPU cluster need to be allocated, and step 306 is executed. If the compiler determines that the compilation tasks in a single NPU cluster do not include branch tasks, it indicates that the identifiers of multiple NPU cores in a single NPU cluster do not need to be allocated, and the compilation of the task is completed using the maximum concurrent computing power of all cores in a single NPU cluster, and step 313 is executed.

[0117] In this embodiment of the invention, the compiler executes in parallel to determine whether the compilation task in each NPU cluster includes multiple branch tasks.

[0118] Step 306: The compiler clusters multiple branch tasks to generate multiple task category data.

[0119] The compiler uses the first operator of each branch task in a single NPU cluster as the initial cluster center. It then iteratively clusters the other operators of each branch task using the K-means algorithm until the sum of the distances from all operators to the cluster centers is minimized. Tasks are then partitioned based on the operator cluster categories, generating task category data. For example, when a compilation task in a single NPU cluster has multiple branch tasks including branch task 1 and branch task 2, branch task 1 and branch task 2 are clustered to generate multiple task category data, which include operator cluster data 1 and operator cluster data 2.

[0120] Step 307: The compiler divides hardware resources according to multiple task category data and generates hardware allocation results.

[0121] Hardware resources include identifiers for multiple NPU clusters and identifiers for multiple NPU cores within a single NPU cluster. For example, multiple compilation tasks include compilation task 1 and compilation task 2, compilation task 1 includes branch task 1 and branch task 2, compilation task 2 includes branch task 3 to branch task 6, multiple task category data includes operator clustering data 1 to operator clustering data 6, identifiers for multiple NPU clusters include cluster 1 to cluster 3, identifiers for multiple NPU cores in cluster 1 include core 1 to core 4, identifiers for multiple NPU cores in cluster 2 include core 5 to core 8, and identifiers for multiple NPU cores in cluster 3 include core 9 to core 16. The compiler partitions clusters 1 through 3 based on compilation tasks 1 and 2. It assigns compilation task 1 to cluster 1 and compilation task 2 to cluster 3. The compiler then partitions cores 1 through 4 of cluster 1 based on operator clustering data 1 and operator clustering data 2 from compilation task 1, generating hardware allocation results. These results include allocating the NPU cores corresponding to core 1 and core 2 of cluster 1 to branch task 1, and allocating the NPU cores corresponding to core 3 and core 4 of cluster 1 to branch task 2. The compiler then further partitions clusters 1 through 3 based on the operator clustering data 1 and operator clustering data 2 from compilation task 2. Sub-clustering data 3 to operator clustering data 6 divide cores 9 to 16 of cluster 3, generating hardware allocation results. These results include allocating the NPU cores corresponding to core 9 and core 10 of cluster 3 to branch task 3; allocating the NPU cores corresponding to core 11 and core 12 of cluster 3 to branch task 4; allocating the NPU cores corresponding to core 13 and core 14 of cluster 3 to branch task 5; and allocating the NPU cores corresponding to core 15 and core 16 of cluster 3 to branch task 6. The compiler implements hardware resource allocation by dividing multiple NPU clusters and multiple NPU cores using identifiers.

[0122] Step 308: The compiler generates the first compilation instruction based on the hardware allocation result, the compilation parameters input by the user, and the core parameters required for each NPU core to perform calculations.

[0123] User-inputted compilation parameters include, but are not limited to, at least one of weight compression parameters, precision parameters, and scheduling parameters. Core parameters required for computation by each NPU core include, but are not limited to, at least one of minimum memory capacity, padding size, kernel parameters, and stride. The compiler performs binary conversion on at least one of the hardware allocation results, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, kernel parameters, and stride to generate the first compilation instruction.

[0124] Step 309: The compiler sends the first compilation instruction to the scheduler.

[0125] Step 310: The scheduler parses the first compilation instruction to obtain the configuration parameters.

[0126] The configuration parameters include at least one of the following: hardware allocation result, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride. The scheduler performs binary deserialization on the first compiled instruction to obtain at least one of the following: hardware allocation result, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride.

[0127] Step 311: The scheduler determines whether the NPU includes idle cores. If yes, proceed to step 312; otherwise, proceed to step 311.

[0128] If the scheduler determines that the NPU includes idle cores, it means that the idle NPU cores can be scheduled to complete the compilation task, and then proceed to step 312; if the scheduler determines that the NPU does not include idle cores, it means that the NPU cores cannot be scheduled and the compilation task cannot be compiled, then continue to wait for an idle NPU core, and proceed to step 311.

[0129] Step 312: The scheduler schedules hardware resources according to the configuration parameters, and the process ends.

[0130] The scheduler performs tensor segmentation on the feature map and weights of the compilation task based on at least one of the following: hardware allocation results, weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride. The scheduler then schedules hardware resources based on the segmented compilation task. For example, the scheduler performs tensor segmentation on the feature map and weights of compilation task 1 based on the hardware allocation results, weight compression parameters, minimum memory capacity, padding size, and convolution stride. The scheduler then schedules NPU cores corresponding to core 1 through core 4 of cluster 1 based on the segmented compilation task 1.

[0131] Step 313: The compiler generates the second compilation instructions based on the compilation parameters and core parameters.

[0132] The compiler performs binary conversion on at least one of the following parameters: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride, to generate a second compilation instruction.

[0133] Step 314: The compiler sends the second compilation instruction to the scheduler.

[0134] Step 315: The scheduler parses the second compilation instruction to obtain the configuration parameters.

[0135] The configuration parameters include at least one of the following: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride. The scheduler performs binary deserialization on the second compilation instruction to obtain at least one of the following: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride.

[0136] Step 316: The scheduler determines whether the NPU includes idle cores. If yes, proceed to step 317; otherwise, proceed to step 316.

[0137] If the scheduler determines that the NPU includes idle cores, it means that the idle NPU cores can be scheduled to complete the compilation task, and then proceed to step 317; if the scheduler determines that the NPU does not include idle cores, it means that the NPU cores cannot be scheduled and the compilation task cannot be compiled, and it continues to wait for an idle NPU core, then proceed to step 316.

[0138] Step 317: The scheduler schedules hardware resources according to the configuration parameters, and the process ends.

[0139] The scheduler performs tensor segmentation on the feature map and weights of the compilation task based on at least one of the following: weight compression parameters, precision parameters, scheduling parameters, minimum memory capacity, padding size, convolution kernel parameters, and convolution stride. The scheduler then schedules hardware resources based on the segmented compilation task. For example, the scheduler performs tensor segmentation on the feature map and weights of compilation task 1 based on the weight compression parameters, minimum memory capacity, padding size, and convolution stride. Based on the segmented compilation task 1, the scheduler schedules NPU cores corresponding to core 1 through core 4 of cluster 1.

[0140] Step 318: The compiler calculates the computational requirements for each compilation task.

[0141] The computational requirements include time. The compiler calculates the time for each compilation task based on the number of parameters for each task.

[0142] Step 319: The compiler selects a specific number of compilation tasks from the multiple compilation tasks input by the user. The computational requirements of the specific number of compilation tasks are all less than the computational requirements of the other compilation tasks input by the user. The specific number is equal to the number of NPU clusters.

[0143] The compiler selects a specific number of compilation tasks based on the computational requirements of each compilation task, and uses the Shortest Job First (SJF) algorithm to prioritize the compilation tasks with smaller computational requirements and allocate them to the corresponding NPU clusters.

[0144] In this embodiment of the invention, other compilation tasks with high computational demands will wait to be assigned to the corresponding NPU cluster.

[0145] Step 320: The compiler assigns each of the specified number of compilation tasks to the corresponding NPU cluster and executes step 305 in parallel.

[0146] The compiler allocates a specific number of compilation tasks to the corresponding NPU cluster based on the number of NPU cores in the NPU cluster. The compiler determines whether the compilation tasks in a single NPU cluster include multiple branch tasks.

[0147] In another task core compilation method provided by this invention, the compiler receives multiple compilation tasks input by the user and obtains NPU hardware resources. If the compiler determines that the number of compilation tasks is less than or equal to the number of NPU clusters, it allocates each compilation task to the corresponding NPU cluster. If the compiler determines that the compilation tasks include multiple branch tasks, it clusters the multiple branch tasks to generate multiple task category data and divides the hardware resources according to the multiple task category data to generate hardware allocation results corresponding to each branch task. The compiler generates a first compilation instruction based on the hardware allocation results, the compilation parameters input by the user, and the core parameters required for each NPU core calculation, and sends the first compilation instruction to the scheduler. The scheduler parses the first compilation instruction to obtain configuration parameters. If the scheduler determines that the NPU includes idle cores, it schedules the hardware resources according to the configuration parameters. Thus, by adaptively dividing the compilation tasks through the compiler, the NPU hardware resources are rationally allocated, the utilization rate of NPU hardware resources is improved, the NPU's computing speed is increased, and the performance of neural network models executing tasks on the NPU is improved.

[0148] Figure 4 This is a schematic diagram of the structure of a task compilation device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the device includes: a receiving module 11, an acquisition module 12, a judgment module 13, a generation module 14, and a sending module 15.

[0149] The receiving module 11 is connected to the acquiring module 12, the acquiring module 12 is connected to the judging module 13, the judging module 13 is connected to the generating module 14, and the generating module 14 is connected to the sending module 15.

[0150] The receiving module 11 is used to receive at least one compilation task input by the user; the acquiring module 12 is used to acquire the hardware resources of the NPU; the judging module 13 is used to judge whether the compilation task includes multiple branch tasks; the generating module 14 is used to divide the hardware resources according to the multiple branch tasks and generate a hardware allocation result if the judging module 13 judges that the compilation task includes multiple branch tasks; and to generate a first compilation instruction according to the hardware allocation result, compilation parameters and core parameters; the sending module 15 is used to send the first compilation instruction to the scheduler so that the scheduler can schedule the hardware resources according to the first compilation instruction.

[0151] In this embodiment of the invention, the generation module 14 includes a first generation submodule 141 and a second generation submodule 142. The first generation submodule 141 and the second generation submodule 142 are connected.

[0152] The first generation submodule 141 is used to cluster multiple branch tasks and generate multiple task category data; the second generation submodule 142 is used to divide hardware resources according to the multiple task category data and generate hardware allocation results.

[0153] In this embodiment of the invention, the generation module 14 is further configured to generate a second compilation instruction based on the compilation parameters and core parameters if the judgment module 13 determines that the compilation task does not include a branch task; the sending module 15 is further configured to send the second compilation instruction to the scheduler so that the scheduler can schedule hardware resources according to the second compilation instruction.

[0154] In this embodiment of the invention, there are multiple compilation tasks, and the device further includes an allocation module 16. The allocation module 16 is connected to the judgment module 13.

[0155] The judgment module 13 is also used to determine whether the number of compilation tasks is less than or equal to the number of NPU clusters; the allocation module 16 is used to allocate each compilation task to the corresponding NPU cluster if the judgment module 13 determines that the number of compilation tasks is less than or equal to the number of NPU clusters, and to trigger the judgment module 13 to execute the step of determining whether the compilation task includes multiple branch tasks in parallel.

[0156] In this embodiment of the invention, there are multiple compilation tasks, and the device further includes a calculation module 17 and a selection module 18. The calculation module 17 is connected to the judgment module 13 and the selection module 18.

[0157] The calculation module 17 is used to calculate the computational requirements of each compilation task if the judgment module 13 determines that the number of compilation tasks is greater than the number of NPU clusters. The selection module 18 is used to select a specific number of compilation tasks from the multiple compilation tasks input by the user. The computational requirements of the specific number of compilation tasks are all less than the computational requirements of other compilation tasks in the multiple compilation tasks input by the user. The specific number is equal to the number of NPU clusters. The allocation module 16 is also used to allocate each compilation task in the specific number of compilation tasks to the corresponding NPU cluster, triggering the judgment module 13 to execute the step of judging whether the compilation task includes multiple branch tasks in parallel.

[0158] In the technical solution of the task compilation device provided in this embodiment of the invention, the compiler receives at least one compilation task input by the user. The compiler determines whether the compilation task includes multiple branch tasks. If it is determined that the compilation task includes multiple branch tasks, the compiler divides the hardware resources of the acquired embedded neural network processor (NPU) according to the multiple branch tasks, generates a hardware allocation result, and generates a first compilation instruction based on the hardware allocation result, the compilation parameters input by the user, and the core parameters required for the calculation of each acquired NPU core. The compiler sends the first compilation instruction to the scheduler so that the scheduler can schedule the hardware resources according to the first compilation instruction. Thus, the compiler achieves reasonable allocation of NPU hardware resources and improves the NPU's computing speed.

[0159] Figure 5 A schematic diagram of a compiler provided for an embodiment of the present invention, such as... Figure 5 As shown, the compiler 21 includes a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, it implements the task compilation method in the embodiment. To avoid repetition, it will not be described in detail here.

[0160] The compiler 21 includes, but is not limited to, a processor 211 and a memory 212. Those skilled in the art will understand that... Figure 5 This is merely an example of compiler 21 and does not constitute a limitation on compiler 21. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the compiler may also include input / output devices, network access devices, buses, etc.

[0161] The processor 211 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0162] The memory 212 can be an internal storage unit of the compiler 21, such as the hard disk or RAM of the compiler 21. The memory 212 can also be an external storage device of the compiler 21, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the compiler 21. Furthermore, the memory 212 can include both internal storage units of the compiler 21 and external storage devices. The memory 212 is used to store computer programs and other programs and data required by network devices. The memory 212 can also be used to temporarily store data that has been output or will be output.

[0163] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0164] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A task compilation method, characterized in that, The method includes: At least one compilation task that receives user input; Determine whether the compilation task includes multiple branch tasks; If it is determined that the compilation task includes multiple branch tasks, the hardware resources of the acquired embedded neural network processor (NPU) are divided according to the multiple branch tasks to generate hardware allocation results. Based on the hardware allocation results, the compilation parameters input by the user, and the core parameters required for each NPU core to compute, a first compilation instruction is generated; The first compilation instruction is sent to the scheduler so that the scheduler can schedule the hardware resources according to the first compilation instruction; The number of compilation tasks is multiple, and the method further includes: If it is determined that the number of compilation tasks is greater than the number of NPU clusters, then the computational requirements of each compilation task are calculated. A specific number of compilation tasks are selected from multiple compilation tasks input by the user. The computational requirements of these specific number of compilation tasks are all less than the computational requirements of other compilation tasks input by the user. The specific number is equal to the number of NPU clusters. Each compilation task in a specific number of compilation tasks is assigned to the corresponding NPU cluster, and the step of determining whether the compilation task includes multiple branch tasks is executed in parallel.

2. The method according to claim 1, characterized in that, The step of allocating the hardware resources of the acquired embedded neural network processor (NPU) based on multiple branch tasks to generate hardware allocation results includes: Cluster multiple branch tasks to generate multiple task category data; The hardware resources are divided according to data from multiple task categories to generate hardware allocation results.

3. The method according to claim 1, characterized in that, Also includes: If it is determined that the compilation task does not include branch tasks, then a second compilation instruction is generated based on the compilation parameters and core parameters; The second compilation instruction is sent to the scheduler so that the scheduler can schedule the hardware resources according to the second compilation instruction.

4. The method according to claim 1, characterized in that, The number of compilation tasks is multiple. Before determining whether the compilation tasks include multiple branch tasks, the method further includes: If it is determined that the number of compilation tasks is less than or equal to the number of NPU clusters, then each compilation task is assigned to the corresponding NPU cluster, and the step of determining whether the compilation task includes multiple branch tasks is executed in parallel.

5. A task compilation apparatus, characterized in that, The device includes: The receiving module is used to receive at least one compilation task from user input; The acquisition module is used to acquire NPU hardware resources; The judgment module is used to determine whether the compilation task includes multiple branch tasks; The generation module is used to divide the hardware resources according to the multiple branch tasks and generate a hardware allocation result if the judgment module determines that the compilation task includes multiple branch tasks; and to generate a first compilation instruction according to the hardware allocation result, compilation parameters and core parameters. The sending module is used to send the first compilation instruction to the scheduler so that the scheduler can schedule the hardware resources according to the first compilation instruction; The number of compilation tasks is multiple, and the device further includes: a calculation module, a selection module, and an allocation module; The calculation module is used to calculate the computational requirements of each compilation task if the judgment module determines that the number of compilation tasks is greater than the number of NPU clusters. The selection module is used to select a specific number of compilation tasks from multiple compilation tasks input by the user. The computational requirements of the specific number of compilation tasks are all less than the computational requirements of other compilation tasks in the multiple compilation tasks input by the user. The specific number is equal to the number of NPU clusters. The allocation module is used to allocate each of the specified number of compilation tasks to the corresponding NPU cluster, and triggers the judgment module to perform the step of judging whether the compilation task includes multiple branch tasks in parallel.

6. The apparatus according to claim 5, characterized in that, The generation module includes: The first generation submodule is used to cluster multiple branch tasks and generate multiple task category data; The second generation submodule is used to divide the hardware resources according to multiple task category data and generate hardware allocation results.

7. The apparatus according to claim 5, characterized in that, Also includes: The generation module is also used to generate a second compilation instruction based on the compilation parameters and core parameters if the judgment module determines that the compilation task does not include a branch task. The sending module is also used to send the second compilation instruction to the scheduler, so that the scheduler can schedule the hardware resources according to the second compilation instruction.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method of any one of claims 1 to 4.

9. A compiler, characterized in that, include: One or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs including instructions that, when executed by the compiler, cause the compiler to perform the method of any one of claims 1 to 4.

Images