Instruction-level parallel scheduling method and device in deep learning compiler

Abstract

The invention provides an instruction-level parallel scheduling method and device in a deep learning compiler, and the method comprises the steps: decomposing a calculation graph of a deep learning task, decomposing a task represented by each calculation graph node on the calculation graph into a hardware calculation instruction according to a current task decomposition scheme, and obtaining an instruction set of the calculation graph; according to the data flow dependency relationship between the nodes on the calculation graph, obtaining a dependency relationship graph between the instructions in the instruction set; dividing the hardware calculation instructions in the instruction set into each sub-core in the processor; according to the dependency graph, determining a directed acyclic graph containing the dependency among the hardware calculation instructions on the sub-cores in the processor, and according to an emission model of the processor and the reciprocal throughput rate of the hardware calculation instructions, determining the execution sequence of the hardware calculation instructions divided to the sub-cores on the sub-cores, and the processor executes the instruction scheduling to obtain an execution result of the deep learning task.

Description

Technical Field

[0001] The present invention relates to the technical fields of program compilation, program instruction-level parallelism, deep learning compilers, deep learning, and processor design, and in particular to an instruction-level parallel scheduling method, device, processor, electronic device, and storage medium in a deep learning compiler. Background Art

[0002] Currently, general-purpose intelligent processors that combine general-purpose computing capabilities with specialized deep learning computing capabilities are widely used in personal computers and data centers. Unlike general-purpose intelligent processors, this type of processor combines the advantages of general-purpose processors and specialized intelligent processors, and can provide efficient computing performance and flexibility in various application scenarios. More specifically, in general-purpose intelligent processors represented by GPUs, two main computing units are usually implemented simultaneously within a hardware thread: intelligent computing units (such as Tensor Cores in GPUs) and general-purpose computing units (such as CUDACores in GPUs). Intelligent computing units, with the help of specialized tensor computing hardware, can efficiently perform tensor calculations common in deep learning tasks. On the other hand, general-purpose computing units mainly perform vector calculations and have a wider instruction set and flexibility, allowing general-purpose intelligent processors to simultaneously provide high-performance general-purpose computing capabilities. Both intelligent computing units and general-purpose computing units in general-purpose intelligent processors generally provide high computing performance, and in many specific implementations, they may perform calculations in parallel using instruction-level parallelism (ILP). In addition to the versatility and flexibility of general-purpose computing units, it is possible to use both of them to perform deep learning calculations simultaneously, improving the hardware computing throughput while reducing synchronization overhead through reasonable task scheduling, accelerating the training and inference process of deep learning models, and achieving ILP optimization of deep learning programs.

[0003] In order to maximize the inherent parallelism in deep learning computing tasks to improve computing performance, general-purpose intelligent processors are usually designed as hierarchical massively parallel multi-core processors with parallel sub-cores at multiple levels and shared storage between sub-cores at each level. Figure 1A schematic diagram of the structure of the above-mentioned general-purpose intelligent processor 1, taking a GPU as an example, is shown. The overall structure is divided into three levels, with each L0-level sub-core executing a hardware thread, which contains parallel intelligent computing units T and general-purpose computing units V. The hardware characteristics of the general-purpose intelligent processor make it have a complex programming model. When programming, it is necessary to consider details such as the decomposition of tasks and how tasks are scheduled on sub-cores, as well as the mapping between tasks and hardware instructions, which greatly increases its programming difficulty. In order to simplify the programming process and fully utilize the computing power of the hardware, end-to-end deep learning compilers are widely used in the training and deployment of deep learning models. Specifically, end-to-end deep learning compilers can convert high-level deep learning models, mainly represented by computational graphs, into programs that can be executed by the target intelligent processor hardware through a series of data transmission, computational task scheduling and optimization. In practical applications, end-to-end deep learning compilers greatly simplify the training and deployment process of deep learning models, while also making more comprehensive use of the computing power of general-purpose intelligent processors.

[0004] Among existing deep learning compiler implementations, task-block-based designs have achieved promising results and widespread adoption. This deep learning compiler divides the compilation process into three phases: task decomposition, task scheduling, and code generation. Task decomposition decomposes a given deep learning task into a set of parallelizable subtasks. Task scheduling, given a given task decomposition scheme, finds a task partitioning scheme that allocates each subtask to the target processor's subcores. Finally, code generation generates a program executable on the target processor based on the decomposition and partitioning schemes obtained in the previous phase. In this compilation process, the quality of the scheduling scheme provided by task scheduling can largely determine the performance of the resulting compiler-generated program. Furthermore, if the task blocks (subtasks) generated by task decomposition have a direct correspondence with hardware computational instructions, task scheduling can directly determine whether the hardware computational instructions in the resulting compiler-generated program can be efficiently executed in parallel across the different computational units of a general-purpose intelligent processor in an ILP format. Therefore, in task-block-based compiler design, task scheduling is a critical technology for achieving ILP optimization of deep learning programs.

[0005] Among existing designs, Rammer is a typical deep learning compiler based on task blocks. Its design is mainly based on a task block abstraction called rTask, which divides the compilation process of deep learning tasks into the following parts: Figure 2 The three main steps are:

[0006] Task Decomposition 21: For a given computational graph representation of a deep learning model, Rammer first decomposes each computational graph node into a set of rTasks. Among them, rTask is the core abstraction of Rammer. Each rTask represents a subtask, that is, a task block, in the computation performed by the computational graph node, and each rTask is homogeneous, that is, the task performed by each rTask is the same algorithm, or in other words, it is a subtask of the task represented by the original computational graph node. For example, if a matrix multiplication operator is decomposed into multiple rTasks, each rTask still performs matrix multiplication; the scale of the tasks performed by each rTask is consistent. For example, if a 32x32 addition operator is split into 16 rTasks, then a possible legal split is to make each rTask perform an 8x8 addition.

[0007] For each set of rTasks obtained by disassembly, Rammer calls it an rOperator. There is a one-to-one correspondence between rOperator and the computational graph nodes in the original computational graph, and inherits the data dependency relationship of the original computational graph nodes to form a data flow graph (DFG) between rOperators. The data dependency relationship in this data flow graph is passed to the rTask, and the data dependency relationship between the rTasks is obtained. In the subsequent task scheduling stage, the execution order of the rTasks obtained by task scheduling must follow this dependency relationship. In addition, Rammer uses deep learning operator libraries (such as NVIDIA's cuDNN) or existing deep learning compilers (such as TVM) to obtain one or more code implementations of each rTask under the target intelligent processor, and obtains the execution time of the rTask under each implementation by measurement.

[0008] Task Scheduling 22: After completing the task decomposition, Rammer performs task scheduling. The rTask obtained after decomposition is divided into multiple kernel functions (kernel functions) through the scheduling algorithm. Each kernel records the mapping relationship between the rTask and the sub-core in the target intelligent processor, as well as the execution order of the rTask. Rammer's scheduling algorithm maintains a kernel list during execution. At each step of the algorithm, (1) an independent rOperator is taken out from the current rOperator data flow graph, and all the rTasks contained in it are placed in the currently open kernel. (2) For the currently open kernel, for all the rTasks contained in it, a better schedule is found based on the execution time measurement results of the rTask on the target intelligent processor. Specifically, it finds a mapping relationship between the rTask contained in the currently open kernel and the sub-cores in the target intelligent processor, as well as the execution order of the rTasks, and based on the execution time measurement results of each rTask implementation obtained in the task decomposition phase, by adding the measured execution time of the rTasks on each sub-core, it predicts the execution time of the kernel after the new rTask is added. If the rTask in the removed rOperator is added to the currently open kernel, a program implementation with a predicted execution time not longer can be found, then the scheduling result is accepted. Otherwise, (3) the scheduling result is rejected, the currently open kernel is closed and appended to the kernel list, and then all rTasks in the selected rOperator are scheduled to a newly opened kernel. The above scheduling algorithm is executed in a loop until all rOperators in the data flow graph are scheduled. At this time, the algorithm obtains a kernel list, which is the result of task scheduling.

[0009] Code Generation 23: After completing task scheduling, Rammer generates code for the scheduling results. Specifically, based on the mapping relationship between the rTasks recorded in the kernel and the sub-cores in the target intelligent processor, as well as the execution order of the rTasks, Rammer sequentially converts each rTask in the kernel list into a function call in the programming language and programming model of the target hardware, combined with the rTask implementations obtained in the task decomposition phase, to generate a program that can actually be executed on the target intelligent processor.

[0010] Overall, Rammer establishes a deep learning task decomposition and scheduling mechanism based on its designed task block abstraction, rTask. rTask and its code implementation are extracted from deep learning operator libraries or from code generated by existing deep learning compilers. This allows it to provide a relatively rich and fine-grained decomposition of deep learning operators. Furthermore, combined with its designed rTask scheduling algorithm, its design has the potential to fully utilize the computing power of intelligent processors by combining different rTask implementations.

[0011] As mentioned above, existing task-block-based deep learning compilers, represented by Rammer, can relatively well solve the problems of task decomposition and scheduling, as well as task-to-instruction mapping, during the deep learning compilation process. However, despite this, existing task-block-based deep learning compilers, represented by Rammer, cannot achieve ILP optimization for general-purpose intelligent processors. That is, they generate code that efficiently utilizes multiple computing units simultaneously within the processor in the form of instruction-level parallelism (ILP), resulting in a waste of hardware resources and missing the opportunity to further improve the execution performance of deep learning tasks. In the task decomposition and task scheduling stages, the key points for deep learning compilers to effectively achieve ILP optimization are:

[0012] 1. In the task decomposition stage, the deep learning compiler should first perform fine-grained decomposition of a given deep learning task. The resulting subtasks should have a direct correspondence with the instructions executed by various computing units in a general-purpose intelligent processor. That is, the calculation of the given task is decomposed into subtasks that can be completed by instructions executed in parallel by multiple computing units in the form of ILP. This is the prerequisite for achieving ILP optimization.

[0013] 2. During the task scheduling phase, the compiler must allocate the decomposed subtasks to the various subcores of the general-purpose intelligent processor and may need to find the appropriate order for executing the subtasks in parallel on each subcore using ILP. This requires, on the one hand, that the compiler be able to perceive the performance differences between different scheduling schemes in the presence of ILP in order to find a schedule with better performance. On the other hand, given the potentially large search space for scheduling, the compiler's scheduling algorithm should provide appropriate search heuristics for ILP optimization to find a high-quality schedule as quickly as possible.

[0014] Regarding the existing technology Rammer, in terms of task decomposition, on the one hand, Rammer completely relies on extracting rTask from the existing deep learning library code or the code generated by the deep learning compiler. So far, no deep learning library contains code that implements ILP optimization, and no deep learning compiler can generate such code. On the other hand, in this design, the granularity of the task block is not controlled by the compiler, and the compiler lacks awareness of the implementation method of the task block. Therefore, the compiler cannot determine the correspondence between the subtasks obtained by disassembly and the hardware instructions, which makes it impossible for the compiler to achieve the premise of ILP optimization in the task decomposition stage. (2) In terms of task scheduling, as mentioned above, Rammer's task scheduling algorithm uses a simple delay addition performance model when evaluating the performance of the scheduling, which makes it unable to correctly estimate the performance of a certain scheduling when ILP exists. Summary of the Invention

[0015] The purpose of the present invention is to solve the problem that the task scheduling technology used by the above-mentioned existing deep learning compilers cannot achieve instruction-level parallel optimization. A technology for general intelligent processors is proposed, which can perform instruction-level parallel scheduling of hardware computing instructions when the deep learning compiler can decompose a given task into multiple hardware computing instructions that can be parallelized by ILP.

[0016] Specifically, if Figure 4 The present invention proposes an instruction-level parallel scheduling method in a deep learning compiler, which includes:

[0017] The task decomposition step 41 decomposes the computation graph of the deep learning task, decomposing the task represented by each computation graph node on the computation graph into hardware computation instructions according to the current task decomposition scheme, thereby obtaining an instruction set of the computation graph; and obtaining a dependency graph between instructions in the instruction set based on the data flow dependency between nodes on the computation graph;

[0018] Instruction partitioning step 42, the hardware computing instructions in the instruction set are partitioned into the respective sub-cores in the processor;

[0019] In the instruction sorting step 43, based on the dependency graph, a directed acyclic graph containing the dependency relationships between the hardware computing instructions on the sub-cores of the processor is determined, and based on the processor's emission model and the reciprocal throughput of the hardware computing instructions, the execution order of the hardware computing instructions allocated to the sub-cores on the sub-cores is determined to obtain the final instruction schedule of the deep learning task. The processor executes the instruction schedule to obtain the execution result of the deep learning task.

[0020] The instruction-level parallel scheduling method in the deep learning compiler, wherein the instruction partitioning step includes:

[0021] Select the sub-core with the smallest current load in the processor as the target core;

[0022] The hardware computing instructions in the instruction set are scored based on whether they have data reuse and / or synchronization relationships with existing hardware computing instructions in the target core, and the hardware computing instructions with the highest scores are selected and allocated to the target core.

[0023] The instruction-level parallel scheduling method in the deep learning compiler, wherein the instruction sorting step includes:

[0024] Initially, a variable t is maintained for each computational unit u in the processor. u ∈{t0…t U-1}, t u =0;

[0025] Loop through the steps to find all t u The computing units with a value of 0 form the set {u0}. Find all currently executable instructions executed in {u0} from the directed acyclic graph DAG as candidate instructions; according to the reciprocal throughput T of the candidate instructions -1 , the functional unit u executed, the data used by the candidate instructions, score the candidate instructions, select the candidate instruction with the highest score and append it to the list, so that t u =t u +T -1 ; Set the remaining time t of the emission of all computing units u u Subtract the minimum remaining time of all current calculation units to get

[0026] The sequential determination step is executed again in the loop until all hardware computing instructions on the sub-core are added to the list, and the hardware computing instructions in the list are in the execution order.

[0027] The instruction-level parallel scheduling method in the deep learning compiler includes the following steps:

[0028] Step 1: Given the upper limit N of random walk steps and the random walk sampling volume R, randomly select a decomposition scheme from all enumerated task decomposition schemes as the current task decomposition scheme;

[0029] Step 2: Mark the selected current task decomposition scheme as explored, and perform the instruction partitioning step and the instruction sorting step on it to obtain the instruction schedule s;

[0030] Step 3: Predict the execution time t of the processor to execute the instruction schedule s s ;

[0031] Step 4: Sample R unexplored decomposition schemes from all enumerated task decomposition schemes according to their similarity to the current task decomposition scheme, and obtain their instruction schedules {s1, ..., s R}, and the execution time predicted by the performance model

[0032] Step 5: Select the instruction schedule with the shortest predicted execution time The decomposition scheme corresponding to s′ is taken as the currently selected decomposition scheme.

[0033] Step 6: If the current number of walking steps is less than the upper limit N, the current number of walking steps is increased by 1, and step 4 is executed again. Otherwise, the instruction schedule corresponding to the currently selected decomposition scheme is returned as the final instruction schedule.

[0034] The instruction-level parallel scheduling method in the deep learning compiler uses a performance model to predict the execution time t of the processor to execute the instruction schedule s. s ;

[0035] The performance model is used to predict the time consumed by a sub-core to execute a given instruction schedule. The processor includes multiple sub-cores, and the longest execution time is selected as the execution time of the processor to execute the instruction schedule.

[0036] The performance model consists of a sub-core instruction emission simulation, a computing unit simulation, and a data read and write simulation module; the instruction emission simulation module is used to receive the instruction sorting result, and emit the instruction corresponding to the instruction to the computing unit simulation module according to the processor's emission model; the computing simulation module executes the instruction according to the sub-core's time; the computing simulation module also sends the instruction's data read and write request to the data read and write simulation module, and the data read and write simulation module simulates the instruction's data reading and writing time consumption based on the data reuse between instructions, and transmits a blocking signal back to the instruction emission simulation module through the computing simulation module when a structural conflict occurs in the data read and write request.

[0037] The present invention also proposes an instruction-level parallel scheduling device in a deep learning compiler, which includes:

[0038] The task decomposition module is used to decompose the computational graph of the deep learning task. According to the current task decomposition scheme, the task represented by each computational graph node on the computational graph is decomposed into hardware computing instructions to obtain the instruction set of the computational graph; according to the data flow dependency between the nodes on the computational graph, the dependency graph between the instructions in the instruction set is obtained;

[0039] An instruction partitioning module is used to partition the hardware computing instructions in the instruction set into each sub-core in the processor;

[0040] An instruction sorting module is used to determine, based on the dependency graph, a directed acyclic graph containing the dependencies between the hardware computing instructions on the sub-cores of the processor, and determine, based on the processor's emission model and the reciprocal throughput of the hardware computing instructions, the order in which the hardware computing instructions allocated to the sub-cores are executed on the sub-cores, thereby obtaining the final instruction schedule for the deep learning task. The processor executes the instruction schedule to obtain the execution result of the deep learning task.

[0041] The instruction-level parallel scheduling device in the deep learning compiler, wherein the instruction partitioning module is used to:

[0042] Select the sub-core with the smallest current load in the processor as the target core;

[0043] The hardware computing instructions in the instruction set are scored based on whether they have data reuse and / or synchronization relationships with existing hardware computing instructions in the target core, and the hardware computing instructions with the highest scores are selected and allocated to the target core.

[0044] The instruction-level parallel scheduling device in the deep learning compiler, wherein the instruction sorting step includes:

[0045] Initial module, used to maintain a variable t for each computing unit u in the processor u ∈{t0...t U-1}, t u =0;

[0046] Loop module, used to find all t u The computing units with a value of 0 form the set {u0}. Find all currently executable instructions executed in {u0} from the directed acyclic graph DAG as candidate instructions; according to the reciprocal throughput T of the candidate instructions -1 , the functional unit u executed, the data used by the candidate instructions, score the candidate instructions, select the candidate instruction with the highest score and append it to the list, so that t u =t u +T -1 ; Set the remaining time t of the emission of all computing units u u Subtract the minimum remaining time of all current calculation units to get

[0047] The sequence determination module is used to execute the loop steps again until all the hardware computing instructions on the sub-cores are added to the list, and the hardware computing instructions in the list are the execution order.

[0048] The instruction-level parallel scheduling device in the deep learning compiler includes an instruction scheduling search module for:

[0049] Given the upper limit N of random walk steps and the random walk sampling volume R, a decomposition scheme is randomly selected from all enumerated task decomposition schemes as the current task decomposition scheme;

[0050] Mark the selected current task decomposition scheme as explored, and call the instruction partition module and the instruction sorting module to obtain the instruction schedule s;

[0051] Predict the execution time t of the processor to execute the instruction schedule s s ;

[0052] According to the similarity with the current task decomposition scheme, R unexplored decomposition schemes are sampled from all enumerated task decomposition schemes, and their instruction schedules {s1, ..., s R}, and the execution time predicted by the performance model

[0053] Select the instruction schedule with the shortest predicted execution time The decomposition scheme corresponding to s′ is used as the currently selected decomposition scheme;

[0054] If the current number of walking steps is less than the upper limit N, the current number of walking steps will be increased by 1, and R unexplored decomposition schemes will be sampled from all enumerated task decomposition schemes based on the similarity with the current task decomposition scheme. Otherwise, the instruction schedule corresponding to the currently selected decomposition scheme will be returned as the final instruction schedule.

[0055] The instruction-level parallel scheduling device in the deep learning compiler, wherein the instruction scheduling search module is used to:

[0056] Use the performance model to predict the execution time t of the processor to execute the instruction schedule s s ;

[0057] The performance model is used to predict the time consumed by a sub-core to execute a given instruction schedule. The processor includes multiple sub-cores, and the longest execution time is selected as the execution time of the processor to execute the instruction schedule.

[0058] The performance model consists of a sub-core instruction emission simulation, a computing unit simulation, and a data read and write simulation module; the instruction emission simulation module is used to receive the instruction sorting result, and emit the instruction corresponding to the instruction to the computing unit simulation module according to the processor's emission model; the computing simulation module executes the instruction according to the sub-core's time; the computing simulation module also sends the instruction's data read and write request to the data read and write simulation module, and the data read and write simulation module simulates the instruction's data reading and writing time consumption based on the data reuse between instructions, and transmits a blocking signal back to the instruction emission simulation module through the computing simulation module when a structural conflict occurs in the data read and write request.

[0059] The present invention also proposes a storage medium for storing a computer program of the instruction-level parallel scheduling method in the deep learning compiler.

[0060] The present invention also proposes a processor with a deep learning compiler, including the instruction-level parallel scheduling device.

[0061] The present invention also provides an electronic device, which has the processor.

[0062] From the above scheme, it can be seen that the advantages of the present invention are:

[0063] The instruction-level parallel scheduling technology proposed in this invention enables the deep learning compiler to obtain an instruction scheduling solution that can efficiently achieve ILP optimization within the instruction scheduling space during the task scheduling phase within a limited compilation time. Combined with the instruction-level parallel scheduling technology proposed in this invention, the deep learning compiler can achieve ILP optimization for deep learning programs, that is, improve the computational throughput of deep learning programs and reduce synchronization overhead in the programs. Specifically:

[0064] (1) On consumer-grade GPUs, at full fp16 (IEEE 16-bit floating-point format) precision, the computational throughput is improved by up to 25% compared to existing designs; at mixed precision of tf32 (NVIDIA's 19-bit Tensorfloat-32 floating-point format) and fp32 (IEEE 32-bit floating-point format), the computational throughput is improved by up to 50% compared to existing designs.

[0065] (2) On data center-class GPUs, at full fp16 precision, the computational throughput is improved by up to 50% compared to existing designs; at tf32-fp32 mixed precision, the computational throughput is improved by up to 12.5% ​​compared to existing designs; at full fp32 precision, the computational throughput is improved by up to 100% compared to existing designs.

[0066] (3) On data center or workstation-class CPUs, at full bf16 (Google’s proposed 16-bit floating point format) precision, the computational throughput is improved by up to 6% compared to existing designs.

[0067] (4) By dividing the hardware computing instructions that need to be synchronized to the same processor sub-core for execution, the compiler system proposed in the present invention can reduce the synchronization overhead in deep learning programs. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a schematic diagram of the hierarchical structure of a general intelligent processor;

[0069] Figure 2 A schematic diagram of the compilation process of Rammer in the prior art;

[0070] Figure 3 is a system block diagram of the performance model of the present invention;

[0071] Figure 4 Flow chart of the method of the present invention;

[0072] Figure 5 This is a structural diagram of the device of the present invention;

[0073] Figure 6 This is a schematic diagram of the processor structure of the present invention;

[0074] Figure 7 It is a schematic diagram of the structure of the electronic device of the present invention.

[0075] Reference numerals:

[0076] 1-General intelligent processor;

[0077] Level 11-L2;

[0078] 12-Shared memory;

[0079] 13-L2 cache;

[0080] 14-L1 level;

[0081] 15-sub-core local cache;

[0082] 16-level L0 cache;

[0083] 17-Intelligent computing unit;

[0084] 18-L0 level;

[0085] 19-General computing unit;

[0086] 21-Task decomposition steps;

[0087] 22-Task scheduling steps;

[0088] 23-Code generation step;

[0089] 3- Performance model;

[0090] 31-sorting results;

[0091] 32-sub-core instruction issuance simulation;

[0092] 33-sub-core computing unit simulation;

[0093] 34-sub-core data reading and writing simulation module;

[0094] 35-Data obtained from micro-test;

[0095] 41-Task decomposition steps;

[0096] 42-Instruction division steps;

[0097] 43-Instruction sorting step;

[0098] 51-Task decomposition module;

[0099] 52-instruction partitioning module;

[0100] 53-instruction sorting module;

[0101] 6-Processor;

[0102] 7-Electronic equipment. DETAILED DESCRIPTION

[0103] It should be noted that, in a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0104] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0105] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0106] The above embodiments can be implemented in whole or in part through software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired method (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0107] It should also be understood that the term "and / or" in this document simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " in this document generally indicates an "or" relationship between the related objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0108] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0109] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0110] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0111] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0112] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0113] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0114] When conducting micro-testing of general-purpose intelligent processors and related research on deep learning compilers for them, the inventors found that the hardware implementation of existing general-purpose intelligent processors brings many opportunities to utilize the various computing units contained therein in parallel in the form of ILP, thereby improving the computational throughput of a single operator in a deep learning task, and reducing the synchronization overhead between multiple operators, thereby achieving ILP optimization of deep learning programs. However, the inventors noticed that existing deep learning compilers are unable to achieve this ILP optimization, resulting in a waste of hardware resources and missing the opportunity to further improve the execution performance of deep learning tasks. More specifically, in task scheduling, after the compiler decomposes the task into subtasks, it needs to determine the mapping relationship between the subtasks and the processor sub-cores, and may need to determine the execution order of the subtasks on each sub-core. Generally, the above-mentioned task scheduling process needs to be implemented through a specific scheduling algorithm, and the scheduling algorithms designed in existing designs generally adopt a simple model that estimates program performance by summing the hardware instruction delays. This performance model can have a high accuracy rate on dedicated intelligent processors that execute intelligent computing unit instructions serially. However, for general-purpose intelligent processors, it cannot provide accurate performance estimates in scenarios where ILP exists in program memory. As a result, the compiler cannot provide a task scheduling solution and its corresponding computing instruction sequence that can achieve ILP optimization for deep learning tasks.

[0115] During the task decomposition phase, the compiler of the present invention decomposes a given task into subtasks that have a direct correspondence with the hardware computing instructions and can be executed in parallel in the form of ILP. Therefore, the following content directly refers to the computational graph representation of the deep learning task as having been decomposed into a set of hardware computing instructions. During the task scheduling phase, the compiler divides the hardware computing instructions onto the sub-cores of the target processor. This process is called instruction partitioning. Subsequently, for the set of hardware computing instructions divided onto a certain sub-core, the compiler needs to find the execution order of these hardware computing instructions so that the hardware computing instructions corresponding to these hardware computing instructions can be efficiently executed in parallel in the form of ILP. The present invention refers to the above process of determining the execution order of hardware computing instructions as instruction sorting.

[0116] In addition, for a given set of hardware computing instructions and a given deep learning task, the compiler can enumerate multiple task decomposition schemes. For the above-mentioned task decomposition-instruction partitioning-instruction sorting process, the three together construct a search space as an instruction scheduling space, and different task decomposition-instruction partitioning-instruction sorting combinations, and instruction scheduling will also have different performance on the target processor. In order to find high-performance scheduling within a limited compilation time, in contrast to existing work, the present invention introduces a model for predicting the performance of a certain instruction scheduling under the influence of instruction-level parallelism (ILP) for a general-purpose intelligent processor, and uses the performance model to search for high-performance instruction scheduling. Specifically, the performance model gives a performance estimate of instruction scheduling based on the instruction emission model of the target general-purpose intelligent processor sub-core, the instruction delay obtained based on micro-testing, and the data reuse between instructions.

[0117] To illustrate the above-mentioned features and effects of the present invention more clearly and easily, the following embodiments are specifically described below with reference to the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are for illustrative purposes only. The scope of protection of the present invention is not limited to the disclosed embodiments; the present invention is defined by the appended claims.

[0118] For a given deep learning task represented by a computational graph, such as image classification or semantic segmentation, each deep learning task has a corresponding graph representation. For example, convolutional network models such as Alexnet and Resnet, or multi-layer perceptrons based on matrix multiplication, or a transformer model also based on matrix multiplication. The compiler decomposes the task represented by each computational graph node on the computational graph into a set of hardware computation instructions through task decomposition. A computational graph node can represent a single operator such as matrix multiplication (gemm) or convolution (conv), or an activation operator such as relu, or multiple complex operations such as a conv+relu fusion operator or a reducer. In short, a computational graph node represents a task that can be described using multi-layer loop statements.

[0119] By inheriting the dataflow dependencies between computational graph nodes, we can determine the dependencies between instructions. In the subsequent compilation process, the compiler needs to follow these dependencies to complete the instruction partitioning, instruction sorting, and instruction scheduling search processes, resulting in an instruction scheduling solution that can efficiently achieve ILP optimization.

[0120] Instruction division steps:

[0121] For a given task decomposition scheme, this paper proposes a task scheduling process to divide the hardware computing instructions obtained from the task decomposition into various processor sub-cores. More specifically, the compiler relies on the following algorithm to complete this process:

[0122] (1) Select a sub-core with the smallest current load. In this algorithm, the load on the sub-core is represented by the sum of the reciprocal throughputs of all hardware computing instructions assigned to the sub-core.

[0123] (2) Select a hardware computing instruction with the highest score from the union of the instruction sets obtained from the node decomposition of the entire computation graph and schedule it to the selected sub-core.

[0124] Specifically, the algorithm scores each candidate instruction based on the following: (a) the data reuse relationship between the candidate instruction and the existing instructions on the sub-core, that is, whether the execution of the two utilizes the same data, and the amount of the same data. The algorithm gives instructions with a larger amount of data reuse a higher score, and the specific scoring method can be diverse. In some implementations, the compiler uses the number of data bytes that can be reused as a scoring indicator. (b) the synchronization relationship between the candidate instruction and the existing instructions on the sub-core, that is, whether the two have a common successor on the instruction dependency graph. Similar to the amount of data reuse, the specific scoring method can be diverse. In some implementations, the compiler uses the number of common successors between the candidate instruction and the existing instructions on the sub-core on the instruction dependency graph as an indicator.

[0125] (3) Assign the highest-scoring instruction to the current sub-core. If the instruction is synchronized with the instruction blocks assigned to other sub-cores, then an additional record is made that synchronization between the current sub-core and the other sub-cores needs to be established after the corresponding instruction is executed.

[0126] The algorithm loops until all instructions are partitioned.

[0127] Instruction sorting steps:

[0128] After the instruction division is completed, each processor sub-core is assigned some hardware computing instructions, and during the instruction division process, the compiler also records the synchronization information between instructions. Furthermore, by inheriting the data flow dependency on the original calculation graph, the dependency between instructions on the sub-core can be determined to form a directed acyclic graph (DAG) between instructions. During the instruction sorting process, the compiler needs to combine the emission model of the target processor and the reciprocal throughput of the instructions to determine an order in which each instruction is efficiently executed on the sub-core. On the one hand, in order to ensure the efficiency of the instruction execution order as much as possible, the sorting algorithm should try to enable instructions to be emitted without blocking, and the sorting should maximize data reuse between adjacent instructions; on the other hand, in order to ensure the dependency between instructions, the execution order must be the topological sorting of the above-mentioned DAG. In particular, for a sequential single-issue processor, the compiler in the present invention completes instruction sorting according to the following algorithm:

[0129] Assume that there are U computing units in the sub-core. For example, there is an intelligent computing unit T and a general computing unit V in the sub-core, then U = 2. The algorithm maintains a variable t for each computing unit u. u ∈{t0...t U-1}, is called the remaining time of the unit’s launch, and the algorithm maintains an instruction list. Initially, the algorithm makes all t u =0.

[0130] (1) Find all computing units whose remaining emission time is 0 and form a set {u0}, that is,

[0131] (2) Find all currently executable instructions in DAG that are executed in {u0}. Executable instructions are instructions that have no incoming edges on the DAG, that is, instructions whose dependencies have been satisfied. The algorithm obtains the reciprocal throughput T of the instructions from the information recorded by the compiler. -1 , the functional unit u executed, and the data used by the instruction, score the instruction so that T -1 The instruction with the larger value and data reuse relationship with the previous instruction in u has a higher score. After that, the instruction with the highest score is taken from the DAG and appended to the list so that t u =t u +T -1 .

[0132] (3) The remaining time t of the emission of all computing units u u Subtract the minimum remaining time of all current calculation units to get Back to (1)

[0133] The above algorithm is repeated until all instructions are added to the list, and the final list represents the execution order of the instructions.

[0134] Finally, the compiler will combine the DAG of the instructions on the sub-core and the synchronization relationship recorded during the instruction partitioning process, and insert the corresponding synchronization code into the instruction list to ensure the correctness of the program.

[0135] Instruction scheduling search steps:

[0136] As described above, after the instruction partitioning and instruction sorting steps, the instruction set resulting from task decomposition is partitioned by the compiler into the sub-cores, and the execution order of the instructions on each sub-core is also determined. Programs corresponding to instruction schedules composed of different task decomposition, instruction partitioning, and instruction sorting combinations will have different performance on different processors. Preferably, to reduce search time, the compiler of the present invention uses the target machine's performance model 3 to search for high-performance instruction schedules.

[0137] Specifically, the system block diagram of the performance model 3 used in the present invention is as follows: Figure 3 As shown, it is used to predict the time it takes for a single processor sub-core to execute a given instruction sequence. The execution time of the entire intelligent processor is represented by the longest sub-core execution time. This performance model 3 consists of a sub-core instruction issuance simulation 32, a computing unit simulation 33, and a data read and write simulation module 34. All three rely on data 35 collected from pre-microtesting of the target processor.

[0138] Performance Model 3 is equivalent to an instruction-level simulator. The execution of this performance model simulates and predicts the time taken for a given instruction sequence. Each time a new instruction sequence is given, Performance Model 3 initializes a timer variable to 0. All "times" simulated by Performance Model 3 will cause this timer to increment. Ultimately, the value of this timer is the estimated execution time. Specifically:

[0139] In the above modules, the instruction issuance simulation module receives the instruction sorting result 31 and, according to the processor's issuance model (e.g., sequential single issuance), issues the corresponding instruction to the computation unit simulation module 33. The issuance is suspended when the computation unit simulation module 33 issues a blocking signal. The computation simulation module 33 records the execution time of instructions in each computation unit based on microtest data and issues a blocking signal when a computation unit structural conflict occurs. The computation simulation module 33 also sends data read and write requests for the instructions to the data read and write simulation module 34. This module simulates the data read and write time consumption of the instructions based on the microtest results 35 and the data reuse between instructions. If a structural conflict occurs in the data read and write requests, the blocking signal is transmitted back to the instruction issuance simulation module 32 via the computation simulation module. In particular, since the location of the data required for the instruction in the processor's memory hierarchy has not yet been determined during the instruction scheduling search phase, the data read and write simulation module 34 assumes that all the data required for the instruction is stored in the processor's lowest-level memory, such as registers.

[0140] Among them, the structural conflict of the computing unit mainly occurs on a sequential single-issue machine where the issuance rate and execution rate of instructions do not match. For example, for a sequential single-issue machine, and an instruction that requires 8 cycles to collect operands (that is, in the absence of data read conflicts, the fastest one is issued every 8 cycles), but execution requires 16 cycles. Consider the consecutive execution of two such instructions. At this time, it is possible that the first instruction is still occupying the computing unit and has not yet completed execution, while the second instruction has already been issued to the instruction queue of the computing unit. At this time, a computing unit structural conflict occurs, and the second instruction must wait for the first instruction to be executed before it can start execution. Such a situation will cause additional time consumption.

[0141] The performance model 3 proposed in the present invention can accurately predict the performance of a certain instruction schedule under the influence of instruction-level parallelism (ILP) for a general-purpose intelligent processor. Therefore, the compiler can explore the instruction scheduling space and find high-performance instruction scheduling by evaluating the performance of the instruction scheduling corresponding to the decomposition scheme enumerated in the task decomposition phase. In particular, the specific exploration strategy can be diverse. In the implementation of some compilers, the compiler can use a random walk strategy to explore the scheduling space according to the following process:

[0142] (1) Given the upper limit N of random walk steps and the random walk sampling volume R, a decomposition scheme is randomly selected from the enumerated task decomposition schemes as the starting point.

[0143] (2) Mark the selected decomposition scheme as explored, perform instruction partitioning and instruction sorting on it, and obtain an instruction schedule s.

[0144] (3) Use the performance model to predict the execution time t of ss .

[0145] (4) Sample R new decomposition schemes that are similar to the currently selected decomposition scheme and have not been explored from the enumerated decomposition schemes, and obtain their instruction schedules {s1, ..., s R}, and the execution time predicted by the performance model The similarity of the decomposition schemes can be determined by calculating the number of common instructions in the two decomposition schemes.

[0146] (5) Select the instruction schedule with the shortest predicted execution time and better performance The decomposition scheme corresponding to s′ is taken as the currently selected decomposition scheme.

[0147] (6) If the current number of walking steps is less than the upper limit N, the current number of walking steps should be increased by 1 and return to (4). Otherwise, the instruction schedule corresponding to the currently selected decomposition scheme is returned as the final instruction schedule.

[0148] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiment.

[0149] like Figure 5 As shown, the present invention also proposes an instruction-level parallel scheduling device in a deep learning compiler, which includes:

[0150] The task decomposition module 51 is used to decompose the computation graph of the deep learning task, decompose the task represented by each computation graph node on the computation graph into hardware computing instructions according to the current task decomposition scheme, and obtain the instruction set of the computation graph; based on the data flow dependency relationship between the nodes on the computation graph, obtain the dependency relationship diagram between the instructions in the instruction set;

[0151] An instruction partitioning module 52 is used to partition the hardware computing instructions in the instruction set into each sub-core in the processor;

[0152] The instruction sorting module 53 is used to determine a directed acyclic graph containing the dependencies between the hardware computing instructions on the sub-cores of the processor based on the dependency graph, and determine the execution order of the hardware computing instructions allocated to the sub-cores on the sub-cores based on the processor's emission model and the reciprocal throughput of the hardware computing instructions, to obtain the final instruction schedule of the deep learning task. The processor executes the instruction schedule to obtain the execution result of the deep learning task.

[0153] The instruction-level parallel scheduling device in the deep learning compiler, wherein the instruction partitioning module is used to:

[0154] Select the sub-core with the smallest current load in the processor as the target core;

[0155] The hardware computing instructions in the instruction set are scored based on whether they have data reuse and / or synchronization relationships with existing hardware computing instructions in the target core, and the hardware computing instructions with the highest scores are selected and allocated to the target core.

[0156] The instruction-level parallel scheduling device in the deep learning compiler, wherein the instruction sorting step includes:

[0157] Initial module, used to maintain a variable t for each computing unit u in the processor u ∈{t0...t U-1}, t u =0;

[0158] Loop module, used to find all t u The computing units with a value of 0 form the set {u0}. Find all currently executable instructions executed in {u0} from the directed acyclic graph DAG as candidate instructions; according to the reciprocal throughput T of the candidate instructions -1 , the functional unit u executed, the data used by the candidate instructions, score the candidate instructions, select the candidate instruction with the highest score and append it to the list, so that t u =t u +T -1 ; Set the remaining time t of the emission of all computing units u u Subtract the minimum remaining time of all current calculation units to get

[0159] The sequence determination module is used to execute the loop steps again until all the hardware computing instructions on the sub-cores are added to the list, and the hardware computing instructions in the list are the execution order.

[0160] The instruction-level parallel scheduling device in the deep learning compiler includes an instruction scheduling search module for:

[0161] Given the upper limit N of random walk steps and the random walk sampling volume R, a decomposition scheme is randomly selected from all enumerated task decomposition schemes as the current task decomposition scheme;

[0162] Mark the selected current task decomposition scheme as explored, and call the instruction partition module and the instruction sorting module to obtain the instruction schedule s;

[0163] Predict the execution time t of the processor to execute the instruction schedule s s ;

[0164] According to the similarity with the current task decomposition scheme, R unexplored decomposition schemes are sampled from all enumerated task decomposition schemes, and their instruction schedules {s1, ..., s R}, and the execution time predicted by the performance model

[0165] Select the instruction schedule with the shortest predicted execution time The decomposition scheme corresponding to s′ is used as the currently selected decomposition scheme;

[0166] If the current number of walking steps is less than the upper limit N, the current number of walking steps will be increased by 1, and R unexplored decomposition schemes will be sampled from all enumerated task decomposition schemes based on the similarity with the current task decomposition scheme. Otherwise, the instruction schedule corresponding to the currently selected decomposition scheme will be returned as the final instruction schedule.

[0167] The instruction-level parallel scheduling device in the deep learning compiler, wherein the instruction scheduling search module is used to:

[0168] Use the performance model to predict the execution time t of the processor to execute the instruction schedule s s ;

[0169] The performance model is used to predict the time consumed by a sub-core to execute a given instruction schedule. The processor includes multiple sub-cores, and the longest execution time is selected as the execution time of the processor to execute the instruction schedule.

[0170] The performance model consists of a sub-core instruction emission simulation, a computing unit simulation, and a data read and write simulation module; the instruction emission simulation module is used to receive the instruction sorting result, and emit the instruction corresponding to the instruction to the computing unit simulation module according to the processor's emission model; the computing simulation module executes the instruction according to the sub-core's time; the computing simulation module also sends the instruction's data read and write request to the data read and write simulation module, and the data read and write simulation module simulates the instruction's data reading and writing time consumption based on the data reuse between instructions, and transmits a blocking signal back to the instruction emission simulation module through the computing simulation module when a structural conflict occurs in the data read and write request.

[0171] The present invention also proposes a storage medium for storing a computer program of the instruction-level parallel scheduling method in the deep learning compiler. It should be understood that the storage medium in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0172] like Figure 6As shown, the present invention also proposes a processor 6 with a deep learning compiler, including the instruction-level parallel scheduling device. The instruction-level parallel scheduling device may be located in the compiler of the processor. The processor described in the present invention is the control center of the electronic device, which may be a processor or a general term for multiple processing elements. For example, it may be one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs). Optionally, the processor can perform various functions of the electronic device by running or executing software programs stored in the memory, and calling data stored in the memory.

[0173] like Figure 7 As shown, the present invention further provides an electronic device 7 having the processor 6 therein. The electronic device 7 can be a data transmitting device, a data processing device, a data receiving device, a data display device, etc. Specifically, the electronic device 7 can include: a server, a desktop computer, a laptop computer, a smartphone, a tablet computer, an embedded computer, etc., wherein the embedded computer includes a vehicle such as a smart car. The use of a high-performance processor improves the data processing efficiency of the electronic device 7.

[0174] In summary, the present invention utilizes instruction partitioning technology to determine the partitioning scheme for hardware computing instructions across processor sub-cores based on the set of hardware computing instructions derived from task decomposition. Combined with the target processor model, this technology determines an instruction partitioning scheme that balances the load across sub-cores, maximizes data reuse between instructions, and reduces synchronization costs between sub-cores, given a given task decomposition scheme.

[0175] This invention uses instruction sequencing technology to determine the execution order of instructions allocated to a sub-core after instruction partitioning. This technology combines the target processor's issue model and the reciprocal throughput of instructions to determine an execution order that maximizes non-blocking and efficient ILP parallel execution of instructions on the sub-core.

[0176] The present invention uses a scheduling search technology, that is, a technology for searching for high-performance instruction scheduling in the instruction scheduling space, and finds a high-performance instruction scheduling within a limited compilation time based on a performance model that takes into account the ILP characteristics of the processor.

[0177] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the description and implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. An instruction-level parallel scheduling method in a deep learning compiler, characterized in that: include: The task decomposition step decomposes the computational graph of the deep learning task and decomposes the task represented by each computational graph node on the computational graph into hardware computing instructions according to the current task decomposition scheme to obtain the instruction set of the computational graph; According to the data flow dependency relationship between the nodes on the computation graph, a dependency graph between the instructions in the instruction set is obtained; Instruction partitioning step, the hardware computing instructions in the instruction set are divided into each sub-core in the processor; In the instruction sorting step, based on the dependency graph, a directed acyclic graph containing the dependency relationships between the hardware computing instructions on the sub-cores of the processor is determined, and based on the processor's emission model and the reciprocal throughput of the hardware computing instructions, the execution order of the hardware computing instructions allocated to the sub-cores on the sub-cores is determined to obtain the final instruction schedule of the deep learning task. The processor executes the instruction schedule to obtain the execution result of the deep learning task.

2. The instruction-level parallel scheduling method in a deep learning compiler according to claim 1, wherein: The instruction division steps include: Select the sub-core with the smallest current load in the processor as the target core; The hardware computing instructions in the instruction set are scored based on whether they have data reuse and / or synchronization relationships with existing hardware computing instructions in the target core, and the hardware computing instructions with the highest scores are selected and allocated to the target core.

3. The instruction-level parallel scheduling method in a deep learning compiler according to claim 1 or 2, wherein: The instruction sorting steps include: Initially, a variable t is maintained for each computational unit u in the processor. u ∈{t0…t U-1 }, t u =0; Loop through the steps to find all t u The computing units with a value of 0 form the set {u0}. Find all currently executable instructions executed in {u0} from the directed acyclic graph DAG as candidate instructions; according to the reciprocal throughput T of the candidate instructions -1 , the functional unit u executed, the data used by the candidate instructions, score the candidate instructions, select the candidate instruction with the highest score and append it to the list, so that t u =t u +T -1 ; Set the remaining time t of the emission of all computing units u u Subtract the minimum remaining time of all current calculation units to get The sequential determination step is executed again in the loop until all hardware computing instructions on the sub-core are added to the list, and the hardware computing instructions in the list are in the execution order.

4. The instruction-level parallel scheduling method in a deep learning compiler according to claim 1, wherein: Including instruction scheduling search steps: Step 1: Given the upper limit N of random walk steps and the random walk sampling volume R, randomly select a decomposition scheme from all enumerated task decomposition schemes as the current task decomposition scheme; Step 2: Mark the selected current task decomposition scheme as explored, and perform the instruction partitioning step and the instruction sorting step on it to obtain the instruction schedule s; Step 3: Predict the execution time t of the processor to execute the instruction schedule s s ; Step 4: Based on the similarity with the current task decomposition scheme, sample R unexplored decomposition schemes from all enumerated task decomposition schemes and obtain their instruction schedules {s1,…,s R }, and the execution time predicted by the performance model Step 5: Select the instruction schedule with the shortest predicted execution time The decomposition scheme corresponding to s' is used as the currently selected decomposition scheme. Step 6: If the current number of walking steps is less than the upper limit N, the current number of walking steps is increased by 1, and step 4 is executed again. Otherwise, the instruction schedule corresponding to the currently selected decomposition scheme is returned as the final instruction schedule.

5. The instruction-level parallel scheduling method in a deep learning compiler according to claim 4, wherein: Use the performance model to predict the execution time t of the processor to execute the instruction schedule s s ; The performance model is used to predict the time consumed by a sub-core to execute a given instruction schedule. The processor includes multiple sub-cores, and the longest execution time is selected as the execution time of the processor to execute the instruction schedule. The performance model consists of a sub-core instruction emission simulation, a computing unit simulation, and a data read and write simulation module; the instruction emission simulation module is used to receive the instruction sorting result and transmit the corresponding instruction to the computing unit simulation module according to the processor's emission model; the computing simulation module is based on the time it takes for the sub-core to execute the instruction; The computing simulation module simultaneously sends the data read and write request of the instruction to the data read and write simulation module. The data read and write simulation module simulates the data reading and writing time consumption of the instruction based on the data multiplexing between instructions, and transmits the blocking signal back to the instruction transmission simulation module through the computing simulation module when a structural conflict occurs in the data read and write request.

6. An instruction-level parallel scheduling device in a deep learning compiler, characterized in that: include: The task decomposition module is used to decompose the computational graph of the deep learning task and decompose the task represented by each computational graph node on the computational graph into hardware computing instructions according to the current task decomposition scheme to obtain the instruction set of the computational graph; According to the data flow dependency relationship between the nodes on the computation graph, a dependency graph between the instructions in the instruction set is obtained; An instruction partitioning module is used to partition the hardware computing instructions in the instruction set into each sub-core in the processor; An instruction sorting module is used to determine, based on the dependency graph, a directed acyclic graph containing the dependencies between the hardware computing instructions on the sub-cores of the processor, and determine, based on the processor's emission model and the reciprocal throughput of the hardware computing instructions, the order in which the hardware computing instructions allocated to the sub-cores are executed on the sub-cores, thereby obtaining the final instruction schedule for the deep learning task. The processor executes the instruction schedule to obtain the execution result of the deep learning task.

7. The instruction-level parallel scheduling device in the deep learning compiler according to claim 6, characterized in that: This instruction is divided into modules for: Select the sub-core with the smallest current load in the processor as the target core; The hardware computing instructions in the instruction set are scored based on whether they have data reuse and / or synchronization relationships with existing hardware computing instructions in the target core, and the hardware computing instructions with the highest scores are selected and allocated to the target core.

8. The instruction-level parallel scheduling device in a deep learning compiler according to claim 6 or 7, characterized in that: The instruction sorting steps include: Initial module, used to maintain a variable t for each computing unit u in the processor u ∈{t0…t U-1 }, t u =0; Loop module, used to find all t u The computing units with a value of 0 form the set {u0}. Find all currently executable instructions executed in {u0} from the directed acyclic graph DAG as candidate instructions; according to the reciprocal throughput T of the candidate instructions -1 , the functional unit u executed, the data used by the candidate instructions, score the candidate instructions, select the candidate instruction with the highest score and append it to the list, so that t u =t u +T -1 ; Set the remaining time t of the emission of all computing units u u Subtract the minimum remaining time of all current calculation units to get The sequence determination module is used to execute the loop steps again until all the hardware computing instructions on the sub-cores are added to the list, and the hardware computing instructions in the list are the execution order.

9. The instruction-level parallel scheduling device in a deep learning compiler according to claim 6, wherein: Includes instruction scheduling search module for: Given the upper limit N of random walk steps and the random walk sampling volume R, a decomposition scheme is randomly selected from all enumerated task decomposition schemes as the current task decomposition scheme; Mark the selected current task decomposition scheme as explored, and call the instruction partition module and the instruction sorting module to obtain the instruction schedule s; Predict the execution time t of the processor to execute the instruction schedule s s ; According to the similarity with the current task decomposition scheme, R unexplored decomposition schemes are sampled from all enumerated task decomposition schemes, and their instruction schedules {s1,…,s R }, and the execution time predicted by the performance model Select the instruction schedule with the shortest predicted execution time The decomposition scheme corresponding to s' is used as the currently selected decomposition scheme; If the current number of walking steps is less than the upper limit N, the current number of walking steps will be increased by 1, and R unexplored decomposition schemes will be sampled from all enumerated task decomposition schemes based on the similarity with the current task decomposition scheme. Otherwise, the instruction schedule corresponding to the currently selected decomposition scheme will be returned as the final instruction schedule.

10. The instruction-level parallel scheduling device in the deep learning compiler according to claim 9, characterized in that: This instruction schedules the search module to: Use the performance model to predict the execution time t of the processor to execute the instruction schedule s s ; The performance model is used to predict the time consumed by a sub-core to execute a given instruction schedule. The processor includes multiple sub-cores, and the longest execution time is selected as the execution time of the processor to execute the instruction schedule. The performance model consists of a sub-core instruction emission simulation, a computing unit simulation, and a data read and write simulation module; the instruction emission simulation module is used to receive the instruction sorting result and transmit the corresponding instruction to the computing unit simulation module according to the processor's emission model; the computing simulation module is based on the time it takes for the sub-core to execute the instruction; The computing simulation module simultaneously sends the data read and write request of the instruction to the data read and write simulation module. The data read and write simulation module simulates the data reading and writing time consumption of the instruction based on the data multiplexing between instructions, and transmits the blocking signal back to the instruction transmission simulation module through the computing simulation module when a structural conflict occurs in the data read and write request.

11. A storage medium for storing a computer program for executing the instruction-level parallel scheduling method in the deep learning compiler according to any one of claims 1 to 5.

12. A processor with a deep learning compiler, characterized in that: The processor includes the instruction-level parallel scheduling device as described in any one of claims 6-10.

13. An electronic device, characterized in that: The electronic device has the processor according to claim 12 therein.

Images