Compilation and execution method of neural network algorithm task, chip and electronic equipment

Abstract

The application provides a method for compiling and executing a neural network algorithm task, a chip and an electronic device, comprising: dividing the neural network algorithm task into a plurality of task units according to time slices, so that when the plurality of neural network algorithm tasks are executed simultaneously, time division multiplexing dynamic scheduling is performed on the task units as a scheduling unit; compiling the task units to obtain a compiled instruction stream; and sending the compiled instruction stream to an NPU chip, so that the NPU chip controls the execution of the corresponding algorithm task based on the compiled instruction stream when scheduling and executing the scheduling unit. Thus, in a time window, a plurality of neural network algorithm tasks are interleaved and executed in the form of task units, ensuring that different algorithm task running processes do not affect each other, and flexible scheduling can meet the computing power and bandwidth requirements of different algorithm tasks.

Description

Technical Field

[0001] This application relates to the field of electronic circuit technology, and in particular to methods for compiling and executing neural network algorithm tasks, chips, and electronic devices. Background Technology

[0002] An NPU (Neural Network Processing Unit) simulates neurons and synaptic structures through circuitry. Each neuron is abstracted as an activation function, whose input is determined by the outputs of its connected neurons and the synapses linking them. To express specific knowledge, users typically need to adjust the synaptic values, network topology, and other parameters within the artificial neural network (using specific algorithms). This process is called "learning." After learning, the artificial neural network can use the acquired knowledge to solve specific problems.

[0003] Current NPU chips typically require the simultaneous execution of multiple neural network algorithms. This raises the issue of computing power allocation among these algorithms, and it's crucial to ensure that the execution processes of different algorithms do not interfere with each other. Furthermore, algorithms handling real-time tasks have bandwidth requirements, necessitating that the bandwidth of real-time tasks not be preempted by the bandwidth of non-real-time tasks. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method for compiling and executing neural network algorithm tasks, a chip, and an electronic device to solve the problems existing in the prior art.

[0005] This application provides a method for compiling a neural network algorithm task, comprising: dividing the neural network algorithm task into several task units according to time slices so that when multiple neural network algorithm tasks are executed simultaneously, time-division multiplexing dynamic scheduling is performed using the task units as a scheduling unit; compiling the task units to obtain a compilation instruction stream; and sending the compilation instruction stream to an NPU chip so that the NPU chip controls the execution of the corresponding algorithm task based on the compilation instruction stream when scheduling the execution of the scheduling unit. In this application, the neural network algorithm task is divided into several task units so that when multiple neural network algorithms run simultaneously, time-division multiplexing dynamic scheduling can be performed using task units as the scheduling unit, ensuring that the running processes of different algorithm tasks do not interfere with each other, while flexible scheduling can meet the computing power and bandwidth requirements of different algorithm tasks.

[0006] In one implementation of the first aspect, compiling the task unit to obtain a compilation instruction stream includes: dividing the task unit into several thread tasks according to the task content in each task unit, and dividing each thread task into several subtasks; compiling the task unit sequentially according to the execution order of the neural network algorithm task to obtain a first compilation instruction stream and a second compilation instruction stream, wherein the first compilation instruction stream includes the execution waiting relationship between the thread tasks in each task unit, and the second compilation instruction stream includes the configuration information of the subtasks in each thread task and the execution waiting relationship of the subtasks.

[0007] In one implementation of the first aspect, the thread task includes a computation thread task and / or an I / O thread task.

[0008] In one implementation of the first aspect, sending the compilation instruction stream to the NPU chip so that the NPU chip can schedule the execution of the scheduling unit to execute the algorithm task based on the compilation instruction stream includes: sending the first compilation instruction stream to the NPU chip so that the scheduling module can switch the hardware resources for executing the thread task based on the first compilation instruction stream when scheduling the execution of a certain task unit; and sending the second compilation instruction stream to the task execution controller of the NPU chip so that the task execution controller can control the hardware resources to execute the corresponding thread task based on the second compilation instruction stream.

[0009] A second aspect of this application provides a method for executing a neural network algorithm task, comprising: acquiring a compilation instruction stream of multiple neural network algorithm tasks, wherein the compilation instruction stream is compiled using the method described in the first aspect of this application; performing time-division multiplexing dynamic scheduling on the multiple neural network algorithm tasks using the task unit as a scheduling unit, so as to interleave the task units of the multiple neural network algorithm tasks within a time window; and controlling the execution of the algorithm tasks of the task units based on the compilation instruction stream. In this application, performing time-division multiplexing dynamic scheduling on multiple neural network algorithm tasks using the task unit as a scheduling unit ensures that the running processes of different algorithm tasks do not interfere with each other, while the flexible scheduling can meet the computing power and bandwidth requirements of different algorithm tasks.

[0010] In one implementation of the second aspect, time-division multiplexing dynamic scheduling of the multiple neural network algorithm tasks, with the task unit as a scheduling unit, includes: obtaining the time slice length, task priority switching rules, and task execution volume threshold within a time window; starting a time window and dividing the time window into several time slices according to the time slice length so that at most one task unit of the neural network algorithm task is executed in any time slice; obtaining the task execution consumption of each neural network algorithm task in the time window; dynamically allocating the neural network algorithm tasks to be run in each time slice of the time window based on the task execution consumption, task execution volume threshold, and task priority switching rules of each neural network algorithm task; and scheduling the corresponding task units based on the task execution order of the neural network algorithm tasks to be run allocated in the time slice to determine the task units to be executed in the time slice.

[0011] In one implementation of the second aspect, the task execution threshold includes a computing power threshold and / or a bandwidth threshold, and correspondingly, the task execution consumption includes computing power consumption and / or bandwidth consumption.

[0012] In one implementation of the second aspect, the neural network algorithm tasks to be run in each time slice of the time window are dynamically allocated based on the task execution consumption, task execution threshold, and task priority switching rules of each neural network algorithm task. This includes: when each time slice starts, determining whether the task execution consumption of each neural network algorithm task in the current time window has reached the task execution threshold, closing the permission of the neural network algorithm task whose task execution consumption has reached the task execution threshold in the current time window and waiting for the next time window to arrive before reopening it; and switching one of the remaining neural network algorithm tasks as the neural network algorithm task to be run in the current time slice based on the task priority switching rules.

[0013] In one implementation of the second aspect, the task priority switching rule includes a fixed priority switching rule or a dynamic priority switching rule.

[0014] A third aspect of this application provides a chip configured to simultaneously perform multiple neural network algorithm tasks using the method described in the second aspect of this application.

[0015] A fourth aspect of this application provides an electronic device comprising: a chip as described in a third aspect of this application.

[0016] As described above, this application provides a method for compiling and executing neural network algorithm tasks, a chip, and an electronic device. During the compilation of the neural network algorithm task, each algorithm task is divided into several task units according to time slices so that when multiple neural network algorithm tasks are executed simultaneously, the task units are used as a scheduling unit for time-division multiplexing dynamic scheduling. During the time-division multiplexing dynamic scheduling process, dynamic allocation is performed according to the computing power and bandwidth requirements of the neural network algorithm task so that multiple neural network algorithm tasks are interleaved and executed in the form of task units within a time window, ensuring that the running processes of different algorithm tasks do not affect each other. At the same time, flexible scheduling can meet the computing power and bandwidth requirements of different algorithm tasks. Attached Figure Description

[0017] Figure 1 A schematic diagram of the architecture of an NPU chip is shown in one example to illustrate the relevant technology.

[0018] Figure 2 An example illustrating the relevant technology is a schematic diagram of the NPU core architecture in an NPU chip.

[0019] Figure 3 A flowchart illustrating a compilation method for a neural network algorithm task in one embodiment of this application is shown.

[0020] Figure 4 This document presents a structural diagram of a task unit of a neural network algorithm task in one embodiment of this application.

[0021] Figure 5 A flowchart illustrating the execution method of a neural network algorithm task in one embodiment of this application is shown.

[0022] Figure 6 This document presents a schematic flowchart illustrating the execution method of a neural network algorithm task in one embodiment of this application.

[0023] Figure 7 This document presents a schematic diagram of the architecture of an NPU chip in one embodiment of this application.

[0024] Figure 8 A schematic diagram of the NPU chip architecture is shown in another embodiment of this application. Detailed Implementation

[0025] The following specific examples illustrate the implementation methods of this application. Those skilled in the art can easily understand other advantages and effects of this application from the information disclosed herein. This application can also be implemented or its modules applied through other different specific embodiments. Various details in this application can also be modified or changed according to different viewpoints and application modules without departing from the spirit of this application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0026] The embodiments of this application will now be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily implement the application. This application may be embodied in many different forms and is not limited to the embodiments described herein.

[0027] In this application, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics represented in connection with that embodiment or example, which are included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics represented may be combined in any suitable manner in any one or a group of embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate different embodiments or examples represented in this application, as well as features of different embodiments or examples.

[0028] Furthermore, the terms "first" and "second" are used for illustrative purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the representation of this application, "a set" means two or more, unless otherwise explicitly specified.

[0029] To clearly illustrate this application, devices unrelated to the description are omitted, and the same or similar constituent elements throughout the specification are given the same reference numerals.

[0030] Throughout this specification, when it is said that a device is "connected" to another device, this includes not only "direct connection" but also "indirect connection" by placing other components in between. Furthermore, when it is said that a device "comprises" a certain constituent element, unless otherwise stated otherwise, this does not exclude other constituent elements, but rather implies that other constituent elements may be included.

[0031] While the terms first, second, etc., are used in some examples herein to refer to various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, first interface and second interface, etc., are used. Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, step, operation, element, module, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, modules, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0032] The technical terms used herein are for reference only to specific embodiments and are not intended to limit the scope of this application. The singular form used herein includes the plural form unless the statement explicitly indicates otherwise. The word "comprising" as used in the specification means to specify a particular characteristic, region, integer, step, operation, element, and / or component, and does not exclude the presence or addition of other characteristics, regions, integers, steps, operations, elements, and / or components.

[0033] Although not explicitly defined, all terms, including technical and scientific terms used herein, shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. Terms defined in commonly used dictionaries shall be further interpreted as having a meaning consistent with the relevant technical literature and the message of the present invention, and shall not be over-interpreted as having an ideal or overly formulaic meaning unless otherwise defined.

[0034] With the development of artificial intelligence, neural network processing unit (NPU) chips are widely used. For example, with the gradual upgrading of intelligent driving technology, NPU chips in in-vehicle intelligent systems are used to process video data, image data, or combinations thereof, to achieve real-time image processing and artificial intelligence recognition through machine learning algorithms. Artificial intelligence recognition includes vehicle recognition, lane line recognition, and pedestrian recognition. Typically, NPU chips require multiple neural network algorithms to run simultaneously, such as image quality improvement algorithms and artificial intelligence recognition algorithms running concurrently, or two different artificial intelligence recognition algorithms running simultaneously. This raises the issue of computing power allocation for each algorithm and the need to ensure that the running processes of different algorithms do not interfere with each other. Furthermore, for real-time tasks such as those used for real-time image processing, there are bandwidth requirements, which necessitates that the bandwidth of real-time tasks not be preempted by the bandwidth of non-real-time tasks.

[0035] like Figure 1 The diagram shown illustrates the architecture of an NPU chip as an example of related technologies. Figure 1 The NPU chip 1 includes an on-chip memory 11 (OCM), multiple NPU cores 12, a read / write structure 13 for high-speed data exchange between the on-chip memory 11, the NPU cores 12, and external memory (DDR), and some additional logic circuits 14. "OCM" refers to on-chip memory, used for data interaction with external memory (DDR). The read / write structure 13 adopts a tensor processing architecture, i.e. Figure 1 In this context, "Fabric" is short for Tensor Processing Fabric, which serves as a pathway for high-speed data exchange.

[0036] like Figure 2 As shown, the NPU core 12 includes a computing module 121, a data read / write module (DMA) 122, a task execution controller 123, and additional control circuitry and buffer registers 124. The computing module 121 includes a convolution calculation submodule (conv) and a vector accelerator module (vector). "DMA" refers to Direct Memory Access, primarily used for data interaction with external memory (DDR), such as moving data from external memory to on-chip memory 11 and storing calculation results from on-chip memory 11 into external memory. The computing module 121 and the data read / write module (DMA) 122 constitute the hardware resources for executing algorithm tasks. The task execution controller 123 reads the instruction stream and controls the computing module 121 and the data read / write module 122 to execute algorithm tasks.

[0037] When multiple neural network algorithm tasks run simultaneously on a single NPU chip, related technologies employ, such as Figure 1 The diagram shows the architecture of an NPU chip with multiple NPU cores. Each NPU core's computing power is sufficient to run a specific neural network algorithm task. Figure 1 The demonstrated NPU chip architecture features three NPU cores 12, each capable of executing a separate neural network algorithm. Figure 1 The NPU chip described above can support the simultaneous execution of three neural network algorithms. However, this technology has the following drawbacks: 1. The additional control circuits and buffer registers inside multiple NPU cores are identical, and the control circuits of multiple NPU cores cannot be shared, resulting in wasted area; 2. The data read / write module (DMA) needs to be divided into multiple parts to ensure that multiple NPU cores do not interfere with each other, for example in... Figure 1 Under this architecture, if the OCM is 12MB, then the OCM needs to be divided into three parts and used by the three NPU cores respectively; 3. If the computing power inside the NPU cores is different (asymmetric design), the different NPU core structures will make it impossible for the NPU cores to be reused. If the computing power inside the NPU cores is completely equal, it will result in some NPU cores having insufficient computing power, while some NPU cores have too much computing power, and they cannot be dynamically allocated to each other, resulting in difficulties in use.

[0038] Based on this, embodiments of this application provide a method for compiling and executing neural network algorithm tasks, a chip, and an electronic device. During the compilation of neural network algorithm tasks, each algorithm task is divided into several task units according to time slices so that when multiple neural network algorithm tasks are executed simultaneously, the task units are used as a scheduling unit for time-division multiplexing dynamic scheduling. During the time-division multiplexing dynamic scheduling process, dynamic allocation is performed according to the computing power and bandwidth requirements of the neural network algorithm tasks so that multiple neural network algorithm tasks are interleaved and executed in the form of task units within a time window, ensuring that the running processes of different algorithm tasks do not affect each other, while flexible scheduling can meet the computing power and bandwidth requirements of different algorithm tasks.

[0039] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0040] like Figure 3 As shown, an embodiment of this application provides a method for compiling a neural network algorithm task. The method is applied to a compiler and includes steps S11 to S13.

[0041] Step S11: Divide the neural network algorithm task into several task units according to time slices so that when multiple neural network algorithm tasks are executed at the same time, the task unit is used as a scheduling unit for time-division multiplexing dynamic scheduling.

[0042] Step S12: Compile the task unit to obtain the compilation instruction stream;

[0043] Step S13: Send the compilation instruction stream to the NPU chip so that the NPU chip can control the execution of the corresponding algorithm task based on the compilation instruction stream when scheduling the scheduling unit.

[0044] In this embodiment, the neural network algorithm task is divided into several task units so that when multiple neural network algorithms run simultaneously, the task units can be used as the scheduling unit for time-division multiplexing and dynamic scheduling. This ensures that the running processes of different algorithm tasks do not affect each other, and the flexible scheduling can meet the computing power and bandwidth requirements of different algorithm tasks.

[0045] like Figure 4 The diagram shown illustrates the structure of a task unit in a neural network algorithm task according to an embodiment of this application. Each neural network can be divided into multiple units, such as... Figure 4 The computation process of each task unit shown can be described as follows: the data read / write module (DMA) reads data from external memory into on-chip memory (OCM), the computation module (vector / conv) waits for the data download to complete, the computation module (vector / conv) starts the computation and stores the result into on-chip memory (OCM), and the data read / write module (DMA) waits for the computation module (vector / conv) to complete the computation and writes the computation result from on-chip memory (OCM) to external memory.

[0046] For ease of description, the computation tasks of the computation module (vector / conv) are classified as COMPUTE threads, and the data read and write tasks of the data read and write module (DMA) are called IO threads (input / output threads).

[0047] Based on the above, compiling the task units to obtain compilation instruction streams includes: first, dividing the task units into several thread tasks according to the task content of each task unit, and each thread task into several subtasks, wherein the thread tasks include computation thread tasks and / or IO thread tasks; then, compiling the task units sequentially according to the execution order of the neural network algorithm tasks to obtain a first compilation instruction stream and a second compilation instruction stream, wherein the first compilation instruction stream includes the execution waiting relationships between the thread tasks in each task unit, and the second compilation instruction stream includes the configuration information of the subtasks in each thread task and the execution waiting relationships of the subtasks.

[0048] The compiler needs to divide each neural network algorithm task into time slices during compilation, ensuring that the duration of each task unit does not exceed 0.1ms. This allows for scheduling of multiple neural network algorithm tasks on a unit-by-unit basis, based on their computational power and bandwidth requirements, enabling interleaved execution. During compilation, the compiler separates the instruction streams for computational and I / O threads, generating corresponding first and second compilation instruction streams for each. These compilation instruction streams are then sent to the NPU chip, allowing the NPU chip to execute algorithm tasks based on these compiled instruction streams when scheduling the execution of the task.

[0049] Preferably, sending the compilation instruction stream to the NPU chip so that the NPU chip schedules the execution of the scheduling unit to execute the algorithm task based on the compilation instruction stream includes: sending the first compilation instruction stream to the NPU chip so that the scheduling module switches the hardware resources for executing the thread task based on the first compilation instruction stream when scheduling the execution of a certain task unit; and sending the second compilation instruction stream to the task execution controller of the NPU chip so that the task execution controller controls the hardware resources to execute the corresponding thread task based on the second compilation instruction stream.

[0050] In the above embodiments, the first compilation instruction stream includes the execution waiting relationship between the thread tasks in each of the task units, which can be exemplified as <COMPUTE.unit0 needs to wait for IO.unit0 to complete>, that is, the computing thread task of task unit unit0 can only be executed after the IO thread task of task unit unit0 is completed. Further, sending the first compilation instruction stream to the NPU chip, so that when the scheduling module schedules and executes a certain task unit, it switches the hardware resources for executing the thread task based on the first compilation instruction stream, which can be understood as that when executing a certain task unit, the scheduling module needs to switch the hardware resources for executing the thread task based on the first compilation instruction stream. When executing a computing thread task, it is necessary to switch to the hardware resource of the computing module (vector / conv) according to the first compilation instruction stream to execute the corresponding computing thread task. When executing an IO thread task, it is necessary to switch to the hardware resource of the data read-write module (DMA) according to the first compilation instruction stream to execute the IO thread task.

[0051] For each computing thread task and IO thread task, each further includes a plurality of subtasks, and the second compilation instruction stream includes configuration information of the subtasks in each thread task and the execution waiting relationship of the subtasks. For example, for the computing thread task of calculating r=a+b+c, it is divided into two subtasks, which are a first subtask and a second subtask respectively, where the first subtask is calculating x=a+b, and the second subtask is calculating r=x+c. Then the configuration information of each subtask included in the second compilation instruction stream is the configuration of the specific calculation process, and the execution waiting relationship of the subtasks included in the second compilation instruction stream is that the second subtask needs to wait for the first subtask to complete the calculation and obtain x before it can start calculation.

[0052] Based on the above, each neural network algorithm task is divided into several task units, each task unit is divided into a computing thread task and / or an IO thread task, the two threads are completely independent, and the only connection is that the execution waiting relationship between the thread tasks in the task unit given by the first compilation instruction stream needs to be perceived by the scheduling module of the NPU chip, while the first compilation instruction stream is generated offline, and the process of the neural network algorithm task has been compiled offline. Therefore, by sending the compilation instruction streams (the first compilation instruction stream and the second compilation instruction stream) to the NPU chip, and performing time-division multiplexing dynamic scheduling on the NPU chip, multiple neural network algorithm tasks can run simultaneously.

[0053] As Figure 5 shown, an execution method for a neural network algorithm task provided by an embodiment of the present application is presented, the method is applied to an NPU chip, and the method includes steps S21 to S23.

[0054] Step S21: Obtain the compilation instruction stream of multiple neural network algorithm tasks, wherein the compilation instruction stream is obtained by compiling using the compilation method in the above embodiment;

[0055] Step S22: Perform time-division multiplexing dynamic scheduling on the multiple neural network algorithm tasks using the task unit as a scheduling unit, so as to interleave the task units of the multiple neural network algorithm tasks in one time window;

[0056] Step S23: Execute the algorithm task of the task unit based on the compilation instruction flow control.

[0057] In this embodiment, multiple neural network algorithm tasks are dynamically scheduled using time-division multiplexing with the task unit as a scheduling unit. This ensures that the running processes of different algorithm tasks do not affect each other, while the flexible scheduling can meet the computing power and bandwidth requirements of different algorithm tasks.

[0058] Preferably, the time-division multiplexing dynamic scheduling of the multiple neural network algorithm tasks in step S22, using the task unit as a scheduling unit, includes: obtaining the time slice length, task priority switching rules, and task execution volume threshold within a time window; starting a time window and dividing the time window into several time slices according to the time slice length so that at most one task unit of the neural network algorithm task can be executed in any time slice; obtaining the task execution consumption of each neural network algorithm task in the time window; dynamically allocating the neural network algorithm tasks to be run in each time slice of the time window based on the task execution consumption, task execution volume threshold, and task priority switching rules of each neural network algorithm task; and scheduling the corresponding task units based on the task execution order of the neural network algorithm tasks to be run allocated in the time slice to determine the task units to be executed in the time slice.

[0059] The task execution threshold includes a computing power threshold and / or a bandwidth threshold, and correspondingly, the task execution consumption includes computing power consumption and / or bandwidth consumption. Based on the task execution consumption of each neural network algorithm task, the task execution threshold, and the task priority switching rule, the neural network algorithm tasks to be run in each time slice of the time window are dynamically allocated. This includes: when each time slice starts, determining whether the task execution consumption of each neural network algorithm task in the current time window has reached the task execution threshold; closing the permissions of neural network algorithm tasks whose task execution consumption has reached the task execution threshold in the current time window and waiting for the next time window to arrive before reopening them; and switching one neural network algorithm task from the remaining neural network algorithm tasks as the neural network algorithm task to be run in the current time slice based on the task priority switching rule. The task priority switching rule includes a fixed priority switching rule or a dynamic priority switching rule.

[0060] The following describes the time-division multiplexing dynamic scheduling strategy used in this embodiment for executing neural network algorithm tasks. Since different neural network algorithms require different computing power and bandwidth, this embodiment needs to consider the allocation of computing power and bandwidth for different neural network algorithms when performing time-division multiplexing dynamic scheduling.

[0061] The computing power allocation for time-division multiplexing is as follows:

[0062] Assuming a time slice length of t = 0.1 ms and a computation window T1 = 1 ms for the computation thread task, if the computational power allocation between the two neural network algorithm tasks is 3:7 and they do not affect each other, then within the time T1, a maximum of 3 time slices of computation will be allocated to neural network algorithm task 1, and 7 time slices will be allocated to neural network algorithm task 2.

[0063] The bandwidth allocation for time-division multiplexing follows the same principle as the computing power allocation for time-division multiplexing described above. The bandwidth allocation for time-division multiplexing is as follows:

[0064] Assuming the bandwidth allocation for the two neural network algorithm tasks is 4:6, and the system bandwidth is BW, then within the time window T2 of the IO thread task, neural network algorithm task 1 can read / write a maximum of 0.4*BW*T2 of data, and neural network algorithm task 2 can read / write a maximum of 0.6*BW*T2 of data.

[0065] Therefore, when executing neural network algorithm tasks within a time window, it is necessary to determine whether the task execution consumption of each neural network algorithm task in that time window has reached the task execution consumption threshold. The task execution consumption includes computing power consumption and bandwidth consumption. Once the task execution consumption of a neural network algorithm task in a time window exceeds the threshold, the privilege of that neural network algorithm task is closed, and the neural network algorithm task will no longer be executed in that time window. Instead, other neural network algorithm tasks are selected for execution according to the task priority switching rules in that time window until the next time window arrives. In the next time window, time-division multiplexing is performed. This allows multiple neural network algorithm tasks to be executed in an interleaved manner, and precise allocation of bandwidth and computing power can be achieved.

[0066] like Figure 6 The diagram illustrates a detailed flowchart of an execution method for a neural network algorithm task according to an embodiment of this application. Figure 6 The diagram illustrates the scheduling and execution of two neural network algorithm tasks.

[0067] Step S31: Acquire data and start the NPU chip to execute. The acquired data includes the time slice length, task priority switching rules, and task execution thresholds (including computing power thresholds and bandwidth thresholds) within a time window.

[0068] Step S32: The scheduling module of the NPU chip reads the first compilation instruction stream.

[0069] Step S33: The scheduling module waits for the task, specifically the task unit.

[0070] Step S34: When the task unit of a certain neural network algorithm task meets the requirements, the scheduling module will continue to determine whether the neural network algorithm task has obtained the allocated computing power threshold and / or bandwidth threshold within the time window. If the threshold is reached, the permission of the neural network algorithm task will be terminated, and the task will wait for the end of the current time window before continuing in the new time window. If the threshold is not reached, step S35 will continue.

[0071] Step S35: Dynamically allocate the neural network algorithm tasks to be run in each time slice of the time window based on the task execution consumption, task execution threshold, and task priority switching rules of each neural network algorithm task.

[0072] Specifically, if two neural network algorithm tasks arrive simultaneously... Figure 6At step S35, the system will execute the task unit with the highest priority according to the preset priority, while the other task unit will be restricted and wait for the higher priority task unit to finish before continuing. If only one neural network algorithm task is being executed, then the task unit of that neural network algorithm task will be executed directly.

[0073] Step S36: Enter the task unit execution stage. In this stage, the data read / write module (DMA) is switched to execute the IO thread task in the task unit based on the first compilation instruction stream, and / or the computation module (vector / conv) is switched to execute the computation thread task in the task unit. During the execution of the IO thread task and / or the computation thread task, the algorithm task is specifically executed based on the second compilation instruction stream.

[0074] Step S37: After one task unit is completed, proceed to step S33 to wait for the next task unit.

[0075] In addition, the execution process also includes:

[0076] Step S38: Calculate the task execution consumption (computing power consumption and bandwidth consumption) of each neural network algorithm task within the time window and give the task priority switching rules. Then, in step S34, perform computing power and bandwidth judgment to decide whether to restrict one or more neural network algorithm tasks from being executed in the time window. At the same time, in step S35, switch the executing neural network algorithm tasks based on the task priority switching rules.

[0077] like Figure 7 The diagram illustrates the architecture of an NPU chip according to an embodiment of this application. The chip is configured to simultaneously execute multiple neural network algorithm tasks using a neural network algorithm task execution method provided in the above embodiments. Figure 7The NPU chip 2 includes an on-chip memory 21, an NPU core 22, a read / write structure 23 for high-speed data exchange, and some additional logic circuits 24. One NPU core 22 is provided, which includes a computing module 221, a data read / write module (DMA) 222, a task execution controller 223, and additional control circuits and buffer registers 224. The computing module 221 includes a convolution calculation submodule (conv) and a vector accelerator module (vector). "DMA" refers to Direct Memory Access, mainly used for data interaction with external memory (DDR), such as moving data from external memory to on-chip memory and storing calculation results from on-chip memory 21 to external memory. The computing module 221 and the data read / write module (DMA) 222 constitute the hardware resources for executing algorithm tasks. The task execution controller 223 is used to read the instruction stream and control the computing module 221 and the data read / write module 222 to execute algorithm tasks. In addition, a scheduling module 25 is configured in the chip. The scheduling module 25 only needs to have simple online scheduling capabilities. It can use a time-division multiplexing dynamic scheduling strategy to schedule tasks when executing neural network algorithm tasks. The specific scheduling strategy has been described in detail in the above embodiments and will not be repeated here.

[0078] Preferably, the scheduling module 25 is not limited to a customized hardware circuit, but can be any circuit that can perform the function. For example, the RISCV processor can be used as the entire scheduling module 25, or the CPU can be used to perform the scheduling directly, or a circuit can be customized to perform the scheduling function according to the requirements.

[0079] Figure 7 The chip shown in the illustrated embodiment can achieve time-division multiplexing of multiple neural network algorithm tasks within a single NPU core. In other embodiments, chips can also have multiple NPU cores, each implementing time-division multiplexing of multiple neural network algorithm tasks. For example... Figure 8 In one embodiment shown, the NPU chip 3 includes an on-chip memory 31, multiple NPU cores 32, a read / write structure 33 for high-speed data exchange, and some additional logic circuits 34. A scheduling module 35 is configured in the NPU chip 3. The scheduling module 35 enables time-division multiplexing of multiple neural network algorithm tasks in each NPU core 32, thereby ensuring that more neural network algorithm tasks can run simultaneously.

[0080] This application embodiment may also provide an electronic device, including the chip. In some embodiments, the electronic device includes, but is not limited to, a server, desktop computer, laptop computer, mobile terminal, smart wearable device, gateway device, vehicle controller, security equipment, IoT device, etc.

[0081] As described above, the embodiments of this application provide a method for compiling and executing neural network algorithm tasks, a chip, and an electronic device. During compilation, the neural network algorithm tasks are divided into task units for compilation. Combined with time-division multiplexing calls based on task units during execution, each neural network algorithm task is divided into time-slice granular task units, which are then interleaved for execution. This avoids the design of requiring a separate NPU core for each neural network algorithm task, thus avoiding wasted area of ​​control circuits and buffer registers within numerous cores. Furthermore, the time-division multiplexing strategy ensures that only one neural network algorithm task is executing at any given time. OCM (at least a portion of OCM) can be reused without isolation, preventing wasted area. Additionally, the allocation of computing power and bandwidth is flexible; neural network algorithm tasks requiring less computing power / bandwidth are allocated less, while those requiring more are allocated more, providing greater flexibility and adaptability to the different needs of various neural network algorithm tasks.

[0082] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A compilation method for a neural network algorithm task, characterized in that, include: The neural network algorithm task is divided into several task units according to time slices so that when multiple neural network algorithm tasks are executed at the same time, the task unit is used as a scheduling unit for time-division multiplexing dynamic scheduling. Compiling the task units to obtain compilation instruction streams includes: dividing the task units into several thread tasks based on the task content of each task unit, and dividing each thread task into several subtasks; sequentially compiling the task units according to the execution order of the neural network algorithm tasks to obtain a first compilation instruction stream and a second compilation instruction stream, wherein the first compilation instruction stream includes the execution waiting relationship between the thread tasks in each task unit, and the second compilation instruction stream includes the configuration information of the subtasks in each thread task and the execution waiting relationship of the subtasks; The compilation instruction stream is sent to the NPU chip so that when the NPU chip schedules the execution of the task unit, it controls the execution of the corresponding algorithm task based on the compilation instruction stream.

2. The method according to claim 1, characterized in that, The thread tasks include computation thread tasks and / or I / O thread tasks.

3. The method according to claim 1, characterized in that, Sending the compiled instruction stream to the NPU chip so that the NPU chip schedules the execution of the task unit to perform the algorithm task based on the compiled instruction stream includes: The first compilation instruction stream is sent to the NPU chip so that when the scheduling module in the NPU chip schedules the execution of a certain task unit, it switches the hardware resources for executing the thread task based on the first compilation instruction stream. The second compilation instruction stream is sent to the task execution controller of the NPU chip, so that the task execution controller controls the hardware resources to execute the corresponding thread task based on the second compilation instruction stream.

4. A method for executing a neural network algorithm task, characterized in that, include: Obtain a compilation instruction stream for multiple neural network algorithm tasks, wherein the compilation instruction stream is obtained by compiling using the method described in any one of claims 1 to 3; The task unit is used as a scheduling unit to perform time-division multiplexing dynamic scheduling of the multiple neural network algorithm tasks, so as to interleave the task units of the multiple neural network algorithm tasks within a time window. The algorithm task of the task unit is executed based on the compilation instruction flow control.

5. The method according to claim 4, characterized in that, The time-division multiplexing dynamic scheduling of the multiple neural network algorithm tasks, using the task unit as a scheduling unit, includes: Get the time slice length, task priority switching rules, and the task execution volume threshold within a time window; Start a time window and divide the time window into several time slices according to the time slice length so that at most one task unit of the neural network algorithm task is executed in any time slice; Obtain the task execution consumption of each of the neural network algorithm tasks within the time window; Based on the task execution consumption, task execution threshold, and task priority switching rules of each neural network algorithm task, the neural network algorithm tasks to be run in each time slice of the time window are dynamically allocated. The task units to be executed in the time slice are determined by scheduling the corresponding task units according to the task execution order of the neural network algorithm task to be run allocated in the time slice.

6. The method according to claim 5, characterized in that, The task execution threshold includes a computing power threshold and / or a bandwidth threshold. Correspondingly, the task execution consumption includes computing power consumption and / or bandwidth consumption.

7. The method according to claim 5, characterized in that, Based on the task execution cost, task execution threshold, and task priority switching rules of each neural network algorithm task, the neural network algorithm tasks to be run in each time slice of the time window are dynamically allocated, including: When each time slice starts, it is determined whether the task execution consumption of each neural network algorithm task in the current time window has reached the task execution consumption threshold. If the task execution consumption of the neural network algorithm task that has reached the task execution consumption threshold is closed in the current time window, it will be reopened when the next time window arrives. Based on the task priority switching rule, switch one of the remaining neural network algorithm tasks as the neural network algorithm task to be run in the current time slice.

8. The method according to claim 5, characterized in that, The task priority switching rules include fixed priority switching rules or dynamic priority switching rules.

9. A chip, characterized in that, The chip is configured to simultaneously execute multiple neural network algorithm tasks using the method described in any one of claims 4 to 8.

10. An electronic device, characterized in that, Includes the chip as described in claim 9.

Images