Processing method and processing system utilizing a multi-core neural network accelerator

Abstract

The invention is a processing method for carrying out a first processing task flow and at least one further processing task flow on a multi-core neural network accelerator having processing units (11, 12), a command scheduler, and a memory, wherein each processing task flow comprises commands (20, 21, 22, 23) and dependencies between at least some of the commands (20, 21, 22, 23). Each command (20, 21, 22, 23) is executed by one particular processing unit (11, 12). The scheduling of the commands (20, 21, 22, 23) is determined by a merged command schedule, maintaining the dependencies within the respective processing task flows and reducing idle times of the processing units (11, 12). The merged command schedule is determined by a command schedule merger unit based on the input command schedules and their timelines. The invention also relates to a processing system for carrying out the method.

Description

[0001] PROCESSING METHOD AND PROCESSING SYSTEM UTILIZING A MULTI-CORE NEURAL NETWORK ACCELERATOR

[0002] TECHNICAL FIELD

[0003] The subject matter disclosed herein relates to a processing method, a processing system, a computer program product, and a computer-readable medium for carrying out a first processing task flow and at least one further processing task flow on a multicore neural network accelerator. More particularly, the disclosed subject matter relates to systems and methods for reducing execution time and latency by enabling interleaved multitasking in multi-core neural network accelerators, e.g., in the field of image processing.

[0004] BACKGROUND ART

[0005] Neural networks can be used effectively to solve complex problems that are difficult or impossible to handle using traditional mathematical methods. However, running neural networks on a conventional computer is an extremely resource-intensive process, primarily due to structural differences between neural networks and conventional computers. Therefore, target hardware platforms, so-called neural network accelerator architectures have been developed on which neural networks can be run efficiently. Such hardware provides the same or even a higher computing performance at a lower power consumption than a traditional GPU.

[0006] Most neural network accelerator architectures have multiple processing units, in other words, processing cores. Multi-core neural network accelerator hardware solutions are generally known in the art and are used for executing neural network workloads, or in other words, processing task flows generally consisting of up to hundreds or thousands of basic commands distributed on the cores. Neural network accelerator architectures can have identical or specialized processing units; in the second case, for example, some units can be specialized for performing convolution, and other parts can carry out pooling and other functions. In the case of image processing, the multi-core neural network accelerator hardware executes a neural network on an input image. The execution of the neural network, on a given input image, at a given resolution, will determine a workload. This workload is divided into many commands with dependencies between them. It is not possible to perform the processing of the commands in any order, but the independent commands can be executed in parallel. As the accelerator has more than one core, the commands can be scheduled / distributed between the cores to a certain level.

[0007] US 2021 / 0303346 A1 discloses a system and a method for queuing commands in a deep learning processor. A queue processor cooperates with a scheduler (part of a software compiler), which generates a static schedule of a parallel process for execution by the multicore processor system, including the queue processor based on a directed acyclic graph representing the dependencies between tasks in the process. The scheduler can calculate a number of command streams that can be utilized for execution of the parallel process based on the topography of the directed acyclic graph, the resources of the multicore processor system, and a user's power consumption and performance (e.g., inference time, memory utilization) specifications.

[0008] US 2022 / 0207783 A1 discloses a real-time low latency computer vision / machine learning compute accelerator with a smart convolutional neural network scheduler. This known solution aims to address the problem of processing one continuous image stream with as small latency as possible, requires close integration of the camera / image provider driver / API and the processing hardware (GPU), and uses runtime optimization based on the arrival time of input image slices. Furthermore, this known solution is tailored for GPUs, where every compute unit is identical, so it does not matter which work item is executed on which compute unit.

[0009] In parallelizing the commands between several cores, a problem can easily arise, namely that the cores must wait for each other due to the dependencies of the commands. For instance, it is often the case in such processing that results of convolutions are to be processed via pooling operations, meaning that convolution operations are to be carried out on processing units of a first type, and pooling the results on processing units of a second type can be started only after the results of the convolutions are available. Such dependencies can also be present in the case of subsequent operations on the same type, e.g., subsequent convolution operations, between processing units of the same type. Therefore, due to synchronization dependencies between these commands, which can be considered as defining a computational graph, there may be idle times when a core cannot execute any command and has to wait for another core(s).

[0010] The known methods do not solve the problems of these idle times. Thus, there is a need for a solution that allows for improvement over existing methods and systems. There is a need for a processing method, a processing system, a computer program product, and a computer-readable medium, eliminating as much as possible the shortcomings of known techniques. There is a particular need for a solution that is based on compiling the command execution into automatic scheduling that reduces the idle times of the processing units in case more than one processing task flow is to be carried out. It's also preferable to solve the problem without modifying the existing hardware of neural network accelerator architectures.

[0011] DISCLOSURE OF INVENTION

[0012] It is an object of the invention to provide an efficient interleaved, practically simultaneous processing of two or more processing task flows by a multi-core neural network accelerator, thereby reducing idle times of the processing units and, in turn, reducing the total processing time. Another objective of the invention is to improve the utilization of the cores with respect to idle times. It is a further object of the invention to provide a solution that is based on compiling the command execution into a scheduling that reduces idle times of the processing units in case more than one processing task flow is to be carried out.

[0013] The above and other objects have been achieved by the processing method according to claim 1 , by the processing system according to claim 9, by the computer program product according to claim 10, and by the computer-readable medium according to claim 11 . Preferred embodiments are defined in the dependent claims. The invention reduces idle times of the processing units when multiple workloads are scheduled to run concurrently on a single processing hardware, i.e., on one neural network accelerator. The invention achieves better utilization of the processing units by interleaved compiling the neural network functions, more specifically, the corresponding processing task flows, into a merged command schedule. Therefore, the invention involves a compile-time optimization, i.e. an optimization performed during the compilation process, before the code is executed. This contrasts with runtime optimization, which occurs while the program is running. The runtime part of the solution is straightforward and is carried out in line with the compiled commands. The invention works on a hardware architecture in which more than one computing core (processing unit) is present in the accelerator, and the distribution of the commands is determined by the compiling step.

[0014] The basic idea of the invention is to schedule additional commands into the cores when idle points are reached in the execution. To do this, the idle periods need to be determined. If the hardware architecture has profiling capability, then it can be used for measuring the idle periods; otherwise, some kind of estimation needs to be calculated. By knowing the idle intervals commands of another, independent processing task flow can be scheduled into them, e.g., by starting the processing of another image or frame. The processed task flows are independent of each other in the sense that there are no dependencies between the processing commands belonging to different processing task flows. More generally, two or more processing workloads are merged into a single workload for the neural network accelerator.

[0015] BRIEF DESCRIPTION OF DRAWINGS

[0016] Characteristics, objectives, and advantages of embodiments of the subject matter will become apparent from the following description, which is given solely by way of illustration, is non-limiting, and is to be read with reference to the appended drawings in which

[0017] Fig. 1 schematically illustrates a general embodiment of a multi-core neural network accelerator;

[0018] Fig. 2 schematically illustrates an example of a command schedule compiled for execution; Fig. 3 schematically illustrates a scheduler and processing units of a multi-core neural network accelerator, as well as a time diagram of command processing of the command schedule of Fig. 2;

[0019] Fig. 4 schematically illustrates the processing units of Fig. 3 and a time diagram of command processing of the command schedule of Fig. 2 on serial execution on two frames / images;

[0020] Fig. 5 schematically illustrates the processing units of Fig. 3 and a time diagram of command processing of the command schedule of Fig. 2 on interleaved execution on two frames / images;

[0021] Fig. 6 schematically illustrates the merged command schedule compiled for execution according to Fig. 5;

[0022] Fig. 7 schematically illustrates an exemplary multi-stage compilation process;

[0023] Fig. 8 schematically illustrates another example of serial processing of two frames by a multi-core neural network accelerator;

[0024] Fig. 9 schematically illustrates another example of an interleaved processing of two frames by the multi-core neural network accelerator as of Fig. 8;

[0025] Fig. 10 illustrates a schematic example of processing two frames of two cameras with the same neural network, according to the invention;

[0026] Fig. 11 illustrates a schematic example of processing two frames of two cameras with two neural networks, according to the invention;

[0027] Fig. 12 illustrates two exemplary ways of determining the time schedule of a command schedule; and

[0028] Fig. 13 illustrates the estimated and measured time diagrams of a same command schedule.

[0029] MODES FOR CARRYING OUT THE INVENTION

[0030] The subject matter disclosed herein comprises a processing method for carrying out a first processing task flow and at least one further processing task flow on a multicore neural network accelerator. In the following, an exemplary image processing is described for use as the method, as well as exemplary suitable architectures and further exemplary details. Fig. 1 illustrates a diagram of a generalized hardware architecture of a multi-core neural network accelerator 10. The system comprises multiple numbers of processing units 11 , 12, typically of multiple types, a command scheduler 13 and a memory 14. The scheduler 13 is connected to the memory 14, and it can read a command schedule from it. The processing units 11 , 12 also have individual respective connections with the memory 14 to read data inputs required for command execution and to write respective data outputs or command execution results. The command scheduler 13 schedules commands (in other words, command executions) on the processing units 11 , 12, as denoted by solid line arrows, and when a command execution is ready, the processing unit 11 , 12 notifies the scheduler 13 about result availability, as denoted by dashed line arrows. The scheduler 13 schedules the next and subsequent commands by taking into account such result-availability information. Thus, each command is executed by one particular processing unit 11 , 12 by being scheduled by the command scheduler 13 to the particular processing unit 11 , 12 for execution.

[0031] Each processing task flow comprises said commands and dependencies between at least some of the commands of the processing task flow. According to the invention, a compile-time optimization is carried out on the first processing task flow and on the at least one further processing task flow, wherein the compile-time optimization comprises, for each command within the processing task flows, scheduling the command to a specific processing unit 11 , 12 via the command scheduler 13 for execution.

[0032] Fig. 2 illustrates an example of a command schedule 40 compiled for execution. The command schedule 40 is stored in the memory 14 for scheduling command execution, each command 20, 21 being executable by one specific processing unit 11 , 12 and being assigned to that one specific processing unit 11 , 12, as indicated in Fig. 2 by Core # indications below Cmd # command numbers. Dependencies 30 are marked with arrows, wherein each arrow points from a dependent command to the command(s), on the results of which it depends. The inventive method comprises, during command execution, reading data for the command from the memory 14 by the specific particular processing unit 11 , 12, and writing the result of the executed command 20, 21 , back into the memory 14 by the specific processing unit 11 , 12.

[0033] Fig 3. illustrates the scheduler 13 and the processing units 11 , 12 of an exemplary multi-core neural network accelerator 10, as well as a simplified time diagram of command processing of the command schedule 40 of Fig. 2. Again, the command scheduler 13 schedules the commands 20, 21 on the processing units 11 , 12, as denoted by the solid line arrows, and when a command execution is ready, the processing unit 11 , 12 notifies the scheduler 13 about result-availability, as denoted by the dashed line arrows. As depicted in the example, the dependencies 30 between the commands 20, 21 mean that commands 20, 21 for a specific processing unit 11 , 12 must be executed in that order as they are in the command schedule 40. The command schedule 40 of Fig. 2 is fed to the scheduler 13 or is read from the memory 14 by the scheduler 13, which distributes the processing to the processing units 11 , 12, taking the dependencies 30 into account. Fig. 3 illustrates that there are gaps in the timeline for each processing unit 11 , 12, the gaps representing idle periods when the respective processing unit 11 , 12 has nothing to do. The total processing time of the command schedule 40 of Fig. 2 is t1 ; in a particular experiment, t1 = 11.9 ms.

[0034] Fig. 12 illustrates two possible exemplary ways of determining the timeline of a command schedule. If the accelerator has some kind of detailed measurement or profiling capability, then a profiler module can be applied which executes the command schedule on the accelerator and during or after the execution gathers the start and end times of each scheduled command, so the timeline can be created. If the accelerator has no such feature, then any suitable estimation method can be used in an estimator module to build such a timeline. The estimation method can be very simple or quite complicated, which partly depends on the accelerator, the point is to provide a timeline estimate of the order of the scheduled commands and of the idle periods therebetween for each core of the accelerator. The better is the estimation, the more it helps merging multiple command schedules together. Fig. 13 illustrates the difference between a measured and an estimated timeline of the same command schedule. In this case a simpler estimation algorithm was chosen which assumes that each command takes the same time, meanwhile the measurement shows small differences in the lengths of the commands. Regardless of such small differences, the order of commands and idle periods on each core are the same in both the measured and the estimated diagrams.

[0035] Fig. 4 illustrates the processing units of Fig. 3 and a time diagram of command processing of the command schedule 40 of Fig. 2 in a serial execution on two frames / images. For simplicity, the estimated schedule is displayed on this and further figures. In the serial execution, firstly, the first frame is processed, and when it is finished, the second one is processed; there is no parallelism between these processing tasks. As it can be seen, the workload according to Fig. 3 is executed twice, sequentially, which takes twice the time, i.e. , here the total processing time is t1 +t1 ; being 23.8 ms in the given example.

[0036] However, if we consider the above as one workload, then commands from the second image can be scheduled when the first commands for the first frame reach their dependency limit and the cores go to idle. Thereby, a batch of two frames can be created as a merged workload to achieve a better scheduling and to increase core utilization.

[0037] As illustrated in Fig. 5, according to a first aspect of the invention, a processing method is provided for carrying out a first processing task flow and at least one further processing task flow on the multi-core neural network accelerator 10, preferably having at least one processing unit 11 of a first type and at least one processing unit 12 of a second type different from the first type. The processing task flows comprise commands 20, 21 , 22, 23; each command 20, 21 , 22, 23 is to be executed by one specific processing unit 11 , 12, which is assigned to the particular command 20, 21 , 22, 23 by means of command scheduling.

[0038] Each processing task flow further comprises dependencies 30. The dependencies 30 comprise a first result-availability which is an availability of one or more results written into the memory 14 by a first subset of the processing units 11 , 12. The first resultavailability is a condition for starting command execution on a second subset of the processing units 11 , 12, wherein the first subset contains at least one processing unit 11 , 12 not contained in the second subset, or in other words, the second subset contains at most some, but not all of the processing units 11 , 12 of the first subset.

[0039] The dependencies 30 further comprise a second result-availability which is an availability of further one or more results written into the memory 14, wherein the second result-availability is a condition for continuing command execution on the at least one processing unit 11 , 12 not contained in the second subset. The second result-availability can be the availability of the results of the second subset, but this is not necessary; any subsequent result-availability in the processing graph may constitute a condition for such command execution continuing. In this way, a potential idle gap would be present, without the invention, on the at least one processing unit 11 , 12 not contained in the second subset.

[0040] In other words, dependencies 30 are maintained between commands by:

[0041] - checking whether a first result-availability condition is met, wherein the first result-availability condition is met when one or more results, written into the memory 14 by a first set of processing units 11 , are available, and starting command execution on a second set of processing units 12 only after first result-availability condition is met, where the first subset of processing units 11 includes at least one processing unit 11 not included in the second subset; and

[0042] - checking whether a second result-availability condition is met, wherein the second result-availability condition is met when further one or more results, written into the memory 14, are available, and only continuing command execution on at least one processing unit 11 not included in the second subset after the second resultavailability condition is met.

[0043] The term subset is to be interpreted in its broadest possible meaning; generally, it can consist of one, some, or all of the processing units 11 , 12 of the multi-core neural network accelerator 10., An exemplary first result-availability in Figs. 3 to 5 is the finished processing of Cmd 2 and Cmd 3 on processing units Core 0 and Core 1 , both being, e.g., of convolutional type. In the depicted example, the condition of the first result-availability is fulfilled when the timeline reaches the right side of the blocks of Cmd 2 and Cmd 3 representing the respective command executions. An exemplary second result-availability in Figs. 3 to 5 is the finished processing of Cmd 5 on processing unit Core 2 being, e.g., of pooling type. It is noted that execution times of parallel executed commands are not necessarily equal, as schematically depicted for simplicity, so in case a result-availability is constituted by the availability of more than one result, then result-availability prevails upon the last of those becomes available.

[0044] As illustrated in Fig. 6, the scheduling of the commands 20, 21 , 22, 23 by the command scheduler 13 is determined by a merged command schedule 41 maintaining the dependencies within the respective processing task flows. The merged command schedule 41 includes, for the at least one processing unit 11 , 12 not contained in the second subset and between the first and second resultavailabilities of the first processing task flow, a scheduled start of execution of at least one command 22, 23 of the at least one further processing task flow.

[0045] By filling - at least partly - the gaps or idle time periods as above, latency times of the processing units 11 , 12 can be reduced. The invention utilizes the fact that the processing units 11 , 12 serve for executing universal commands, into which each and every processing task flow can be compiled. By means of corresponding scheduling and memory handling, commands belonging to different processing task flows can be scheduled in an interleaved or merged command schedule 41 as illustrated in Fig. 6.

[0046] Fig. 5 also illustrates that commands 22, 23 from the second processing task flow are scheduled into the idle period gaps of the first processing task flow, without affecting the dependencies of the commands of the second processing task flow. In this way, the two workloads can be completed in a shorter time t2 <= t1 + t1 than in the case of sequential execution; in a given example, t2 = 16.37 ms has been achieved.

[0047] As illustrated in Figs. 5 and 6, the at least one further processing task flow also comprises dependencies 31 comprising result-availability conditions, and scheduling of the commands 22, 23 of the at least one further processing task flow can be carried out by maintaining the dependencies 31 , while being merged with the first processing task flow by means of the merged command schedule 41 .

[0048] Fig. 7. illustrates an exemplary multi-step compilation process. As the first step, the neural networks are compiled using the original compiler belonging to the accelerator. This creates the command queues associated with the neural networks. Then, timelines corresponding to all the command queues are determined. This subprocess is illustrated by the previously discussed Fig. 12. Once these timelines or schedulings are available, the final phase of the multi-step compilation follows, where a command schedule merger unit 15 merges the original command queues, taking into account the previously determined timelines or schedulings associated with them. The figure illustrates the case when workloads from two different neural networks are to be run on the accelerator, but of course, the number of neural networks can be larger, or the same neural network can appear in multiple instances. The point is that the command schedule merger unit 15 takes at least two workloads as input and creates a timely optimized workload from them.

[0049] There are several types of rules and algorithms that can be used for the merge process; with the goal of making the merged command queues execute faster together than if the command queues were run separately, and of course with not harming the dependencies of the original commands. Examples may include:

[0050] - A simplest solution is to schedule into an idle period of the first command queue one or more commands from the next command queue that the dependencies allow. Here the lengths of the idle periods are not considered, so it may happen that the inserted commands "postpone" subsequent commands of the first command queue.

[0051] - A slightly more sophisticated solution is to take into account the lengths of idle periods and commands, and try to select from the available commands of the second command queue on this basis. Here the selection algorithms can be based on e.g. first fit, best fit, etc.

[0052] - Both of the previously described methods enable to insert more than one commands into a gap if there is more available command from the second command queue and the overall runtime becomes more optimal this way. - A further possible method handles the merge process more freely and allows the merge algorithm not to necessarily insert commands from the second command queue into idle periods, but to merge the command queues in any way that takes into account further factors in addition to the dependencies in the command queues, and thus to achieve a more optimal runtime overall. Further factors may include taking into account the number of cores and transferring commands from one to the other between the same cores. Further, if the memory traffic generated by each command can be measured or estimated, the interaction of commands running at the same time can also be taken into account as a function of the total available bandwidth.

[0053] This multi-step optimization process can still be considered a compile-time optimization, as all steps need to be performed only once, before the actual execution of the workloads. The merged command queue can then be reused any number of times, and the time spent on the optimization process does not impact runtime performance.

[0054] Figs. 8 and 9 illustrate another example of serial and interleaved processing of two frames by a multi-core neural network accelerator 10, by which the idle times of the processing units 11 , 12 can be reduced and total processing time can be shortened.

[0055] Fig. 10 illustrates a schematic example of processing two frames or images 52, 53 of two cameras 50, 51 with the same neural network 55, in line with the invention. This diagram shows the process of batching the two different camera images 52, 53 into one workload 54, constituted by the batched input frames. The neural network 55 is transformed into a set 60 of instructions, which instructions are fully equivalent to the functioning of the neural network 55, and can be interpreted by the target hardware, i.e. , by the multi-core neural network accelerator 10. The set 60 of instructions can be called an "application" or a “binary” that can be run by the hardware. If this binary is loaded into the target hardware's memory 14 and run on some input images 52, 53, then the neural network 55 is executed on the target hardware, which is called execution. Fig. 11 illustrates a schematic example of processing two frames or images 52, 53 of two cameras 50, 51 with two neural networks 55, 56, in line with the invention. Here two different camera images 52, 53 are batched, one image 52 with one of the neural networks 55, 56 and the other image 53 with the other neural network 55, 56 into one workload 54. There can be many other use cases, these are only two examples.

[0056] The second use case depicted in Fig. 11 is an example of not only batching multiple frames, but also batching two (or more) neural networks. For example, on a selfdriving car, there can be two camera streams, one that monitors the road and uses the image to self-drive a car and another camera that monitors the driver to see if the hands are on the steering wheel and if the driver is not asleep. These two camera images or camera streams are to be processed practically simultaneously with two different neural networks, as we need a model to keep the car on the road and a different one to decide if a person is awake / conscious. Then the two neural networks 55, 56 are compiled to run as a merged workload on the multi-core neural network accelerator 10. This way, when the processing of the first camera image 52 with the first neural network 55 (which contributes to the self-driving of the car with an outwardfacing camera) reaches a point where it should be waiting, the scheduler 13 will be able to schedule and start processing the second camera image 53 with the commands of the second neural network 56, which is inward looking to see whether the driver is awake.

[0057] Of course, it is also possible to include more than two frames, images, image series, or streams in a merged workload. The above examples show batching of two frames for simplicity, but the invention also works the same way for batching more than two processing task flows.

[0058] Accordingly, in a preferred embodiment, the first processing task flow and the at least one further processing task flow are image processing task flows, and by means of these processing task flows

[0059] - the images 52, 53 from the two or more cameras 50, 51 are processed with the same neural network 55, - the images 52, 53 from the two or more cameras 50, 51 are processed with the more than one neural network 55, 56, or

[0060] - the same image 52, 53 is processed with the more than one neural network 55, 56.

[0061] In the processing method preferably each command 20, 21 , 22, 23 has a unique command ID and each processing unit 11 , 12 has a unique processing unit ID, and the merged command schedule 41 comprises command ID and processing unit ID pairs. The pairs are preferably determined by the command schedule merger unit 15.

[0062] According to a second aspect, the invention is a processing system comprising a multi-core neural network accelerator 10 with a processor adapted to perform the steps of the inventive processing method.

[0063] According to a further aspect, the invention is a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the inventive processing method.

[0064] According to a still further aspect, the invention is a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the inventive processing method.

[0065] Embodiments of the invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment including both hardware and software elements. An embodiment may be implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

[0066] In summary, if multiple workloads are available at a given time, then these workloads can be merged into one merged workload, and when no command can be scheduled to a core from the first workload, then the invention tries to schedule a command from a second workload. This reordering of the commands of the original workloads prevents or reduces idle times of the cores, significantly increasing utilization. The industrial applicability of the invention follows from the above-detailed description, with the following general advantages. Firstly, the runtime decreases; therefore, efficiency increases without any hardware change. The essence of this technology is that the batched execution is a compile-time software optimization, so there is no need to change anything in the hardware, which would be expensive and time-consuming. Secondly, the invention is widely applicable: multiple frames from different cameras, different neural networks, or even multiple frames and different neural networks (e.g., two camera inputs processed by two different neural networks) can be batched in the disclosed framework.

[0067] This written description uses examples to disclose the subject matter, and also to enable any person skilled in the art to practice the subject matter, including making and using any devices or systems and performing any incorporated methods. The patentable scope is defined by the claims and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.

Claims

CLAIMS1. A processing method for carrying out a first processing task flow and at least one further processing task flow on a multi-core neural network accelerator (10) having processing units (11 , 12), a command scheduler (13), and a memory (14), wherein a compile-time optimization is carried out on the first processing task flow and on the at least one further processing task flow, each comprising a series of commands (20, 21 , 22, 23) and dependencies (30) between at least some of those commands (20, 21 , 22, 23), and wherein the compile-time optimization comprises, for each command (20, 21 , 22, 23) within the processing task flows, scheduling the command to a specific processing unit (11 , 12) via the command scheduler (13) for execution; wherein the method comprises:- during command execution, reading data for the command from the memory (14) by the specific particular processing unit (11 , 12), and writing the result of the executed command (20, 21 , 22, 23) back into the memory (14) by the specific processing unit (11 , 12);- maintaining dependencies (30) between commands by:- checking whether a first result-availability condition is met, wherein the first result-availability condition is met when one or more results, written into the memory (14) by a first set of processing units (11 ), are available, and starting command execution on a second set of processing units (12) only after first result-availability condition is met, where the first subset of processing units (11 ) includes at least one processing unit (11 ) not included in the second subset; and- checking whether a second result-availability condition is met, wherein the second result-availability condition is met when further one or more results, written into the memory (14), are available, and only continuing command execution on at least one processing unit (11 ) not included in the second subset after the second result-availability condition is met;- and applying a merged command schedule by the command scheduler (13), which merged command schedule maintains the dependencies within the respective processing task flows and includes, between the first and secondresult-availabilities of the first processing task flow, a scheduled start of execution of at least one command (22, 23) from the at least one further processing task flow.

2. The processing method according to claim 1 , wherein said compile-time optimization comprises creating a timeline for the command execution of a command schedule (41 ) based on estimation or measurement.

3. The processing method according to claim 1 or claim 2, wherein the merged command schedule is generated by a separate command schedule merger unit (15) based in input command schedules, their dependencies and timelines.

4. The processing method according to claim 1 to 3, wherein scheduling is carried out on the basis of result-availability information communicated by each of the processing units (11 , 12) to the command scheduler (13).

5. The processing method according to any of claims 1 to 4, wherein the multi-core neural network accelerator (10) has at least one processing unit (11 ) of a first type and at least one processing unit (12) of a second type different from the first type.

6. The processing method according to claim 5, wherein the at least one processing unit (11 ) of the first type is a convolutional processing unit, and at least one processing unit (12) of a second type is a processing unit adapted to carry out pooling.

7. The processing method according to any of claims 1 to 6, wherein the first processing task flow and the at least one further processing task flow are image processing task flows, and by means of these processing task flows- images (52, 53) from two or more cameras (50, 51 ) are processed with the same neural network (55),- images (52, 53) from two or more cameras (50, 51 ) are processed with more than one neural network (55, 56), or- the same image (52, 53) is processed with more than one neural network (55, 56).- 18 -8. The processing method according to any of claims 1 to 7, wherein each command (20, 21 , 22, 23) has a unique command ID and each processing unit (11 , 12) has a unique processing unit ID, and wherein the merged command schedule (41 ) comprises command ID and processing unit ID pairs.

9. The processing method according to claim 8, wherein the pairs are determined by the command schedule merger unit (15).

10. A processing system comprising a multi-core neural network accelerator (10) with a processor adapted to perform the steps of the processing method of claim 1 .11 . A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the processing method of claim 1 .

12. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the processing method of claim 1.

Images