An on-chip network defragmentation method for multi-task scheduling in neuromorphic chips

Abstract

This invention discloses an on-chip network fragmentation method for multi-task scheduling in neuromorphic chips, comprising: acquiring the relevant structures and configurations of multiple spiking neural network tasks; mapping the spiking neural networks to multiple cores of the on-chip network; and rearranging the core mapping when old tasks end and new tasks are created; using communication cost and core occupancy rate as mapping benefits, and employing a heuristic backtracking random ramping algorithm to progressively find better mapping positions; and evaluating whether the time saved by the mapping in actual application is sufficient to compensate for the time required to set up the mapping on the on-chip network, to finally confirm whether to use the mapping. The method of this invention effectively improves the utilization rate of on-chip network cores, thereby improving chip computing efficiency and average task power consumption.

Description

Technical Field

[0001] This invention belongs to the field of neuromorphic chips, and in particular relates to an on-chip network fragmentation method for multi-task scheduling of neuromorphic chips. Background Technology

[0002] Neural networks are considered the main driving force for the development of artificial intelligence. In recent years, spiking neural networks, which originated from brain science, have attracted the attention of scholars with their rich spatiotemporal neurodynamic characteristics, diverse encoding mechanisms, and event-driven advantages.

[0003] For example, Chinese patent document CN110909869A discloses a neuromorphic computing chip based on a spiking neural network; Chinese patent document CN115392445A discloses a spiking neural network chip architecture, chip, and system.

[0004] This spiking neural network acceleration chip, implemented with high-density on-chip network hardware, adopts a scalable architecture. Multiple computing cores and chips operate simultaneously, exchanging intermediate results data through a routing network. The entire system typically lacks a unified external memory; instead, each computing core has its own independent storage space, exhibiting a decentralized operating mode that results in extremely high parallelism and memory access efficiency.

[0005] However, this makes information communication between different cores of the on-chip network a performance bottleneck. Therefore, efficiently loading spiking neural networks onto neuromorphic chips, especially efficiently loading multiple networks onto the same chip, has become an important means to improve chip utilization and learning and inference performance, and is also a problem that urgently needs to be solved. Summary of the Invention

[0006] This invention provides an on-chip network fragmentation method for multi-task scheduling of neuromorphic chips, which effectively improves the utilization rate of on-chip network cores, thereby improving chip computing efficiency and reducing average power consumption per task.

[0007] A method for defragmenting on-chip networks in multi-task scheduling of neuromorphic chips includes the following steps:

[0008] (1) When assigning spiking neural network tasks to on-chip networks, obtain the relevant configuration of the tasks, including the graph structure of each task after the spiking neural network is divided into clusters, the weights generated by estimating the communication load according to the graph edges, and the estimated number of times the task needs to be executed on the chip.

[0009] (2) When multiple tasks are executed on the on-chip network, for each task, the estimated number of executions remaining when continuing to execute according to the current core configuration is multiplied by the communication consumption of the current task under the current configuration, and recorded as the total communication consumption expected for the task.

[0010] (3) A heuristic random ramping algorithm with backtracking is used to randomly select from two types of actions, linear transformation and fragmentation, to rearrange the core configuration corresponding to the current task of the on-chip network, so as to reduce the total communication consumption of all tasks in the end.

[0011] (4) Whenever a task completes its execution and releases the core it used, execute steps (2) to (3) until all on-chip tasks have been completed.

[0012] At the same time, maintain a list of the least used tasks. When a task is called by a user, its number is inserted at the head of the list. Whenever a new task is added, if there are not enough free cores, the least used tasks are unloaded in turn until there are enough free cores to load the new task. At this time, the new task is loaded in the same way as steps (2) to (3).

[0013] This invention maps spiking neural networks to multiple cores of an on-chip network and rearranges the core mapping when an old task ends and a new task is created. The mapping benefits are calculated based on communication cost and core occupancy rate. A heuristic random ramping algorithm with backtracking is used to gradually find better mapping positions. The time saved by the mapping in actual application is evaluated to see if it is enough to make up for the time required to set up the mapping on the on-chip network, so as to finally confirm whether to use the mapping.

[0014] In step (1), the graph structure is stored in the form of an adjacency list, where the weight is based on the communication traffic generated by the two nodes connected by the edge when inferring on the dataset. The number of times the task needs to be executed on the chip is specified by the user.

[0015] In step (2), the communication cost is obtained by multiplying the weight W of the edge by the Manhattan distance between the two cores connected by the edge. The total communication cost expected for each task is:

[0016]

[0017] Cost j This represents the total communication consumption expected for the j-th task during its lifecycle; BeginCore i and EndCore i These are the two core vertices of the i-th edge; N left =NN done It is the estimated number of executions N minus the number of executions completed N. done That is, the remaining estimated number of executions.

[0018] In step (3), the heuristic random climbing algorithm with backtracking includes three action types: linear transformation, defragmentation, and backtracking to the previously recorded state. Linear transformation involves randomly translating, rotating, and mirroring the cores mapped to the task on the on-chip network. Defragmentation involves assuming there are n tasks and making n attempts, where the i-th attempt only considers the i tasks with the smallest number of cores and remaps their cores.

[0019] For the three action types mentioned above, we try multiple times based on the original state, calculate the total communication cost (TotalCost) of all tasks after executing the above action types, and randomly select an action to execute from all actions that reduce communication cost. The probability of the action is determined by the degree of reduction in communication cost; the greater the reduction, the higher the probability of selection. We also record the old state for later backtracking.

[0020] Repeat the above actions until the total communication cost of all tasks converges or the number of executions exceeds the number specified by the user.

[0021] Furthermore, among the three action types, linear transformation and fragmentation, when generating random actions, calculate the geometric center positions of all occupied cores and increase the probability of generating actions that move to that position according to user-specified parameters.

[0022] Furthermore, among the three action types, when tracing back to a previously recorded state, if there are tasks that have been completed or temporarily unloaded in the previously saved records, the corresponding core is also set to unoccupied before further action generation is considered.

[0023] Furthermore, when backtracking to a previously recorded state, a stack stores all the previous old states so that the backtracking action can be performed under any circumstances.

[0024] Furthermore, in step (4), the task list is implemented by a queue, and each task is assigned a unique auto-incrementing ID.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] This invention can fully schedule and improve the utilization of on-chip networks, reduce the pulse packet communication overhead of on-chip networks, increase the number of running tasks, improve the utilization of neuron cores, and reduce the power consumption per task on the chip. Attached Figure Description

[0027] Figure 1 This is a flowchart of an on-chip network fragmentation method for multi-task scheduling of a neuromorphic chip according to the present invention;

[0028] Figure 2Here is an example of a graph representation of a clustered spiking neural network;

[0029] Figure 3 An example of deploying multiple spiking neural networks on a chip. Detailed Implementation

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate the understanding of the present invention and do not constitute any limitation thereof.

[0031] like Figure 1 As shown, an on-chip network fragmentation method for multi-task scheduling of neuromorphic chips includes the following steps:

[0032] S1, obtain a list of relevant configurations for several spiking neural network tasks, including: the graph structure of the spiking neural network for each task after being segmented into clusters, the weights generated according to the estimated communication load on the graph edges, and the estimated number of times the task needs to be executed on the chip.

[0033] The spiking neural network task is defined as a spiking neural network model, in which neurons have been pre-clustered according to the principle of minimizing inter-node communication overhead, but no actual physical cores have been assigned.

[0034] The specified graph structure is stored as an adjacency list, where the weights are based on the communication traffic generated during inference on the dataset between the two nodes connected by the edge. The number of executions can be specified by the user on an approximate order of magnitude, such as ten or one hundred times.

[0035] S2, when multiple tasks need to run on the chip simultaneously, for each task, record the estimated number of executions remaining when continuing execution according to the current core configuration, multiplied by the current task's communication consumption under the current configuration, and recorded as the total estimated communication consumption of the task.

[0036] The communication cost is obtained by multiplying the edge weight W by the Manhattan distance between the two cores connected by the edge.

[0037] The estimated communication cost of the j-th task over its lifecycle is:

[0038]

[0039] In the formula, BeginCore i and EndCore i These are the two core vertices of the i-th edge; N left =NN done It is the estimated number of executions N minus the number of executions completed N. done The total communication cost for all tasks currently executed on the chip is...

[0040] TotalCost=∑ j Cost j

[0041] Because the on-chip network is a square mesh, its communication routes are determined by the on-chip network router according to routes with Manhattan distance properties. However, if the neuromorphic chip adopts an on-chip network with other geometric properties (such as hexagonal, triangular, etc.), the corresponding distance also needs to be modified to reflect the distance of the actual routing lines.

[0042] S3, following these steps, run the following algorithm on the CPU to attempt to change the core configuration used by all currently running tasks, with the aim of reducing the total communication overhead of the changed mapping.

[0043] The Stochastic Hill Climbing Heuristic With Backtrace algorithm generates a series of actions according to the rules detailed below and selects the best one, which minimizes the Total Cost of communication.

[0044] Action Type 1: Linear Transformation, which performs random translation, rotation, and mirroring actions on the on-chip network of the kernel mapped to the task.

[0045] Action Type 2: Fragmentation. Suppose there are n tasks and n attempts are made. In the i-th attempt, only the top i tasks with the smallest core usage are considered, and cores are remapped for them. A heuristic core aggregation algorithm is adopted.

[0046] Action type 3: Revert to the previously recorded state.

[0047] For all the above action types, try multiple times based on the original state, calculate the TotalCost after executing the above actions, and randomly select an action to execute from all actions that reduce communication consumption. The probability of selecting an action is determined by the degree of reduction in communication consumption; the greater the reduction, the higher the probability of selection.

[0048]

[0049] Wherein, ΔTotalCost j P represents the reduction value of the j-th action that reduced communication consumption. i This represents the probability of selecting the action. The old state is recorded for possible backtracking later. The above actions are repeated until the total communication consumption converges and essentially no longer decreases (the change is less than a threshold) or the number of executions exceeds the user-specified number (the number of action attempts is set by the user as a hyperparameter).

[0050] When generating actions of types one and two, the geometric center positions of all occupied cores are calculated, and the probability of generating an action that moves to that position is increased according to the parameters specified by the user.

[0051] In action type three, if there are tasks that have been completed or temporarily uninstalled in the previously saved records, the corresponding core will also be set to unused before considering generating further actions.

[0052] Action type three uses a stack to store all previous old states so that backtracking actions can be performed under any circumstances.

[0053] S4, whenever a task completes its execution and releases its core, steps S2 to S3 are executed until all on-chip tasks have been completed.

[0054] Maintain a list of least used tasks. When a task is invoked by a user, its number is inserted at the head of the list. The task list is implemented using a queue, and each task is assigned a unique auto-incrementing ID. Whenever a new task is added, if there is insufficient free core area, the least used task is unloaded until there are enough free cores to load the newly created task. At this point, the new task is loaded onto the on-chip network according to steps S2 to S3.

[0055] In this embodiment of the invention, multiple pre-processed (clustered, weighted) spiking neural networks are used as examples to illustrate the key points and advantages of the task scheduling of the present invention. The on-chip network is set to have identical $4×4$ cores. After preprocessing, the icons of the three example tasks are shown in the figure. Figure 2 The expected number of executions, $N$, is marked below each diagram. The scheduling algorithm is executed when initializing all tasks, adding new tasks, and completing old tasks. Here, we take one example of the process for initializing tasks, adding new tasks, and completing old tasks.

[0056] Initialize the task: First, randomly assign a core to each task and perform multiple actions as described in S3 until the total communication consumption converges and no longer decreases or the number of executions exceeds the number specified by the user.

[0057] Add a new task (or reload a temporarily unloaded task onto the on-chip network): If there are not enough cores, start by unloading the least used task one by one until there are enough cores. Then load the new task onto the on-chip network and execute S3 until its termination condition is met (total communication cost converges or too many executions are performed).

[0058] Complete old tasks: After unloading the completed tasks from the on-chip network, rearrange the current tasks on the entire on-chip network as described in S3.

[0059] This process continues until all tasks are completed.

[0060] Figure 3 Taking the initial deployment of three tasks on the on-chip network as an example, appropriate core locations are assigned to these three tasks. This is calculated and allocated by a heuristic algorithm. Based on the constraints of communication overhead and core location, the physical location of the core used by the three tasks is obtained. The algorithm is only triggered again at startup and when tasks are added and completed, so as to remap and deploy the tasks according to the results obtained by the heuristic algorithm.

[0061] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for defragmenting on-chip networks in multi-task scheduling of neuromorphic chips, characterized in that, The process includes the following: (1) When assigning spiking neural network tasks to on-chip networks, obtain the relevant configuration of the tasks, including the graph structure of each task after the spiking neural network is divided into clusters, the weights generated by estimating the communication load according to the graph edges, and the estimated number of times the task needs to be executed on the chip. Maintain a list of least used tasks. When a task is invoked by a user, its number is inserted at the head of the list. Whenever a new task is added, if there are not enough free cores, unload the least used tasks one by one until there are enough free cores to load the new task. (2) When multiple tasks are executed on the on-chip network, for each task, the estimated number of executions remaining when continuing to execute according to the current core configuration is multiplied by the communication consumption of the current task under the current configuration, and recorded as the total communication consumption expected for the task. (3) A heuristic random ramping algorithm with backtracking is used to randomly select from two types of actions, linear transformation and fragmentation, to rearrange the core configuration corresponding to the current task of the on-chip network, so as to reduce the total communication consumption of all tasks in the end. The heuristic random ramping algorithm with backtracking includes three action types: linear transformation, defragmentation, and backtracking to the previously recorded state. Linear transformation involves randomly translating, rotating, and mirroring the cores mapped to the task on the on-chip network. Defragmentation involves assuming there are n tasks and n attempts, where the i-th attempt only considers the i tasks with the smallest number of cores and remaps their cores. For the three action types mentioned above, we try multiple times based on the original state, calculate the total communication cost (TotalCost) of all tasks after executing the above action types, and randomly select an action to execute from all actions that reduce communication cost. The probability of the action is determined by the degree of reduction in communication cost; the greater the reduction, the higher the probability of selection. We also record the old state for later backtracking. Repeat the above steps until the total communication cost of all tasks converges or the number of executions exceeds the number specified by the user; (4) Whenever a task completes its execution and releases the core it used, execute steps (2) to (3) until all on-chip tasks have been completed.

2. The on-chip network fragmentation method for multi-task scheduling of neuromorphic chips according to claim 1, characterized in that, In step (1), the graph structure is stored in the form of an adjacency list, where the weight is based on the communication traffic generated by the two nodes connected by the edge when inferring on the dataset. The number of times the task needs to be executed on the chip is specified by the user.

3. The on-chip network fragmentation method for multi-task scheduling of neuromorphic chips according to claim 1, characterized in that, In step (2), the communication cost is obtained by multiplying the weight W of the edge by the Manhattan distance between the two cores connected by the edge. The total communication cost expected for each task is: Cost j This represents the total communication consumption expected for the j-th task during its lifecycle; BeginCore i and EndCore i These are the two core vertices of the i-th edge; N left =NN done It is the estimated number of executions N minus the number of executions completed N. done That is, the remaining estimated number of executions.

4. The on-chip network fragmentation method for multi-task scheduling of neuromorphic chips according to claim 1, characterized in that, Of the three action types, Linear Transformation and Fragmentation calculate the geometric center positions of all occupied cores during random generation and increase the probability of generating an action that moves to that position according to user-specified parameters.

5. The on-chip network fragmentation method for multi-task scheduling of neuromorphic chips according to claim 1, characterized in that, Among the three action types, when retracing back to a previously recorded state, if there are tasks that have been completed or temporarily unloaded in the previously saved records, the corresponding core will also be set to unoccupied before further action generation will be considered.

6. The on-chip network fragmentation method for multi-task scheduling of neuromorphic chips according to claim 5, characterized in that, When backtracking to a previously recorded state, a stack stores all the previous old states so that the backtracking action can be performed under any circumstances.

7. The on-chip network fragmentation method for multi-task scheduling of neuromorphic chips according to claim 1, characterized in that, In step (1), the task list is implemented by a queue, and each task is assigned a unique auto-incrementing ID.

Images