System and method for mapping a spiking neural network onto a neuromorphic processor

Abstract

A method for mapping a spiking neural network design onto a configurable neuromorphic processor, the method comprising: defining a resource model describing a neuromorphic processor; defining a network definition file describing a spiking neural network design, further comprising: dividing the neurons in the network definition file into one or more partitions, each partition comprising a portion of neurons from one of the layers; selecting one of the partitions; selecting one of the available planes of the resource model, wherein the number of neurons in the plane is greater than or equal to the number of neurons in the selected partition; and mapping the neurons of the selected partition to the neurons of the selected plane of the resource model, wherein the neurons of the selected partition are mapped to the neurons of the selected plane if the neurons of the selected plane have available interconnections that match the interconnections to the neurons of the selected partition.

Description

Technical Field

[0001]

[0001] The disclosed disclosure is in the field of spiking neural networks, and in particular, in the field of hardware implementation of spiking neural networks on hardware accelerators such as neuromorphic chips, and in particular, the hardware implementation is performed by a mapping algorithm.

Background Art

[0002]

[0002] The concept of mapping refers to configuring a hardware accelerator such as a neuromorphic chip to efficiently and reliably execute a network inference task. Network inference involves making predictions or classifications based on input data using a trained neural network model. However, executing neural network inference on dedicated hardware such as a neuromorphic chip requires careful configuration to ensure optimal performance and accuracy.

[0003]

[0003] The mapping process involves addressing various issues that arise when deploying neural network inference on a hardware accelerator.

[0004]

[0004] Hardware accelerators may have limitations in terms of memory, computing resources, and power consumption. Mapping may involve optimizing the neural network model to fit within these constraints while maintaining performance. Efficient data movement between different components of the hardware accelerator is important to minimize latency and maximize throughput. Mapping may also involve designing data flow patterns that minimize data movement overhead. Mapping may further involve selecting or designing algorithms suitable for execution on the hardware accelerator architecture. This may involve algorithm optimization to leverage the parallelism or specialized features of the hardware. Different neural network architectures may have different requirements and characteristics when deployed on a hardware accelerator. Mapping may also involve selecting or designing a network topology that is well suited for efficient execution on the target hardware.

[0005]

[0005] Overall, the goal of the mapping process is to enable reliable and efficient execution of neural network inference on hardware accelerators, taking into account the constraints and characteristics of the underlying hardware architecture.

[0006]

[0006] The concept of mapping can therefore be defined as the process of configuring a hardware accelerator (for example, implemented as a neuromorphic chip) to perform network inference. The mapping process aims to solve certain problems encountered when neural network inference needs to be performed on hardware so that reliable inference can be achieved.

[0007]

[0007] However, when performing the mapping, certain problems need to be resolved.

[0008]

[0008] For example, neurons connected to other neurons via synapses in a neural network should be connected in hardware so that the neurons can send spikes to each other. Under certain special conditions, the hardware should be configured so that spike traffic from off-chip to the neuromorphic hardware array is not hindered. Users of mapping may have a preferred preference for using hardware acceleration functions such as encoders, decoders, etc. Furthermore, users of mapping may configure their neural networks such that neurons and synapses have certain properties that can only be mapped to certain types of hardware neurons and synapses. Finally, mapping may require optimization for dynamic power consumption and latency between neurons on the chip.

[0009]

[0009] Inside the hardware, the components are limited to a certain number and interconnected in a specific way. This is referred to as the chip architecture. Such architectures can generally have limitations when configuring them to run a network. These limitations can arise from the connectivity of the components and from the runtime requirements of the hardware. Furthermore, the hardware user will need to further adjust the hardware configuration based on the requirements of the neural network model. From this, it can be seen that the problems this process solves are highly specific to the target architecture. [Overview of the project]

[0010]

[0010] In order to solve the above, the inventors of the present invention recognized that the mapping process requires a complete representation of the hardware architecture in software and a method of representing the neural network in a general way. In addition, this process must consist of an algorithm that must have at least the following requirements: The algorithm should be able to map loosely connected networks and fully connected networks in a short amount of time. The algorithm should also aim to identify any hardware constraints and find a mapping solution for the network by avoiding these constraints. Finally, the algorithm should be able to identify any conflicts arising from hardware constraints that should result in mapping being impossible.

[0011]

[0011] The present invention comprises a computer implementation mapping system and a computer implementation mapping method for more efficiently configuring a hardware accelerator (neuromorphic chip) or for generating a configuration file to be used to configure a hardware accelerator. The mapping system and method generate a configuration for a hardware accelerator based on a definition of a neural network to be implemented in the hardware accelerator, and provide an improved configuration for the hardware accelerator that enables the hardware accelerator to perform desired neural network inference more efficiently.

[0012]

[0012] In order to solve the problems described above, the subject matter of the claims of the present invention is proposed. Certain aspects of the present invention are described below.

[0013]

[0013] In a first aspect of the present invention, a method for mapping a spiking neural network design onto a configurable neuromorphic processor is disclosed. The method comprises the steps of defining a resource model that describes a neuromorphic processor, wherein the resource model defines one or more planes, each plane comprising a plurality of presynapses, synapses, and spiking neurons arranged in a subnetwork, each presynapse being adapted to receive one or more inputs, each neuron having an output, each synapse connecting one of the presynapses to one of the neurons, the resource model defining a plurality of interconnections for connecting the output of one of the neurons in one of the planes to one or more presynapses in one of the planes, and defining a network definition file that describes a spiking neural network design, wherein the network definition file defines a plurality of neurons interconnected by a plurality of synapses, the neurons being arranged in layers, each neuron in one of the layers being connected to one or more neurons in the same layer or another of the layers, and may comprise at least one of the following.

[0014]

[0014] The method comprises the steps of dividing the neurons in a network definition file into one or more partitions, where each partition comprises a portion of the neurons from one of the layers, selecting one of the partitions, selecting one of the available planes of a resource model, where the number of neurons in the plane is greater than or equal to the number of neurons in the selected partition, and mapping the neurons of the selected partition to the neurons of the selected plane of the resource model, wherein the neurons of the selected partition are mapped to the neurons of the selected plane if the neurons of the selected plane have available interconnections that match the interconnections to the neurons of the selected partition, and further comprising at least one of the following.

[0015]

[0015] In an embodiment of the first aspect, selecting one of the available planes further comprises selecting a plane in which the number of inputs to neurons in the plane is equal to or greater than the number of inputs to neurons in the selected partition.

[0016]

[0016] In the embodiment of the first aspect, selecting one of the available planes further comprises selecting a plane in which the number of external inputs to neurons of the plane is equal to or greater than the number of external inputs to neurons of the selected partition, and the number of regressive inputs to neurons of the plane is equal to or greater than the number of regressive inputs to neurons of the selected partition.

[0017]

[0017] In the first embodiment, mapping neurons in a selected partition comprises assigning one of the neurons in a selected plane to one of the neurons in the selected partition, and assigning the interconnection to the assigned neuron in the selected plane if the neurons in the selected partition have a corresponding interconnection.

[0018]

[0018] In the embodiment of the first aspect, if an interconnection to an assigned neuron in the selected plane corresponding to an interconnection to a neuron in the selected partition cannot be found or has already been assigned, a different neuron in the selected plane is assigned to the neuron in the selected partition.

[0019]

[0019] In the first embodiment, assigning interconnections to the assigned neurons of the selected plane comprises assigning external inputs to the assigned neurons of the selected plane when the neurons of the selected partition have external inputs, and assigning regressive inputs to the assigned neurons of the selected plane when the neurons of the selected partition have regressive inputs.

[0020]

[0020] In the first embodiment, assigning interconnections to assigned neurons in a selected plane comprises assigning connections to assigned neurons via interconnections when neurons in a selected partition have inputs from neurons in different partitions.

[0021]

[0021] In the first embodiment, assigning interconnections to the assigned neurons of the selected plane includes removing synaptic input to the assigned neurons of the selected plane when the neurons of the selected partition do not have corresponding synaptic input.

[0022]

[0022] In an embodiment of the first aspect, the method further comprises defining a set of mapping constraints that describe potentially conflicting interconnections in the resource model.

[0023]

[0023] In the embodiment of the first aspect, the set of mapping constraints includes indications of the quality and type of interconnections to neurons in a partition of the network definition file.

[0024]

[0024] In the embodiment of the first aspect, the set of mapping constraints includes an indication of the quality of interconnection between neurons in different partitions of the network definition file.

[0025]

[0025] In the embodiment of the first aspect, if neurons in the selected partition are successfully mapped to neurons in the selected plane, one of the planes in the resource model is shown as unavailable.

[0026]

[0026] In the embodiment of the first aspect, if the neurons in the selected partition are not successfully mapped to neurons in the previously selected plane, another plane of the resource model is selected.

[0027]

[0027] In an embodiment of the first aspect, the method further comprises merging a partition of the network definition file with another partition of the network definition file when the number of partitions exceeds the number of planes in the resource model.

[0028]

[0028] In a second aspect of the present invention, a system for mapping a spiking neural network design onto a configurable neuromorphic processor is disclosed. The system includes a first memory for storing a resource model that describes the neuromorphic processor, where the resource model defines one or more planes, each plane comprising a plurality of presynapses, synapses, and spiking neurons arranged in a subnetwork, each presynapse being adapted to receive one or more inputs, each neuron having an output, each synapse connecting one of the presynapses to one of the neurons, the resource model defining a plurality of interconnects for connecting an output of one of the neurons in one of the planes to one or more of the presynapses in one of the planes, a second memory for storing a network definition file that describes the spiking neural network design, where the network definition file defines a plurality of neurons interconnected by a plurality of synapses, the neurons being arranged in layers, each neuron in one of the layers being connected to one or more neurons in another one of the layers, and may comprise at least one of the foregoing.

[0029]

[0029] Furthermore, the system may include a processor configured to perform at least one of the following steps: dividing the neurons of the network definition file into one or more partitions, where each partition comprises a portion of the neurons from one of the layers; selecting one of the partitions; selecting one of the available planes of the resource model, where the number of neurons in the plane is greater than or equal to the number of neurons in the selected partition; mapping the neurons of the selected partition to the neurons of the selected plane of the resource model, where the neurons of the selected partition are mapped to the neurons of the selected plane when the neurons of the selected plane have available connections that match the interconnections of the neurons of the selected partition.

[0030]

[0030] Here, embodiments are described by way of example only with reference to the accompanying drawings, in which corresponding reference numerals indicate corresponding parts.

Brief Description of the Drawings

[0031] [Figure 1]

[0031] Generally shows the overall flow of a mapper that compiles an exemplary network for an exemplary processor. [Figure 2]

[0032] Generally shows the flow of operations in the mapper. [Figure 3]

[0033] Generally shows software for representing a hardware architecture in a general way. [Figure 4]

[0034] (E.g., greedy) Generally shows the high-level algorithm of the mapping algorithm.

Modes for Carrying Out the Invention

[0032]

[0035] Certain embodiments are described in further detail below. However, it should be recognized that these embodiments should not be construed as limiting the scope of protection under this disclosure.

[0033]

[0036] This section describes a system and method for mapping neural network-based applications to domain-specific AI accelerators, the mapping being readily adaptable to different neuromorphic chip architectures, and the ability to map different types of neural networks to a given chip architecture.

[0034]

[0037] Figure 1 schematically shows the overall flow 10 of the mapper that compiles the illustrative network 1 to the illustrative processor 5. The target processor, which is AI hardware, could be an AI hardware accelerator chip. In particular, the processor could be a spiking neural network processor. For example, the AI ​​hardware accelerator chip could be an AI hardware accelerator chip such as the spiking neural processor T1, produced by Innatera Nanosystems BV.

[0035]

[0038] In this example, the neural network is defined in a software development kit (SDK), and the mapper in the SDK generates a hardware configuration of the neural network that can be used as input to configure a spiking neural network processor or a simulator that uses this configuration to perform network inference. For example, the SDK could be the Talamo Software Development Kit produced by Innatera Nanosystems BV.

[0036]

[0039] SDK2 accepts Network Definition 1 as input. Network Definition 1 comprises a graph of the spiking neural network to be implemented in the target processor 5. Network Definition 1 may further comprise additional information about the neurons and synapses that make up the spiking neural network. Network Definition 1 may comprise a description of the overall structure of the SNN, including the number of neurons, their connectivity, and the type of connection between them (e.g., feedforward, regression). Furthermore, Network Definition 1 may comprise a description of the neuron model used in the SNN, including dynamics, spiking behavior, and parameters such as membrane potential threshold, refractory period, and synaptic weights. Different types of neurons (e.g., integral firing, leaky integral firing) may have different computational requirements and hardware implementations. Network Definition 1 may further comprise specifications of the synaptic dynamics and plasticity mechanisms used in the network, including parameters such as synaptic weights, time constants, and learning rules (e.g., spike timing-dependent plasticity). This information guides the implementation of synaptic connectivity and plasticity mechanisms in hardware. Furthermore, network definition 1 may optionally include a description of the input stimuli or sensory data that the network will process, including format, encoding scheme, and temporal characteristics. This helps determine how the sensory information is presented to the network and how it affects neuronal activity. Network definition 1 may further include specifications for desired operating modes of the network, such as inference, learning, or a combination of both. Different modes may require different computational resources and memory configurations in the hardware.

[0037]

[0040] SDK2 may optionally provide definitions of performance metrics or objectives that the mapping process should optimize, such as accuracy, throughput, energy efficiency, or latency. This helps guide the mapping algorithm in selecting the most appropriate hardware configuration and optimization strategy. These performance metrics or objectives may be specific to the spiking neural network or the target processor 5.

[0038]

[0041] Furthermore, SDK2 may provide information regarding the hardware constraints and characteristics of the target neuromorphic processor, such as available memory, processing elements, interoperability, and power consumption.

[0039]

[0042] SDK2 includes mapper 2A, which performs mapping of a spiking neural network onto a target processor. This mapping can be performed either offline or online.

[0040]

[0043] SDK2 outputs a set of instructions 3 for deploying a spiking neural network on the target processor 5. This set of instructions 3 could be, for example, a binary file used to configure the target processor 5. Optionally, the set of instructions 3 could be used to configure a hardware-aware simulator that simulates the target hardware.

[0041]

[0044] The configuration of the target processor can be performed in an illustrative manner by setting the synaptic weights connecting the current neuron standard to zero, and the set of instructions provides instructions for setting the synaptic weights for the synaptic elements connecting these two neurons to a specific non-zero value when the mapper determines that a connection between these two neurons is required.

[0042]

[0045] Figure 2 schematically shows the operation flow 20 in the mapper.

[0043]

[0046] The mapper plays a role in enabling mutual alignment between software and hardware, and in converting SNNs onto hardware. The flow of operation in the mapper is described in this embodiment. The present invention proposes two important components of a mapping system: (1) a representation of the hardware architecture in software, and (2) a (e.g., greedy) mapping algorithm.

[0044]

[0047] The configuration file 21, sometimes called the "arch config" file, can describe important parameters related to a specified hardware architecture. This information can be expressed in the form of several essential hardware properties, such as the number of neurons, synapses, segments, etc. This input can be configured by the user, and desired architecture properties can be entered into the configuration file 21.

[0045]

[0048] Next, by configuring the hardware resource model 22, the hardware resource model 23 is initialized and refined using the architecture configuration file as a single operation. The hardware resource model is a representation of the computational and / or memory resources available on the target hardware platform. This model provides detailed information about the characteristics and capabilities of the hardware architecture, which is essential for optimizing the mapping process. For example, as shown, multiple segments or cores 23D may exist in the target hardware, each having presynapses 23A (e.g., arranged in columns), synaptic elements 23B (e.g., arranged in synaptic matrices), and neurons 23C (e.g., arranged in rows).

[0046]

[0049] A hardware resource model typically includes several key components. Firstly, there may be a description of the computing elements available on the neuromorphic processor, such as processing cores, neuromorphic synapses, and dedicated accelerators. This may include details about the types of operations supported by each computing unit, their computational throughput, and any limitations or constraints on their use. Secondly, a specification of the hardware platform's memory hierarchy, including on-chip memory (e.g., registers, caches) and off-chip memory (e.g., DRAM), may be available. This includes details about memory size, access latency, and bandwidth limitations that affect data movement and storage during neural network execution. Thirdly, a description of the interconnection network or fabric connecting different computing units and memory elements within the hardware platform may be available. This includes details about communication bandwidth, latency, and routing algorithms that affect data exchange and synchronization between processing elements. Fourthly, information about the hardware platform's power consumption and energy efficiency characteristics may be available, including power budgets, thermal constraints, and / or energy consumption profiles for different operations. This may be important for optimizing mapping to minimize energy consumption and maintain thermal stability. Fifth, specifications for programming models and interfaces for interacting with the hardware platform may be available, including APIs, libraries, and development tools. This includes details on how the neural network model is represented, compiled, and executed on the hardware, as well as any constraints imposed by the programming environment.

[0047]

[0050] In particular, the hardware resource model may contain information about the neurons and synapses that make up the spiking neural network, or any other neurosynaptic scores present in the target processor.

[0048]

[0051] Therefore, the hardware resource model may include a description of the types of neurons supported by the neurosynaptic score, such as integral firing neurons, leak integral firing neurons, or more complex neuron models. Furthermore, it may include specifications for parameters associated with each neuron type, including membrane potential threshold, reset voltage, refractory period, and spike dynamics. Information about the total number of neurons supported by the core, and any limitations on the number of neurons that can be active simultaneously, may also exist. In addition, the hardware resource model may include details about how neuron dynamics are implemented in hardware, such as integration of input current, calculation of membrane potential, and generation of spikes.

[0049]

[0052] Regarding synapses in a spiking neural network, the hardware resource model can detail the types of synapses supported by the neurosynaptic score, including static, dynamic, and plastic synapses, using various learning rules. Furthermore, specifications for parameters associated with each synaptic type can be provided, such as synaptic weights, time constants for synaptic dynamics, and plasticity parameters. Information can be provided regarding the total number of synapses supported by the core and any limitations on the number of synapses that can be active simultaneously. Finally, details regarding how synapses are connected between neurons, including, for example, the architecture of the synaptic matrix or connectivity fabric, can be detailed by the hardware resource model.

[0050]

[0053] Details regarding the available memory for a spiking neural network can also be provided by the hardware resource model. For example, a description of a memory resource dedicated to storing neuronal state variables such as membrane potential and spike timing for each neuron in the core. Another example might be a specification of a memory resource dedicated to storing synaptic weights and other synaptic parameters, including the size and organization of the synaptic weight matrix. Memory access characteristics such as memory access latency, bandwidth, and information about access patterns for reading and writing neuron and synaptic state information may also be provided.

[0051]

[0054] Hardware resource models can also detail how interconnects and communications are arranged on the target hardware. For example, with respect to neuron-synaptic connectivity, details may be given regarding the interconnect network or routing fabric used to connect neurons and synapses within the core, including communication bandwidth and latency. Finally, inter-core communication information may also be detailed regarding communication interfaces for exchanging spike events or synaptic updates between multiple neurosynaptic scores in a multi-core system.

[0052]

[0055] Hardware resource models can also detail certain power and energy characteristics. For example, a description of the power consumption profile associated with neuronal and synaptic activity may be given, including idle power, dynamic power, and any power-saving mechanisms. Furthermore, metrics related to the energy efficiency of neuronal and synaptic activity may be given, such as energy per spike or energy per synaptic refresh.

[0053]

[0056] The mapping algorithm 26 takes as further input a neural network graph that needs to be implemented in the hardware corresponding to the hardware resource model 23. The network definition 24 can be used to obtain network partitions and / or network layers 25.

[0054]

[0057] The user configures a network definition (e.g., a network graph) to solve a specific problem. Such a network graph can be layer-based, and the layers can be defined by the user. These layers can be divided into partitions (subnetworks). The partitions must be mapped to planes on a hardware chip. This mapping generally involves costs, such as power consumption and area usage on the chip.

[0055]

[0058] After the hardware resource model 23 is initialized and configured based on the configuration file 21 22, the mapping algorithm 26 iterates through various layers or partitions 25 of a given neural network graph 24 and queries 27 the availability of hardware resources and the effectiveness of connectivity between resources using the hardware resource model 23.

[0056]

[0059] In this process, the mapping algorithm 26 consumes certain resources to "map" the network layer or partition onto some part of the hardware architecture only if the selected resources can satisfy the requirements of the network graph partition. In that case, the resource model is updated based on the selection 28. If the requirements cannot be met, the mapping algorithm 26 continues to search for other hardware resources until the requirements can be met and the network graph partition can successfully maintain its structure using the selected hardware resources.

[0057]

[0060] The mapping algorithm 28 terminates when it finds a suitable hardware configuration that can maintain the structure of a given neural network graph and perform network inference.

[0058]

[0061] A hardware resource model is an architectural representation of the target hardware in mapper software. This hardware representation in mapper software is crucial for software scalability and maintainability, and can be adaptable to diverse hardware architectures. It also represents modularity, allowing for the addition of hardware features to the hardware representation without modifying the mapping algorithm. Hardware architecture can be understood as a set of hardware resources and the connectivity between those resources.

[0059]

[0062] Figure 3 schematically illustrates software for representing a hardware architecture in a general way. The main purpose of this software component is to represent the hardware architecture in such a way that different architectures can be explored by the user. This means that the software can create different types of resource models, assuming any architecture, albeit with some constraints.

[0060]

[0063] First, the basic assumptions considered in the illustrative resource model are discussed below.

[0061]

[0064] In illustrative neuromorphic architectures, a fundamental commonality (1) may be that neurons connect to other neurons via synapses that reside on “presynapses,” which are essentially wire connections. These wire connections, i.e., presynapses, can hold multiple synapses that subsequently connect to the neuron as destinations. Another fundamental commonality (2) may be that interneuronal connections are mediated through specific presynapses. Within the constraints of the hardware architecture, the correct presynapses must be selected to enable the connection of one particular neuron to another. This property assumes certain information from the hardware designer regarding the architecture of the target hardware. Another assumption (3) may be that all neurons, synapses, and presynapses belong to a particular “segment” or core that houses these local resources. Any neuron that needs to communicate with another neuron in another segment must go through an “interconnection” that behaves like a bus connecting the two segments. This interconnection must be outside the segments. Alternatively, the output of a neuron in the first segment is received and mapped to a specific presynapse in the second segment, and this presynapse is programmed to connect to a second neuron, also present in the second segment, via a specific synapse. Thus, the output is mapped. Another assumption (4) is that a neuromorphic array in a neuromorphic architecture may hold one or more segments. Another important underlying commonality (5) may be the concept of a “plane.” A set of synapses, neurons, and presynapses may reside in a particular “plane,” i.e., a section limited to a particular “segment.” A “plane” can be defined as a particular layer or subset of neurons in a network. SNNs often consist of multiple layers of neurons with connections (synapses) between them. Each layer can be thought of as a plane of neurons, with connections between planes representing synaptic connections.Each layer of neurons in an SNN can occupy a specific location within the network architecture. The location of a plane can be defined by its depth within the network, with input layers typically located at the bottom and output layers at the top. Within each layer, neurons are typically arranged in a two-dimensional grid or array. This arrangement can define the plane of neurons within that layer. Each layer of neurons in an SNN can perform a specific functional role, such as processing inputs, extracting features, or generating outputs. The boundaries of a plane can be defined by its specific functional role within the network. Another assumption (6) may be that an input channel receiving spikes from the outside can utilize multiple presynapses to connect to other neurons. The number of presynapses for a single input channel may be predetermined information from the hardware designer. Another assumption (7) may be that there can always be presynapses originating from a neuron to route spikes to other neurons. These presynapses can be classified into two illustrative types: "interconnection" and "loopback". Loopback presynapse 37A is assumed to route spikes to other neurons located in the same segment as the originating neuron. Interconnection presynapse 37B is assumed to route spikes to other neurons located in other segments outside the originating neuron's segment.

[0062]

[0065] The resource model may be initialized using one or more of the above assumptions. The resource model is populated with the assumptions using an architecture configuration file.

[0063]

[0066] In this embodiment, the hardware resource model 30 comprises a segment array container 31, which is a container comprising an array of segments. The segment array container 31 can represent the entire target hardware on which the resource model is created, and it is an abstraction of the target hardware. In other words, the segment array container 31 comprises one or more segment containers 32 (satisfying assumption 4 given above). The segment array container 31 may contain further information including the number of segments, the hardware chip name, and a list of segments present in the hardware.

[0064]

[0067] The segment container 32 is a container comprising an array of segment plane containers 33. The segment plane containers 33 represent "planes" within a segment (satisfying assumption 5 given above). The number of planes in a segment is determined via the architectural configuration. The segment container 32 may also contain further information 32A, including segment identification information, the number of segment planes, the number of neurons, the number of presynapses, the number of synapses, the number of presynapses per plane, segment coordinates, and / or the number of neurons per plane.

[0065]

[0068] The segment plane container 33 may include segment plane identification information and further information 33A including neurons, presynapses, and synapses provided within the plane represented by the segment plane container 33 (the latter three are represented by the neuron container 34, the presynapse container 35, and the synapse container 36, respectively).

[0066]

[0069] The neuron container 34 may contain information 34A regarding neuron identification information, neuron location, i.e., where the neuron is located, source synapses 37C connected to the neuron, loopback presynapses 37A connected to the neuron, interconnection presynapses 37B connected to the neuron, neuron threshold, neuron capacitance, neuron switch configuration, neuron refractory period, and / or associated segment identification information. Neuron identification information may be parameters that assign a unique identifier to each neuron in the network. This enables the identification and tracking of individual neurons in the network. Neuron location may specify the spatial coordinates or location of the neuron in the network. It may include information regarding the neuron's location relative to other neurons or within a particular segment or core. Source synapses 37C connected to the neuron may be synaptic connections originating from other neurons and targeting the neuron in question. These synapses transmit signals (spikes) from the presynaptic neuron to the postsynaptic neuron. Loopback presynapses 37A connected to the neuron are connections that project from the neuron itself back onto itself. These connections may contribute to regression or feedback dynamics in the network. The interconnection presynapses 37B connected to a neuron can refer to connections from other segments or neurons in the core that target that neuron. These connections enable communication between different parts of the network. The threshold is a parameter that determines the level of membrane potential that a neuron must reach to generate an action potential (spike). When the membrane potential exceeds this threshold, the neuron fires. Capacitance refers to the ability of a neuron's membrane to store charge. It affects how the neuron's membrane potential changes in response to synaptic input and determines the neuron's response dynamics. Neuron switch configuration can describe whether a neuron is currently active or inactive. In some cases, neurons can be dynamically switched on or off based on network activity or specific conditions.The refractory period of a neuron is the time period after it fires an action potential during which it is temporarily unable to generate another action potential. This prevents the neuron from firing too rapidly and regulates the timing of spikes. Finally, the relevant segment identification information may specify the segment or core to which the neuron belongs (i.e., the segment identification information of the segment container 32).

[0067]

[0070] A neuron can therefore have a set of newly formed loopback presynapses 37A and a set of interconnection presynapses 37B (satisfying assumptions 7 and 3, respectively). A specific number of loopback and interconnection presynapses 37A and 37B for a neuron are configured via an architectural configuration.

[0068]

[0071] The presynaptic container 35 may contain information 35A about multiple destination synapses 38B associated with the presynaptic container 35 (satisfying Assumption 1). The number of synapses that the presynaptic container 35 can hold may be determined using information from the input architecture configuration. Since a presynaptic functions as a "wire" connecting pathways between neurons, it should have a source neuron 38A and at least one destination synapse 39B (the latter connected to a specific neuron) (satisfying Assumption 2). The number of neurons it connects to may be determined using information from the architecture configuration.

[0069]

[0072] The presynapse container 35 may include information 35A such as presynapse identification information, presynapse location (i.e., where the presynapse is located), destination synapse 38B connected to the presynapse, source neuron 38A connected to the presynapse, and associated segment identification information. The presynapse identification information may assign a unique identifier to each presynapse in the network. This enables the identification and tracking of individual presynapses and their properties. Similar to the location of neurons, the location of presynapses may specify the spatial coordinates or location of presynapses in the network. It may include information about the location of presynapses relative to other elements such as neurons or segments. The destination synapse 38B connected to the presynapse may be a synapse that originates from the presynapse and targets a specific postsynaptic neuron. These synapses transmit signals (spikes) from the presynaptic neuron to the postsynaptic neuron. The source neuron 38A connected to the presynapse may specify the neuron from which the presynapse receives input. It identifies the source of the signal that the presynapse transmits to the postsynaptic neuron. The associated segment identification information, similar to the associated segment identification information for neurons, may specify the segment or core to which the presynapse belongs.

[0070]

[0073] A synapse has a source presynapse 39A and a destination neuron 39B, as described earlier, because the spike travels directly from the synapse to the neuron. The corresponding synapse container 36 may also contain information 36A about these structures. The synapse container may contain, for example, row and column identifiers of the synapse matrix in which the represented synapse is contained (or more generally, information about the location of the synapse in hardware), the location of the synapse (i.e., where the synapse is positioned), the weight of the synapse, the source presynapse connected to the synapse, the destination neuron 39B connected to the synapse, and / or related segment identifiers 36A. The row and column identifiers may specify the location of the synapse in hardware, often referring to its position in the synapse matrix. The row identifier indicates the row in which the synapse is located, while the column identifier indicates the column. This information is useful for addressing and accessing synapses in the network architecture. The location of the synapse, as well as the locations of the neuron and presynapse, may specify the spatial coordinates or location of the synapse in the network. It may include information about the location of the synapse relative to other elements such as neurons or presynapses. The synaptic weight may represent the strength or effectiveness of the connection between the presynaptic neuron and the postsynaptic neuron. It determines the effect of presynaptic activity on the membrane potential of the postsynaptic neuron and is important for synaptic plasticity and learning within the network. The source presynapse 39A connected to the synapse specifies the presynaptic neuron from which the synapse receives input. It identifies the source of the signal transmitted by the synapse to the postsynaptic neuron. The destination neuron 39B connected to the synapse may specify the postsynaptic neuron targeted by the synapse. It identifies the neuron that receives the signal transmitted by the synapse originating from the presynaptic neuron. As with neurons and presynapses, the associated segment identification information may specify the segment or core to which the synapse belongs.

[0071]

[0074] As this example shows, interconnections are very simple and are simple one-to-one connections using wires (presynapses) between neurons, so they do not need to be modeled separately.

[0072]

[0075] The specific assumption 6 may be embedded internally during legitimate connectivity checks between resources, which are part of the “queryResourceModel” API module, and the API module 31B resides, for example, within the segment array container 31. This hardware assumption is called “multicasting”. The query API module queries the resource model for a given partition, layer, or entire network supplied as an argument to the query, checks whether the argument can be mapped onto the resource model, and returns components of the resource model that can be used to map the given argument. These components could be, for example, neurons, synapses, presynapses, planes, segments, and / or entire chips. It is assumed that input spikes arriving from the outside can be copied onto a set of presynapses that will transport these spikes into a range of synapses. “updateResourceModel” is an API module that reserves resources requested by the mapping algorithm as consumed. It is a module that interacts between the mapping algorithm and the resource model. Its function is to receive information about the resources to be reserved, and the module reserves those resources. The "buildResourceModel" API module may also be provided within the segment array container 31 and can be used to build a resource model by creating a hardware graph based on hardware constraints and the number of components (neurons, synapses, presynapses, planes, segments, etc.). Another function may be "revertResourceModel," which is an API module that can revert the resource model to a previous state if a particular mapping attempt fails.

[0073]

[0076] Figure 4 schematically shows a high-level algorithm for (e.g., a greedy) mapping algorithm.

[0074]

[0077] Before the algorithm, it is assumed that the resource model is initialized with an architectural configuration and that the neural network is appropriately parsed into a formal graph representation. Parsing a neural network into a formal graph representation involves representing the network's components (neurons, synapses, etc.) as nodes and the connections between them as edges in the graph. An illustrative method is given below.

[0075]

[0078] As the first step, the components of a neural network, such as neurons, synapses, presynapses, postsynapses, segments, and any other related elements, should be identified and represented in the graph. Next, a node should be created for each component identified in step 1. A unique identifier should be assigned to each node to distinguish them from one another. The nodes could be, for example, neurons A, B, etc. Next, edges need to be defined by determining the connections between network components and representing them as edges in the graph. For example, if neuron A is connected to neuron B via a synapse, an edge is created between the nodes representing neurons A and B. Optionally, attributes may be assigned to the edges to represent properties such as synapse weights, connection type, or any other related information. The graph is then constructed based on the nodes and edges defined in the previous step, using a graph data structure or graph representation library (e.g., NetworkX in Python).

[0076]

[0079] An illustrative graph is shown in the embodiment, along with layers 0-1, indicated by reference no. 119, and layers 1-2, indicated by reference no. 120.

[0077]

[0080] As an optional precursor step before the mapping algorithm 100 is used, for a given neural network, the mapper 100 may use a "network analyzer" that essentially parses different properties of the neural network graph and checks them against the connectivity properties of the hardware architecture. If there are any properties of the network graph that are incompatible with the properties of the hardware architecture, the user cannot proceed further, and the mapping process declares the network as incompatible for performing network inference on the hardware.

[0078]

[0081] The mapping algorithm 100 begins by forming graph partitions in step 101. If the user requests partitioning according to network layers, the algorithm creates partitions by layer. The next step 102 is to sort the partitions. Sorting the partitions implies finding the order in which the partitions are selected to map onto the hardware configuration. In step 103, a particular partition P is mapped onto the hardware. In step 104, the partition is divided into fragments. Thus, for a given partition P, the algorithm first decomposes them into fragments. Each fragment consists of one neuron with all of its arriving edges (also known as synapses). An illustrative fragment (indicated by a dashed triangle) is shown in reference no. 121.

[0079]

[0082] For each fragment, the mapping algorithm can find the source nodes for these edges. If the source nodes are already successfully mapped, the algorithm finds a valid path so that the aforementioned incoming edges are mapped to the appropriate synapses. If no valid path is found, the loop iterates by requesting different components from the hardware resource model. If the source nodes are already successfully mapped and no valid path is found to connect to the neuron, the loop terminates and returns to finding a partition that has no mapped source nodes.

[0080]

[0083] For example, in step 105, a valid component to be placed is found. A Boolean constraint check 105A may be performed using the history 106 of visited nodes or components. The algorithm remembers the history 106 of visited resources so that it does not search for resources that have already been traversed and checked for valid connectivity. In step 107, it is checked whether there are enough valid components for a placement to occur. If the answer is no, the algorithm returns to finding a valid component to be placed; if the answer is yes, the mapping algorithm accumulates components for all fragments in step 108. Based on the accumulated components for all fragments of the partition from step 108, the resource model is built and updated in step 110A based on the architecture configuration 109 and / or calibration data 110.

[0081]

[0084] The role of "calibration data" can arise from the need to maintain the accuracy of neural network inference on hardware. If a neural network is run using faulty or non-functional resources, the accuracy of the inference can be dramatically reduced. For this reason, it may be important in some cases for the mapper to receive information about resources that should not be considered during the mapping process. The resource model may, for example, already reserve these resources that should not be considered to be in use, preventing the algorithm from using them to add to the mapping configuration. In this way, for example, those non-functional resources are not used to perform network inference.

[0082]

[0085] Thus, hardware resources are accumulated for each fragment in step 108, and all resources for the entire partition are gathered. Next, in step 112, the algorithm queries the resource model 111 with these accumulated resources to check whether these requested resources can be mapped by saving the graph partition structure.

[0083]

[0086] This prompts a query to determine if the mapping was successful. If the answer is yes, the resource model is updated in step 112A and moved to the next partition in step 112B. The resource model update can be performed using the updateResource API module, and the mapping configuration is populated accordingly.

[0084]

[0087] If the partition mapping is successful in this way, it moves to the subsequent partitions in the given sorted order in step 112B. If it was possible to successfully map all partitions in step 112D, it generates a mapping configuration 114 and terminates. The generated mapping configuration 114 can be used to generate a mapped JSON file which can be used to set up and configure the hardware to correspond to the mappings found by the mapping algorithm 100.

[0085]

[0088] Mapping model 113 is an abstraction model of the mapping output on the hardware resource model by a network-defined mapping algorithm.

[0086]

[0089] If the loop fails to attempt to map the partition over a certain number of tries (e.g., 100 to 1000 tries, preferably 100 to 500 tries, more preferably 100 to 200 tries) in step 116 by returning to the first partition 117 in step 102 (e.g., by trying in a different order to map the partition, or by performing a new partitioning of the neural network), the algorithm terminates in step 118 and declares that the neural network cannot be mapped to the hardware. The algorithm analyzes the results of querying the resource model and concludes whether the request for those resources to map partitions was successful.

[0087]

[0090] It should be noted that any of the features of the embodiments disclosed herein can be appropriately combined.

Claims

1. A method for mapping a spiking neural network design onto a configurable neuromorphic processor, wherein the method is The resource model defines a resource model that describes the neuromorphic processor, wherein the resource model defines one or more planes, each plane comprising a plurality of presynapses, synapses, and spiking neurons arranged in a subnetwork, each presynapse being adapted to receive one or more inputs, each neuron having an output, each synapse connecting one of the presynapses to one of the neurons, and the resource model defines a plurality of interconnections for connecting the output of one of the neurons in one of the planes to one or more of the presynapses in one of the planes. Define a network definition file that describes the spiking neural network design, wherein the network definition file defines multiple neurons interconnected by multiple synapses, the neurons are arranged in layers, and each neuron in one of the layers is connected to one or more neurons in the same layer or in another of the layers. The method further comprises, The neurons in the network definition file are divided into one or more partitions, where each partition comprises a portion of the neurons from one of the layers. Selecting one of the aforementioned partitions, Selecting one of the available planes of the resource model, wherein the number of neurons in the plane is greater than or equal to the number of neurons in the selected partition. Mapping the neurons of the selected partition to the neurons of the selected plane in the resource model, wherein the neurons of the selected partition are mapped to the neurons of the selected plane if the neurons of the selected plane have available interconnections that match the interconnections to the neurons of the selected partition. A method that includes [a certain feature].

2. The method according to claim 1, further comprising selecting one of the available planes such that the number of inputs to the neurons of the plane is equal to or greater than the number of inputs to the neurons of the selected partition.

3. The method of claim 2, further comprising selecting one of the available planes such that the number of external inputs to the neurons of the plane is equal to or greater than the number of external inputs to the neurons of the selected partition, and the number of regressive inputs to the neurons of the plane is equal to or greater than the number of regressive inputs to the neurons of the selected partition.

4. The method according to any one of claims 1 to 3, wherein mapping the neurons of the selected partition comprises assigning one of the neurons of the selected plane to one of the neurons of the selected partition, and assigning the interconnection to the assigned neuron of the selected plane if the neurons of the selected partition have a corresponding interconnection.

5. The method according to claim 4, wherein if an interconnection to the selected plane corresponding to the interconnection to the selected partition to the neuron cannot be found or has already been assigned, a different neuron from the selected plane is assigned to the neuron in the selected partition.

6. The method according to claim 4 or 5, wherein assigning interconnections to the assigned neurons of the selected plane comprises assigning an external input to the assigned neurons of the selected plane when the neurons of the selected partition have an external input, and assigning a regression input to the assigned neurons of the selected plane when the neurons of the selected partition have a regression input.

7. The method according to any one of claims 4 to 6, wherein assigning interconnections to the assigned neurons in the selected plane comprises assigning connections via interconnections to the assigned neurons when the neurons in the selected partition have inputs from neurons in different partitions.

8. The method according to any one of claims 4 to 7, wherein assigning interconnections to the neurons assigned to the selected plane comprises removing synaptic input to the neurons assigned to the selected plane when the neurons in the selected partition do not have corresponding synaptic input.

9. The method according to any one of claims 1 to 8, further comprising defining a set of mapping constraints that describe potentially conflicting interconnections in the resource model.

10. The method according to claim 9, wherein the set of mapping constraints includes indications of the quality and type of interconnections to neurons in the partition of the network definition file.

11. The method according to claim 9 or 10, wherein the set of mapping constraints includes an indication of the quality of interconnections between neurons in different partitions of the network definition file.

12. The method according to any one of claims 1 to 11, wherein if the neurons of the selected partition are successfully mapped to the neurons of the selected plane, one of the planes of the resource model is shown as unavailable.

13. The method according to any one of claims 1 to 12, wherein if the neurons in the selected partition are not successfully mapped to the neurons in the previously selected plane, another plane of the resource model is selected.

14. The method according to any one of claims 1 to 13, further comprising merging a partition of the network definition file with another partition of the network definition file when the number of partitions exceeds the number of planes in the resource model.

15. A system for mapping a spiking neural network design onto a configurable neuromorphic processor, wherein the system is A first memory for storing a resource model describing the neuromorphic processor, wherein the resource model defines one or more planes, each plane comprising a plurality of presynapses, synapses, and spiking neurons arranged in a subnetwork, each presynapse adapted to receive one or more inputs, each neuron having an output, each synapse connecting one of the presynapses to one of the neurons, and the resource model defines a plurality of interconnections for connecting the output of one of the neurons in one of the planes to one or more of the presynapses in one of the planes. A second memory for storing a network definition file describing the spiking neural network design, wherein the network definition file defines a plurality of neurons interconnected by a plurality of synapses, the neurons are arranged in layers, and each neuron in one of the layers is connected to one or more neurons in another of the layers. Processor and The processor is equipped with, The neurons in the network definition file are divided into one or more partitions, where each partition comprises a portion of the neurons from one of the layers. Selecting one of the aforementioned partitions, Selecting one of the available planes of the resource model, wherein the number of neurons in the plane is greater than or equal to the number of neurons in the selected partition. Mapping the neurons of the selected partition to the neurons of the selected plane in the resource model, wherein the neurons of the selected partition are mapped to the neurons of the selected plane if the neurons of the selected plane have available interconnections that match the interconnections to the neurons of the selected partition. A system configured to perform the following actions.

Images