Parallel processing in spiking neural networks

Abstract

This disclosure relates to parallel processing in spiking neural networks. The disclosed embodiments relate to storing critical data in a memory device, such as a flash or DRAM memory device. In one embodiment, an apparatus comprising a plurality of parallel processors configured to: perform a search and match operation that loads a plurality of synaptic identifier bit strings and a plurality of spiking identifier bit strings, the search and match operation further generating a plurality of bit masks; perform a synaptic integration phase that generates a plurality of synaptic current vectors based on the plurality of bit masks, the synaptic current vectors being associated with corresponding synthetic neurons; solve a neural membrane equation for each of the synthetic neurons; and update the membrane potential associated with the synthetic neuron, the membrane potential being stored in the memory device.

Description

Technical Field

[0001] At least some of the embodiments disclosed herein generally relate to neural networks, and in particular, to spiking neural networks. Background Technology

[0002] Spiking neural networks (SNNs) are mathematical models of biological neural networks (BNNs). A BNN consists of interconnected neurons that communicate with each other using spiking. Neurons generate pulses based on inputs from other neurons connected to them. Neuron-to-neuron connections are called synapses, and their strength varies. The inflowing pulse contributes differently to the generated (i.e., postsynaptic) pulse depending on the strength or weight of the corresponding synapse.

[0003] BNNs process information using impulses traveling from neuron to neuron. BNNs learn by adding new synaptic connections, removing synaptic connections, changing the strength of synaptic connections, or altering the delay (e.g., conductivity) in synaptic connections. For example, an individual learning to play a new musical instrument might change the synaptic connections associated with motor skills over time.

[0004] SNNs simulate BNNs by mimicking neurons, synapses, and other elements of BNNs, and by introducing spiking into the mathematical neural network. SNNs can be decoded to execute on multiple processors to simulate spiking emitted within the neural network. A fruit fly has approximately 250,000 neurons, each with about 80 synapses, while the human brain has approximately 86 billion neurons, each with about 1,700 synapses. Therefore, the computational resources required for rapidly processing spiking are considerable, making the scaling of SNNs challenging. Summary of the Invention

[0005] In one aspect, this disclosure is directed to an apparatus comprising: a plurality of parallel processors configured to: perform a search and matching operation that loads a plurality of synaptic identifier bit strings and a plurality of pulse identifier bit strings, the search and matching operation further generating a plurality of bit masks; perform a synaptic integration phase that generates a plurality of synaptic current vectors based on the plurality of bit masks, the synaptic current vectors being associated with corresponding synthetic neurons; solve a neural membrane equation for each of the synthetic neurons; update one or more state variables associated with the synthetic neurons; and write data values ​​indicating the one or more state variables to a memory device.

[0006] In another aspect, this disclosure relates to a method comprising: performing a search and matching operation via a parallel processor, the search and matching operation being the loading of a plurality of synaptic identifier bit strings and a plurality of pulse identifier bit strings, the search and matching operation further generating a plurality of bit masks; performing a synaptic integration phase via the parallel processor, the synaptic integration phase generating a plurality of synaptic current vectors based on the plurality of bit masks, the synaptic current vectors being associated with corresponding synthetic neurons; solving a neural membrane equation for each of the synthetic neurons via the parallel processor; and updating a state variable associated with the synthetic neuron via the parallel processor, the state variable being stored in a memory device.

[0007] In another aspect, this disclosure is directed to a non-transitory computer-readable storage medium for tangibly storing computer program instructions executable by a parallel processor, the computer program instructions defining the following steps: performing a search and matching operation that loads a plurality of synaptic identifier bit strings and a plurality of pulse identifier bit strings, the search and matching operation further generating a plurality of bit masks; performing a synaptic integration phase by the parallel processor that generates a plurality of synaptic current vectors based on the plurality of bit masks, the synaptic current vectors being associated with corresponding synthetic neurons; solving a neural membrane equation for each of the synthetic neurons by the parallel processor; and updating a state variable associated with the synthetic neuron by the parallel processor, the state variable being stored in a memory device. Attached Figure Description

[0008] Many aspects of this disclosure can be better understood with reference to the accompanying drawings. Components in the drawings are not necessarily drawn to scale, but the emphasis is on clearly illustrating the principles of this disclosure. Furthermore, similar reference numerals are used in the drawings to denote corresponding parts in several views.

[0009] Figure 1 These are examples illustrating the architecture of an SNN system according to some embodiments of the present disclosure.

[0010] Figure 2 This is a block diagram illustrating pulse messages transmitted within an SNN system according to some embodiments of the present disclosure.

[0011] Figure 3 This is a block diagram illustrating nodes of a mesh architecture coupled to an SNN system according to some embodiments of the present disclosure.

[0012] Figure 4 This is a block diagram illustrating the memory of a node within an SNN system according to some embodiments of the present disclosure.

[0013] Figure 5This is a block diagram illustrating the functionality and structure of nodes within an SNN system according to some embodiments of the present disclosure.

[0014] Figure 6 This is a block diagram illustrating parallel processing via nodes within an SNN system according to some embodiments of the present disclosure.

[0015] Figure 7A This is a flowchart illustrating a method for performing pulse delivery in a SIMD or MIMD pipeline according to some embodiments of the present disclosure.

[0016] Figure 7B This is a pipeline diagram illustrating a method for performing pulse delivery in a SIMD or MIMD pipeline according to some embodiments of the present disclosure.

[0017] Figure 7C This is a flowchart illustrating a method for performing synaptic integration in a SIMD or MIMD pipeline according to some embodiments of the present disclosure.

[0018] Figure 7D This is a pipeline diagram illustrating a method for performing synaptic integration in a SIMD or MIMD pipeline according to some embodiments of the present disclosure.

[0019] Figure 7E This is a flowchart illustrating a method for performing neuronal dynamics in a SIMD or MIMD pipeline according to some embodiments of the present disclosure.

[0020] Figure 7F This is a pipeline diagram illustrating a method for performing neuronal dynamics in a SIMD or MIMD pipeline according to some embodiments of the present disclosure.

[0021] Figure 8 Examples of networking systems that include an SNN system as a component, according to various embodiments, are described. Detailed Implementation

[0022] This disclosure relates to processing and memory architectures for implementing SNNs. According to embodiments, the memory architecture uses dedicated memory devices configured as “nodes.” A node represents, for example, a group of neurons (e.g., one or more). Nodes can be coupled together via digital fiber to support a large number of neurons, thereby supporting high scalability. This disclosure also relates to SIMD or MIMD pipelines for implementing SNN functionality, including spiking, synaptic integration, and neuronal dynamics. While SIMD implementations are described, MIMD pipelines can be used instead of SIMD pipelines.

[0023] Figure 1 These are examples illustrating the architecture of an SNN system according to some embodiments of the present disclosure.

[0024] The SNN architecture comprises multiple nodes 100. In one embodiment, a node 100 may include a memory device that performs in-memory processing (also referred to as in-memory processing / processing-in-memory, or PIM) to implement the SNN. For in-memory processing, the SNN architecture provides a scalable system that uses computer architecture techniques and constructs nodes 100 to provide SNN functionality. Nodes 100 may include special-purpose memory devices implemented as integrated circuits (ICs). Nodes 100 may be semiconductor chips, dies, or stacks of dies.

[0025] Node 100 may include one or more memory arrays 103. Memory array 103 includes multiple rows and columns and may be defined in terms of row-column size. Figure 1 An example shows memory array 103 having labels r1 to r n The rows and labels are c1 to c n The data array is a series of columns. At each intersection of rows and columns are memory cells configured to store values. For example, the data array may contain four sequentially ordered elements A, B, C, and D. The data array may be stored in memory array 103 such that each element of the data array is stored in a corresponding memory cell. For example, element A may be stored in cell (r1, c1), element B in cell (r1, c2), element C in cell (r1, c3), and element D in cell (r1, c4). Therefore, in this example, the data array is stored along the first row and occupies the first four columns. This is called a “bit-parallel” configuration. As another example, the data array may be stored along the first column and occupy the first four rows. Here, element A may be stored in cell (r1, c1), element B in cell (r2, c1), element C in cell (r3, c1), and element D in cell (r4, c1). This is called a “bit-serial” configuration. Each element (A, B, C, or D) can be a binary number (e.g., zero or one, or high and low values), a discrete value (e.g., a quantized value, a finite number, an integer), or an analog value (e.g., a continuous number, an irrational number). Therefore, memory array 103 is a hardware component used to store data as multiple array elements that can be addressed by rows and columns.

[0026] Besides purely bit-parallel and purely bit-serial storage of data arrays, hybrid storage methods are also possible. Continuing the previous example, elements A and B of the data array can be stored in the first row, and elements C and D can be stored in the second row, such that A and C are stored in the first column, but C and D are stored in the second column. Therefore, A and B are row-aligned, and C is row-aligned with D. However, A and C are column-aligned, and B and D are column-aligned. Furthermore, A and C do not need to be row-adjacent, and B and D do not need to be row-adjacent. Moreover, A and C do not need to be column-adjacent, and B and D do not need to be column-adjacent. Therefore, various embodiments encompass combinations of bit-serial and bit-parallel arrangements.

[0027] According to embodiments, node 100 may include one or more DRAM arrays that digitally store data. In other embodiments, node 100 may include resistive random access memory (ReRAM), 3D crosspoint (3DXP), or other memory devices that implement resistive memory cells or provide memory cells that can bend or modulate their conductance. Such cells may be diodes, transistors with floating gain, and alternative gate transistors, etc. For example, a resistive memory cell stores data by modulating the resistance of the memory cell according to the data it stores. If the resistive memory cell stores binary zero (“0”), then the resistance may be set to a low value so that the memory cell forms a short circuit (e.g., resistor short circuit). If the memory cell stores binary one (“1”), the resistance may be set to a high value so that the memory cell forms an open circuit (e.g., resistor open circuit). The resistance may also be set to a medium resistance to store discrete values ​​(e.g., quantized values). The resistance may also be set within the range of resistances used to store analog values. The memory cell may also include asymmetric elements, such as diodes, in which current flows in one direction but is impeded in the opposite direction. Other asymmetric elements that can serve as memory cells include, for example, transistors and magnetic tunnel junctions (MTJs).

[0028] Node 100 may include controller 109, input filter 112, output filter 115, local bus 118, network interface 121, and other possible integrated components. Controller 109 may be a dedicated processor that implements the logic executed by node 100. Controller 109 may include an IC dedicated to storing data in memory array 103 by organizing data according to different patterns. Controller 109 may include fast memory elements, such as registers, static random access memory (SRAM) arrays, or caches that store transient data for fast access.

[0029] In some embodiments, controller 109 may be implemented as a separate device coupled to node 100. For example, controller 109 may be implemented in an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other special-purpose processor. The controller may therefore be part of a host device coupled to node 100. In some embodiments, as described in more detail herein, controller 109 may include a SIMD, MIMD, or other vector processor.

[0030] Additionally, controller 109 may receive input data, store input data, access input data, read data stored in the memory array, perform pattern matching operations to determine whether the input data matches a pattern stored in a memory device node, and perform other memory operations (e.g., in-memory operations) to implement portions of the SNN. Controller 109 may include microcode that controls the order in which word lines and bit lines are activated. Word lines and bit lines are activated by applying voltage or supplying current to selected word lines and bit lines. This may be referred to as an activation signal. In some embodiments, controller 109 may adjust the strength of the activation signal by changing the voltage or current depending on the application.

[0031] Communication with nodes is achieved using messages called impulse messages. Impulse messages are modeled after electro / chemical signals in a BNN. For example, in a BNN, neurons generate impulses based on other impulses received from inputs to connected neurons. Neuron-to-neuron connections are called synapses, which vary in strength, polarity (excitatory vs. inhibitory), and many other neuroscience aspects (e.g., N-methyl-d-aspartic acid or N-methyl-d-aspartate (NMDA) type, ion channels, and receptor composition, neurotransmitter orientation, etc.). Therefore, the inflow of impulses depends on their synaptic strength (alternatively referred to as weights in this paper), while postsynaptic impulses contribute differently. When modeling a BNN in a computer-implemented SNN, the weights of each synapse can be dynamically adjusted according to various learning rules. Typically, these rules can be based on impulse timing, for example, the timing of the inflow of an impulse being before or after the timing of the generated impulse.

[0032] In BNN, a pulse arriving at the synapse of a neuron (postsynaptic neuron) from another neuron (presynaptic neuron) triggers the release of neurotransmitters in the small gap between the axon and the synapse (called the synaptic cleft). The neurotransmitter binds to receptors (or ion channels) on the postsynaptic neuron. These receptors open “holes” in the neuron’s body in an explosive chain reaction (one receptor triggers the opening of another), thus causing an electrical injection. A small amount of neurotransmitter is sufficient to trigger this chain reaction. Therefore, the arrival of a pulse is normally estimated as a binary all-or-nothing event, and the synaptic strength is proportional to the number of these receptors. Figure 1In the SNN architecture, node 100 processes incoming pulse messages and generates outgoing pulse messages, where each pulse message participates in the electrical and chemical signaling between neurons in the simulated BNN.

[0033] Each node 100 is modeled to represent a cluster of neurons. Terms such as "neuron," "spiking," or "synapse" refer to the biological component in the BNN and the computer-implemented component modeled following its corresponding biological component. A single node 100 can receive spiking messages directed to one or more neurons within the cluster represented by that single node 100. The SNN architecture can address specific neurons contained within node 100 using neuron identifiers. Additionally, the SNN architecture can store synaptic connection IDs to represent synaptic connections between two neurons. Because neurons can jointly connect to several other neurons, there will be significantly more (often substantially more) unique synaptic connection identifiers compared to neuron identifiers.

[0034] Node 100 can generate outgoing pulse messages through the neurons contained within it. Node 100 may include an input filter 112 for processing incoming pulse messages and an output filter 115 for processing outgoing pulse messages.

[0035] By including an input filter 112 for a given node 100, node 100 can filter in incoming pulse messages directed to target neurons within node 100. An output filter 115 can filter out generated pulse messages with target neurons in other nodes 100. Pulse messages generated within node 100 only for neurons within node 100 can remain within node 100. The emission of pulse messages across multiple nodes 100 can behave similarly to selective broadcasting or multicasting targeting a series of neurons and passing them across synapses of one or more nodes 100. Neurons can be addressed (e.g., targeted) by pulse messages using synaptic connection identifiers that associate source neuron IDs with target neuron or synapse IDs.

[0036] The filtering function of input filter 112 or output filter 115 may involve a matching operation performed on a subset of synaptic connections addressable by synaptic connection identifiers (IDs), which link source neurons to target neurons via specific synapses. Such synaptic connection identifiers may be or include source neuron IDs. Source neuron IDs may be part of a spike message descriptor. An addressing scheme assigned via a predetermined algorithm can be used to accelerate the filtering operation performed by input filter 112 or output filter 115. For example, neurons may be assigned such that the node identifier of node 100 matches a subset of positions in the source neuron IDs. A combination of input filter 112 (or output filter 115) and an addressing scheme may also be used with a hybrid identification approach. In some embodiments, input filter 112 (or output filter 115) includes a lookup table that includes the neuron IDs of a specific node 100. Input filter 112 and output filter 115 can be configured to perform a matching operation to match the source neuron ID of an incoming pulse message with the target synapse of a target neuron within node 100, wherein the target neuron is linked to the source neuron via a synaptic connection. An example of pattern matching using resistive or diode memory arrays is described in patent application No. 16 / 902,685 entitled “Matching Patterns in Memory Arrays,” which is incorporated herein by reference in its entirety. The synaptic connection ID can be stored as a pattern in memory array 103. For example, the synaptic connection ID can be stored along specific bit lines (or word lines) of memory array 103. The source neuron ID of a pulse message can be matched against memory array 103 to determine whether a synaptic connection ID exists in memory array 103. The bit line (or word line) can correspond to a key-value pair linked to a portion of memory array 103 containing additional information about the synaptic connection (including connection strength, weight, precise delay value, the last time the connection experienced pulses, and other data). For this purpose, bit lines in the memory array can at least partially correspond to synaptic connections matched to source neuron IDs. Bit lines can be mapped to another memory segment storing synaptic connection parameters used for matching synaptic connections.

[0037] Components of node 100 can be coupled via local bus 118. Local bus 118 provides access to memory 103 for routing commands related to processing pulse messages. Node 100 may also include network interface 121. Network interface 121 provides data and / or control signals between node 100 and other nodes 100 or external systems. Therefore, network interface 121 can couple node 100 to mesh architecture 132.

[0038] The mesh architecture 132 can deliver the generated pulse messages, so that the generated pulse messages can be consumed by all target nodes 100. When the SNN is modeled after the BNN, the delivery time depends on a delay that is unique for each axon but ranges from one millisecond to 100 milliseconds. Real neurons may have a delay that depends on the length of their axonal trunk, which is common to all axonal branches and specific from that common point to the synapse. In some embodiments of the SNN architecture, the pulse message may contain descriptors such as neuron ID, time, delay, and possibly pulse intensity.

[0039] Mesh architecture 132 may require achieving a minimum bandwidth to support all connected nodes 100. Intelligent allocation of neuron and synapse placement can be used to reduce the bandwidth requirements that allow for node interconnectivity. Synapses can be placed adjacently, with their connections to every other synapse entirely within node 100. This reduces outflow of spurious message traffic. Normally, biological neurons have more local connections than distant neurons. Therefore, neural network connectivity naturally supports this allocation. This allocation may also have a decreasing gradient in connectivity with neighboring nodes 100 as they become more distant. Therefore, another technique is selective broadcasting or multicasting, where most spurious traffic is localized within neighboring nodes 100, where the connectivity gradient decreases for more distant nodes 100. Additional filters (e.g., input filter 112 or output filter 115) can be placed along mesh architecture 132 to support selective broadcasting, allowing filters to permit spurious messages with specific neuron IDs to enter the corresponding segments of mesh architecture 132. This reduces redundant traffic.

[0040] The following diagram illustrates the above... Figure 1 The pipeline architecture within the SNN system described herein handles some embodiments of incoming impulse messages. For example, the input filter 112 of node 100 receives impulse messages. The node stores various synaptic connections (identified by synaptic connection IDs). A synaptic connection stores the connection between two neurons (each identified by a corresponding neuron ID). Additionally, node 100 may store parameters (e.g., weights) for each synaptic connection. These parameters determine how impulse messages are transmitted from neuron to neuron. The pipeline architecture supports the ability to perform mathematical operations in parallel with the operation of performing a search operation to match impulse messages with target neurons, using the relevant synaptic connection parameters.

[0041] Figure 2 This is a block diagram illustrating pulse messages transmitted within an SNN system according to some embodiments of the present disclosure.

[0042] SNN architectures can divide the flow of pulse messages into sequential steps in time. That is, pulse message communication occurs within a given time slice (e.g., a time interval or time step). This quantizes the emission of pulse messages into individual sequential time steps. Figure 2 The diagram shows three sequential time steps. Each time step can span one (1) millisecond. In this embodiment, the first time step spans the first millisecond; the second time step spans the second millisecond; the third time step spans the third millisecond; and so on.

[0043] In the first time step, the input filter 112 of node 100 can receive a finite number of pulse messages, including a first pulse message 202a and a second pulse message 202b. In the second time step, the input filter 112 can receive additional pulse messages, including a third pulse message 202c, a fourth pulse message 202d, and a fifth pulse message 202e. In the third time step, the input filter 112 can continue to receive additional pulse messages, including a sixth pulse message 202f and a seventh pulse message 202g. Each pulse message 202a to 202g can conform to a predefined pulse message format 202. The predefined pulse message format 202 can include a set of pulse descriptors 205, 208, 211, and 214. The pulse descriptor can include the source neuron ID 205, the time delay 208, the time quantum 211, the pulse intensity 214, and possibly other information.

[0044] In some embodiments, the source neuron identifier 205 may have a size of 37 bits. The bit length of the source neuron identifier 205 may depend on the number of neurons in the SNN system. For example, 37 bits may be sufficient to address all neurons in an SNN the size of the human brain (e.g., 86 billion neurons). The time quantum 211 may identify the quantized time step that generates the pulse message. For example, the first pulse message 202a and the second pulse message 202b may have the same value for the time quantum 211. The third, fourth, and fifth pulse messages (202c to 202e) may have the same value for the time quantum 211, with the value increasing by one from the previous time step. In some embodiments, the time quantum may have a size of seven (7) bits to cover a range from one (1) millisecond to 100 milliseconds. The range may be bounded by the longest time it takes to transmit a pulse in a BNN. In some cases, if all messages are delivered within the minimum delay time from the time they are generated, then the time quantum 211 may be omitted from the message. The time delay 208 may reflect the delayed nature of the pulse message. In a BNN, the time delay varies with the physical properties of at least the source neurons and axons. Seven (7) bits may be sufficient to cover a range of one (1) millisecond to 100 milliseconds for the time delay information. In some embodiments, the value of time delay 208 may be stored along with the synaptic connection. In some embodiments, pulse intensity 214 may include an integer value representing the continuous intensity of the pulse. In some embodiments, the pulses have the same intensity (e.g., binary intensity), and therefore, pulse intensity 214 may be omitted. In some embodiments, various data may be encoded on the pulse intensity (e.g., pulse polarity and magnitude).

[0045] Figure 3 This is a block diagram illustrating nodes of a mesh architecture coupled to an SNN system according to some embodiments of the present disclosure.

[0046] Figure 3 The embodiments provided provide a high-level overview of the flow of spiking messages into and out of a node, such as node 100. As mentioned above, node 100 represents a cluster of neurons identified by neuron IDs. Additionally, each synapse of a neuron in node 100 is connected to a source neuron, wherein the connection is identified by a synaptic connection ID. Spiking message 202 may travel from the mesh architecture 132 to a specific node 100 at some point. Spiking message 202 is referred to as inflow spiking message 304. Node 100 includes an input filter 112, which is configured to determine which neurons in the inflow spiking messages 304 target neurons in node 100. For example, this could be a case where no inflow spiking message 304 targets neurons in node 100.

[0047] Input filter 112 is configured to perform a matching operation to select a subset (e.g., all, some, or none) of the incoming pulse messages 304 based on whether the incoming pulse message 304 targets neurons in node 100. Input filter 112 can thus reduce the workload performed by node 100 by identifying a subset of the incoming pulse messages 304 that are relevant to node 100. The matching operation may be based at least in part on matching the source neuron ID from the pulse message with a range of synaptic IDs stored in node 100. This range may be represented by bit patterns or sequences.

[0048] After filtering the inflow pulse message 304, node 100 performs two primary operations. One primary operation is to generate an outflow pulse message 307 based on the neurons and synaptic connections 312 of node 100. The other primary operation is to modify the properties of the neurons and synaptic connections 312. The neurons and synaptic connections 312 are digital, mixed-signal, or analog representations of neurons and synaptic connections in a BNN. The neurons and synaptic connections 312 may have various parameters and weights that model and define the inherent properties of the neurons and synaptic connections 312. In this respect, the parameters of the neurons or synaptic connections 312 represent the state of the neurons or synaptic connections. One parameter that can define the state of a neuron may include the cell membrane potential of the neuron. One parameter that can define the state of a synaptic connection is a synaptic strength (weight) value that models the resistance or conductance of the synaptic connection. Another parameter that can define the state of a synaptic connection 312 is a delay value. Various other parameters may be included in a similar manner. The implementation may depend on the synaptic and neuron model selected for the SNN.

[0049] BNNs process information and provide "intelligence" by activating neurons and altering the properties of synapses. Biological inputs (e.g., sensory signals) initiate the triggering of impulses through the BNN. Different groups of neurons are activated in a specific order and at specific timings to ultimately activate a biological output (e.g., hand muscles). BNNs learn by rewiring or reconstructing neural connections by adding new neural connections, removing old ones, increasing resistance between neural connections, introducing delays or decreasing resistance, and reducing delays. This is known as "synaptic plasticity," where changes in neuronal connections are a response to repetitive or absent impulses. Thus, BNNs continue to forward impulses to process inputs and produce outputs while simultaneously rewiring themselves for learning. Similarly, SNN architectures maintain information defining neurons and synaptic connections 312. This information is used to generate outflowing impulse messages 307 and is also dynamically updated for "learning."

[0050] To describe this in more detail, the core principle of SNN learning rules is that "neurons that fire together wire together," a concept known as Hebbian learning. One such rule depends on pulse timing, which is the time between the incoming neuron's pulse and the generating neuron's pulse. This is mathematically modeled as temporal pulse-dependent plasticity (STDP). STDP is a characteristic of biological neurons that adjust their synapses according to pre- and post-pulse timing. For a pre-synaptic pulse arriving before the target neuron generates a pulse, the synapse is enhanced. For a pre-synaptic pulse arriving after the target neuron generates a pulse, the synapse is degraded. The magnitude of the change in synaptic conductance (enhancement or degraded, i.e., increased or decreased) is determined by exponential curves. One curve is the long-duration enhancement (LTP), which determines the magnitude of the change in synaptic conductance for a synapse that receives a pulse before the target neuron generates a pulse. Another curve is the long-duration degrade (LTD), which determines the magnitude of the change in synaptic conductance for a synapse that receives a pulse after the target neuron generates a pulse. The STDP rule allows the SNN to perform "error correction" on each synapse locally and continuously.

[0051] In a computer memory implementation of SNNs, processing STDP may involve storing presynaptic pulses within a window of LTP duration, and then, once a postsynaptic neuron generates a pulse, these events are "replayed" and the synaptic conductance value is adjusted accordingly. Another approach is to implement a "qualification window" feature at the memory cell level or memory architecture level. SNN structural plasticity can be implemented by adding inefficient synaptic connections, such as those determined by plasticity rules, and allowing these inefficient synaptic connections to evolve through STDP calculations or by eliminating synaptic connections whose values ​​decay to extremely high resistance (inefficiency).

[0052] As neurons and synaptic connections 312 change over time via the STDP rule, neurons and their synaptic connections 312 generate outflow pulse messages 307. Output filter 115 determines how to route the outflow pulse messages 307. For example, output filter 115 can broadcast or multicast the outflow pulse messages to other nodes 100 via mesh architecture 132. Output filter 115 can also determine some of the outflow pulse messages 307 that target neurons within the same node 100.

[0053] Figure 4 This is a block diagram illustrating the memory of a node within an SNN system according to some embodiments of the present disclosure.

[0054] The illustrated embodiments provide examples of memory structures for storing information related to neurons and synaptic connections 312, storing, queuing, and prioritizing inflow pulse messages 304 and outflow pulse messages 307, and managing other data related to SNN operations. The illustrated memory structures also provide examples of organizing information to allow pipeline processing of pulse messages 202 handled by node 100.

[0055] As illustrated, a given node 100 includes memory 408. Memory 408 may include one or more memory arrays 103 or other sets of memory cells. Memory 408 may be divided into multiple segments, such as a pulse cache 413 (e.g., a first memory segment), a segment for storing synaptic connection data 421 (e.g., a second memory segment), and a segment for storing synaptic connections and neuron parameters 435 (e.g., a third memory segment). Each memory segment may be defined by one or more memory array identifiers that identify a particular memory array 103, a row (or word line) range, a column (or bit line) range, one or more layer identifiers that identify a layer (e.g., a layer in a 3D memory device), or other groupings of memory cells.

[0056] A first memory segment may be reserved for pulse cache 413. The pulse cache is configured to store pulse messages 202 from a predefined number of pulse groups. For example, pulse cache 413 may store all incoming pulse messages 304 filtered by input filter 112. In other words, pulse messages 202 are filtered such that they include neurons within node 100. Pulse messages 202 that do not target neurons in node 100 are not stored in pulse cache 413.

[0057] In some embodiments, each pulse message 202 is assigned to a corresponding pulse group based on the value of the time delay 208 contained in the pulse message 202, or, in a simpler case, to a group with the most recently arrived pulse. A pulse group may be a "bucket" with a corresponding tag or identifier. The use of pulse groups allows for prioritization of pulse messages with smaller delays over pulses with larger delays, and allows for temporally continuous pulse movement.

[0058] Additionally, for a given time step, a pulse set passes through input filter 112 and is stored in a pulse group within pulse cache 413. A pulse group may have an identifier (e.g., a tag "0") indicating that it is the group with the most recent pulse. The tag for subsequent groups is incremented by one. There may be as many pulse groups as there are time steps in the maximum possible delay. For example, given a maximum delay of 100 milliseconds and a time step of one (1) millisecond, there are 100 pulse groups and associated tags. These pulse groups constitute pulse cache 413, which has temporal locality and a schedule table associated with processing pulses based on this locality. In some embodiments, a pulse message does not need to be stored for the entire duration until it becomes associated with the maximum delay bucket (e.g., 100 milliseconds). In fact, the pulse message is removed (invalidated) once its longest delay has been processed. Therefore, this helps to maintain efficient utilization of the cache.

[0059] In some embodiments, pulse cache 413 includes a small table that stores "tag<=> pointers" to its pulse groups stored in an association in memory. Tags incremented to tag "100" eventually loop back to tag "0". Old pulses can be discarded or overwritten by newly arrived pulses. This incrementing functionality can be implemented by incrementing a single rotation counter (e.g., an increment operation and a modulo operation). The counter identifies the tag for which a newly filtered pulse can be placed into the latest pulse group in the current time step. Alternatively, pulses can be placed in an associated bucket based on delay information in the pulse message. Pulse groups can be described as an opaque memory allocation storing pulse message descriptors; however, physically, they may not be opaque but distributed.

[0060] A second segment of the memory can be reserved for synaptic connection data 421. For example, the second segment of the memory is configured to store data indicating multiple synaptic connections, where each synaptic connection references a set of neuron identifiers. The second segment of the memory can be organized by neurons (424a to 424n). For example, the illustrated embodiment shows data stored for the first neuron 424a to the last neuron 424n of node 100. For each neuron (424a to 424n), the second segment of the memory stores a set of synaptic connections (426a to 426n). Each synaptic connection (426a to 426n) may contain data including a synaptic connection ID 429, a time delay 432, and / or a source neuron ID 433 (e.g., a presynaptic neuron ID). In some embodiments, this synaptic connection ID is the same as or contains the source neuron ID, thus eliminating the need to store both. Synaptic connections in a BNN involve the axon of a source neuron connecting to the dendrites of one or more target neurons. Therefore, synaptic connections (426a to 426n) of a given neuron (424a to 424n) can be accessed and identified using synaptic connection ID 429. Additionally, each synaptic connection (426a to 426n) specifies a source neuron ID 433 for the firing neuron. In some embodiments, the synaptic connection ID is the same as the source neuron ID and is therefore not required. Time delay 432 or other parameters may define the characteristics of the synaptic connection. In some embodiments, the time delay 432 stored in node 100 has a precise value, while the pulse message 202 contains a time delay 208 with a coarse value. In some embodiments, the aforementioned variables may be stored in different segments of the memory array or in different memory arrays corresponding to relevant locations of the delay values.

[0061] Each neuron (424a to 424n) has presynaptic (afferent or source) connections (426a to 426n). These connections can be grouped or ordered by delay values. In a BNN, pulses are transmitted across synaptic connections (426a to 426n). The pulses experience delays, which are how the BNN at least partially encodes information. In other words, the timing of firing neurons is how information in the BNN is processed. In an SNN architecture, delays are modeled using one or more delay values. A pulse message 202 may contain a time delay 208, which is a coarse value. Synaptic connections (426a to 426n) may store time delays 432 with precise values. Put together, the sum of the coarse and precise values ​​of time delays 208 and 432 represents the overall delay of a particular synaptic connection (426a to 426n).

[0062] The coarse time delay 208 may have a range (e.g., between one millisecond and 100 milliseconds). The coarse time delay 208 is quantized in increments of the time step. If high delay accuracy is required, this coarse delay value can be made more precise by adding a precise time delay 432 (e.g., a floating value between zero and one, representing the precise delay within the time step). The precise time delay 432 complements the quantized delay and may be used in some embodiments to improve accuracy. Depending on the embodiment, the precise time delay added to the coarse time delay may include a floating-point or integer or some other custom format.

[0063] Synaptic connections (426a to 426n) of each neuron (424a to 424n) can also be organized and processed in buckets in increments of time steps (e.g., one millisecond) based on a coarse delay value.

[0064] The memory 408 of node 100 may also include a third memory segment reserved for storing neurons and synaptic connection parameters 435 associated with each synaptic connection of a particular neuron. For example, the third memory segment may organize data by neurons (424a to 424n). Data modeled for each neuron (424a to 424n) is stored in this memory segment. This may include the membrane potential 436 and other parameters 438 for each neuron. These parameters may include all synaptic connections associated with the neuron, such as synaptic connections (426a to 426n). The membrane potential in a BNN is an inherent property defining the state of a neuron. The membrane potential changes over time based on the transmembrane current dynamics, at least in part, attributable to received impulses. In other words, the intensity and frequency of the impulses received by the neuron change the neuron's membrane potential over time. In a SNN, the membrane potential 436 is recorded as a value stored in memory for each neuron (424a to 424n). The membrane potential 436 can be continuously updated in response to a specific neuron receiving a pulse message 202. In addition to the membrane potential, other neuronal variables defining the neuron's state can be stored. These variables may include various ion currents, osmotic states, concentrations of certain chemicals, etc.

[0065] Other parameter 438 contains weight values ​​for each synaptic connection (426a to 426n) associated with a specific neuron (424a to 424n). When stored in memory, synaptic connections can be grouped by the neurons associated with them. Synaptic connections can be modeled as having specific weights. The combination of weights for multiple synaptic connections causes the SNN to train and learn. The weights change over time due to STDP. STDP makes neurons act as selector devices. Neurons evolve to exhibit specific weight combinations across their synaptic connections. Quantifying connectivity using weights allows the SNN to generate outflowing impulse messages.

[0066] Synaptic connection parameters 435 are used to perform current integration operations to calculate how the properties of neurons 424a to 424n change over time (e.g., neuronal membrane potential 436) and to determine the outflow pulse message 307 generated by each neuron (424a to 424n) that generates a pulse.

[0067] Figure 4 The organization of the memory 408 of the node shown allows incoming pulse messages to be queued in the pulse cache 413. Synaptic connections can be searched based on the source neuron ID 205 contained in each pulse message 202. Such searches can be performed within each delay bucket or group through which pulses are stored in the cache. This may involve performing an in-memory pattern search technique to match the source neuron ID 205 in the pulse message 202 to the source neuron ID 205 in the synaptic connection data 421 of the second memory segment. As part of the pipeline architecture, in response to a match found in the second memory segment, the matched target neurons (424a to 424n) and / or synaptic connections (426a to 426n) can then be identified and can point to neurons (424a to 424n) or synaptic connections (426a to 426n) in the third memory segment. The synaptic connection parameters 435 stored in the third memory segment can be used to perform current integration, neuronal integration, STDP operation, and other neuromorphic features.

[0068] In some embodiments, pulse messages are not stored in a delay bucket. For example, at each time step, the node permits filtered pulse messages. These pulse messages are matched against the synapse IDs of all neurons in the node. Synapse IDs may be pre-classified, which speeds up the matching process. Pulse IDs can immediately indicate the location (e.g., index) of all target synapses and associated neurons. Each synapse may contain a counter instantiated by a delay value (or zero). The clock of each counter decrements (or increments) until it reaches zero or some other predetermined delay value.

[0069] The end of the counter signifies that a pulse message has arrived at its synapse. This search and matching process can be pipelined into synaptic and neuronal computation, which generates new pulses to be sent to the network.

[0070] One way to implement this operation is by using a self-counting cache consisting of a pulse ID and one or more associated self-decrementing counters, each counter having a signal line. The signal line can be a dual-signal line. For example, the signal line can have a horizontal signal line and a vertical signal line, wherein the intersection of the horizontal and vertical signal lines within the counter grid indicates which counter has expired. Such signal lines can be pull-up or pull-down wires.

[0071] Figure 5This is a block diagram illustrating the functionality and structure of nodes within an SNN system according to some embodiments of the present disclosure.

[0072] The illustrated embodiments are based on Figure 4 The illustrated embodiment describes a memory structure and a pipeline architecture for parallel execution of synaptic connection search and computation. The illustrated embodiment shows a pulse cache 413 organized by a predefined number of buckets (502a to 502n). Each bucket logically or physically corresponds to a different time step in which an inflow pulse message 304 is received. A controller 109 can receive the inflow pulse message 304. An input filter 112 can allow only relevant pulse messages targeting neurons 424 within node 100. Along with filtering, the input filter 112 can perform other operations. These operations include, for example, determining the synapses and neurons targeted by the pulses, placing the pulses into a hardware queue or directly into the pulse cache, and transporting the pulses to the controller. The controller 109 can store the inflow pulse message 304 in a corresponding bucket 502 based on the value of the time delay 208 in the inflow pulse message 304, or, in a simple case, in bucket "1" (the minimum delay bucket). Thus, the inflow pulse message 304 is grouped together by sequentially sorted buckets 502 based on the quantized time delay. The pulse messages of bucket 502 are processed together before moving to the pulse messages of the next bucket 502.

[0073] Similarly, synaptic connection data 421 is organized through a predetermined number of buckets (505a to 505n), and synaptic connection parameters 435 can also be organized through a predetermined number of buckets (508a to 505n). Each bucket 505, 508 may contain a set of memory cells 513 within a memory array (e.g., defined by row / column ranges), wherein the memory cells 513 are coupled to a sense amplifier 516.

[0074] The same number of buckets 502, 505, and 508 may exist for the pulse cache 413, synaptic connection data 421, and synaptic connection parameters 435. For example, 100 buckets ranging from bucket 1 to bucket 100 may exist. A bucket counter may also exist, which increments from 1 to 100 at each time step and then loops back to 1. For example, at clock cycle i, controller 109 processes buckets 502i, 505i, and 508i-1 in the relevant memory segment. Specifically, buckets 502i and 505i are involved in the search and match operation (matching the pulse ID from bucket i of the pulse cache with the synaptic ID in the synaptic connection data 421). The end result of this operation is the determination of which synaptic connections match which pulse messages. This data is used in the next clock cycle with bucket 508i. Furthermore, in clock loop i, controller 109 processes bucket 508i-1 for synaptic connections identified as matched in the previous loop when performing search and match on buckets 502i-1 and 505i-1. Processing bucket 508i-1 may involve current integration, neuronal integration, STDP operation, and other neuromorphic features. In the next clock loop, the bucket counter increments to bucket i+1, and controller 109 processes the second bucket 502i+1, 505i+1, and 508i+1 in each memory segment. The processing is the same as in clock loop i. This process is repeated for all delay buckets. For example, if there are 100 delay buckets, then there are 100 clock loops for a single time step (e.g., clock loop i). Due to this pipelined process, search and match operations occur in parallel with neuromorphic operations. Each time step involves processing all delay buckets, but the main difference between successive time steps is that the delay buckets are rotated by one position, and the delay buckets are searched / matched against different synaptic connection buckets. The term "clock cycle" can be replaced with "step size," etc. The clock cycle or step size used for this process via the delay bucket can be local and separate from the time step applied to the global operation that synchronizes the entire SNN system.

[0075] At each time step, multiple synaptic events (e.g., newly generated pulse messages) are generated immediately after a match is detected between the pulse ID and the synapse ID of the synapse from each neuron 424 in each time step bucket. In this regard, all neurons and all buckets per neuron can be processed in parallel in the pipeline architecture. This involves first performing a search / matching operation to locate the synaptic connection 426. For each successful match, the next step is to integrate the postsynaptic current associated with that match. In a BNN, postsynaptic currents are generated due to local openings in the vesicle membrane, and all successfully matched postsynaptic currents per neuron can be integrated. The integration process can take into account the distance from the synapse to the neuron cell body (requiring a more complex integration scheme), or it can ignore this complexity and simplify it to a simple summation of currents generated according to the synaptic efficacy triggered by the pulse. In an SNN architecture, the current integration operation is performed by accessing a memory segment containing the synaptic connection parameters 435 while performing a search / matching operation on a memory segment containing the synaptic connection data 421. Various optimizations may be used for bucket allocation. This could involve, for example, classifying neurons by connection commonality and storing them in memory, and then assigning neurons to nodes. Another optimization is to inject spike messages into delay buckets based on their minimum latency and discard spike messages from delay buckets based on their maximum latency. This reduces unnecessary matching operations. This would require adding minimum latency information to the spike message descriptor. Another optimization is to classify spike messages in each spike bucket by spike neuron ID and classify synapses in each synapse bucket. This limits the processing scope to only a subset of spike IDs.

[0076] Pulse IDs are a common dependency that can be employed for matching operations. For example, a pulse ID can be mapped to a set of word lines (WLs) driving a segment of the memory array, and each bit line (BL) can respond to a match / mismatch signal. Another aspect requiring parallelization is delay buckets. For example, each delay bucket can be stored in a different memory array and can perform matching operations in parallel with other buckets. Neuron parallelism is another aspect. In this example, neurons can be distributed across multiple subarrays. Alternatively, matching can be performed in a more serial manner (e.g., descending to one bucket at a time in a single array), as long as all buckets complete well before the real-time step (e.g., 1 ms) expires, to ensure Quality of Service (QoS). Various pipelined and multiplexing variations of this approach are possible.

[0077] In some cases, network topology implicitly matches and can avoid matching. For SNN segments with regular connections, such as those performing convolution operations, the network topology closely resembles a memory array. Search and matching operations can be the same for various neurons. Cortical columns have similar but structurally poor topologies. For other SNN segments, the synaptic connections between neurons can vary significantly.

[0078] In some embodiments, both matching and current summation are performed at appropriate locations within the memory array, such that the matching operation is integrated with the current integration (e.g., the matching operation locally gates the current integration to each memory cell (or group of cells)). This may involve forming a conditional memory such that it immediately provides access to the contents of a second group of cells after a pattern match is detected on the contents of a first group. This access is provided at the appropriate location (without passing through a sense amplifier). Thus, multiple patterns can be streamed to multiple WL groups in a device similar to this conditional memory, and BL will produce computation results at the appropriate locations. The potential of such memories is broad and can include cryptography, content-addressable memory, in-memory logic, graph operations, or networks other than SNNs. One potential way to achieve this is a two-layer configuration, where the first layer stores keywords and the second layer stores values ​​that can be conditionally accessed after a matching keyword. Another approach is a NAND string-gated WL of NOR rows containing the synapses of all neurons with synaptic IDs stored in the NAND memory device. Another approach is to gate another segment of the NAND string containing synaptic information.

[0079] The performance of this type of memory can depend primarily on the number of pulse messages received per delay bucket. For example, for an average of 100 pulse messages per delay bucket in node 100, the system might require 100 clock cycles per time step.

[0080] Figure 6 This is a block diagram illustrating parallel processing via nodes within an SNN system according to some embodiments of the present disclosure.

[0081] Figure 6 The operation of node 100 arranged in a pipeline architecture is illustrated to provide parallel processing for finding target synaptic connections 426 and performing current integration calculations using parameters of the target neuron. Figure 6 This shows the pipeline for moving from left to right within a specific time step (e.g., for the current bucket).

[0082] At item 602, node 100 receives the incoming pulse message 304. Filter 112 can filter out pulse messages that are not directed to node 100. Pulse message 602 is received from other interconnected nodes 100 via a mesh architecture.

[0083] At item 605, node 100 updates the pulse group. For example, controller 109 may store the inflow pulse message 602 in the corresponding bucket 502 based on the time delay 208 in the inflow pulse message 602, or in the current bucket 1 in a simpler case. Inflow pulse messages 602 with smaller delays are cached in buckets that are directed toward the current bucket, as indicated by the bucket counter. The circular pointer increment may occur before the cached pulse messages.

[0084] Next, search and match operations are used to process each pulse message stored in all buckets. For example, Figure 1 The processing of the first incoming pulse message (labeled "ISM1") is shown. ISM1 is the pulse message contained in the current bucket based on a bucket counter corresponding to the current time step. Several other pulse messages may exist in the current bucket and other buckets; however, Figure 6 The diagram illustrates the processing of a single inflow pulse message 304. ISM1 originates from a source neuron (e.g., a presynaptic neuron) having a source neuron ID 205. The source neuron may have synaptic connections with one or more target neurons 424 in the current bucket. ISM1 should target each neuron 424 co-connected to the source neuron.

[0085] At item 608, node 100 performs a search and match to identify synaptic connection ID 429. The search and match operation may be an in-memory operation to determine whether the memory is storing a source neuron identifier 433 that matches the source neuron identifier 205 of ISM1. And if so, then the source neuron identifier is located in memory. For example, the search and match operation may involve an in-memory pattern matching operation to determine whether the memory array 103 contains an input pattern (e.g., a bit sequence corresponding to the source neuron identifier 205). Therefore, the search and match operation may involve comparing the bit pattern of the source neuron identifier contained in the impulse message with several bit patterns stored in memory to identify the synaptic connection. This can occur in a bit-serial, bit-parallel, or hybrid manner as discussed above.

[0086] After the match is located, the synaptic connection ID 429 is immediately determined. In some embodiments, key-value pairs are used to associate the source neuron identifier 433 with the synaptic connection ID 429. For example, if the matching neuron identifier 433 is located on a specific bit line and word line, then the bit line and word line are mapped to a specific memory location containing the synaptic connection ID 429 for the synaptic connection. Figure 6The identifier for the first synaptic connection 426 (labeled "SC1") is shown. In other words, the search and matching operation performed on ISM1 yields SC1. Neuromorphic computation for a single neuron may require generating all synaptic connections SC1 for that neuron (i.e., identifying all synapses that receive pulses in the current time step). As described above, ISM1 can target multiple synaptic connections of multiple neurons.

[0087] In some embodiments, the array of source neuron identifiers 433a-n is stored in consecutive rows. For each row of consecutively stored source neuron identifiers 433a-n, the sense amplifier array generates a bitmask indicating which source neuron identifiers 433a-n have matched any of the source neuron identifiers 205 of ISM1 in the current bucket. During the search and matching operation, a medium bitmask is generated by comparing the corresponding bit matches retrieved from memory (the associated delay bucket storing the synapse IDs) and testing each bit of all incoming pulse messages in all buckets. Each bit of this bitmask is updated as subsequent bits for each of the source neuron identifiers 205 are compared. Finally, after row ordering and comparing each bit from these rows with each associated bit of each of the source neuron identifiers 205, the bits in the bitmask can indicate a match. These bitmasks (one bitmask per sense amplifier) ​​can be stored in a fast storage device (e.g., an array of static RAM (SRAM) or fast dynamic RAM (DRAM)) close to each sense amplifier. Bit masks can be used for optimization so that a single bit mismatch eliminates potential matches in subsequent bits of the source neuron identifier 205 flowing into the pulse message 304. Additionally, multiple comparators and additional local fast storage can be added per sense amplifier to hold a wider bit mask. In some cases, for 3D memory topologies, bit groups with the same synapse ID can be distributed across multiple layers or dies in the memory stack, thus allowing parallel comparison operations at each layer or die.

[0088] In some embodiments, source neuron identifiers 433a-n are stored in non-volatile memory to support in-memory search and match operations. Search and match operations can be performed by activating groups of word lines storing source neuron identifiers 433a-n in parallel and also by activating groups of bit lines storing source neuron identifiers 433a-n in parallel. Therefore, pipelined and multiplexed approaches can be used to fully overlap search and match operations with memory access. Some memory access techniques that perform computations on bit lines when applied to non-volatile memory allow this step to be “packed” into fewer operations by performing parallel matches on parallel-activated sublines and sensed bit lines.

[0089] Following identification SC1, node 100 can immediately perform neuromorphic computation. For example, at item 611, node 100 performs neuromorphic computation. This may include performing synaptic current integration and membrane potential calculations, as well as synaptic plasticity calculations including STDP and structural plasticity calculations. These operations mathematically model synaptic plasticity. In some embodiments, pipelined and multiplexed methods are used to fully overlap neuromorphic computation with memory access. Some memory access techniques that perform computations on bit lines when applied to non-volatile memory allow synaptic alterations to be performed at appropriate locations within the memory array 103. The overall neuromorphic computation 611 can be a relatively large computational operation using significant computational resources. Figure 1 As shown, neuromorphic computations (e.g., current integration, membrane potential calculations, etc.) are performed immediately after identifying SC1. Additionally, synaptic plasticity can be interleaved with current integration. Specifically, the LTP based on synaptic events in the previous time step can be calculated in the current time step after detecting neuronal excitation in the previous time step. Neuronal excitation or impulse detection is performed after solving the neuronal model membrane equations, which are based on changes in membrane potential over time and on differential equations of currents calculated by performing current integration. Current integration is based on weight changes relative to past and future postsynaptic impulses. The neuronal weights can be stored as synaptic connection parameters for a specific neuron 424.

[0090] Synaptic plasticity calculations (e.g., STDP and structural plasticity calculations) produce updated values ​​for the synaptic connection parameter 435. Specifically, this may involve calculating new weight values ​​for the synaptic connections. Synaptic plasticity calculations involve using the STPD (LTD and LTP) equations with presynaptic and postsynaptic pulse timing and the current state of each synapse.

[0091] Power efficiency can be optimized when accessing synaptic connection parameters 435 (e.g., weights). For example, a bitmask can be generated to indicate the location of a matching identifier. However, such bitmasks can be sparse in the sense that only a few matches are performed (e.g., 1% of all target identifiers). Each bitmask represents a unique neuron. Memory that uniquely accesses each cell in a column or row can be used to access the synaptic connection parameters 435. Weights from each column or row can be accessed at unique locations in the column or row. However, this is difficult to achieve in memory devices with a shared WL. Therefore, all BLs are accessed per WL. For some memory techniques, nodes can offload or mask some BL accesses to save power while utilizing sparse memory accesses for other computations.

[0092] Due to various neuromorphic computations, node 100 can generate an outflowing pulse message (labeled "OSM1"), which is generated at least in part by locating SC1 in a memory segment and performing in-memory computations in different memory segments to generate OSM1 based on SC1 (depending on the neuron state, OSM1 may or may not be generated in the current time step). While SC1 is identified and OSM1 is generated, search and matching operations can continue in the memory segment storing synaptic connection data 421. For example, ISM1 can target multiple synaptic connections, searching for each of the multiple synaptic connections in the memory segment storing synaptic connection data 421. The pipeline architecture allows additional synaptic connections (labeled "SC2") to be identified while performing neuromorphic computations with respect to SC1. SC2 may involve a second target neuron 424 that also generates pulses via ISM1. SC2 is used to generate a second outflowing pulse message (labeled "OSM2"). Therefore, the operations shown in item 608 are performed (at least in part) in parallel with respect to the operations shown in item 611. The memory architecture of node 100 supports this parallel pipeline processing by storing synaptic connection data 421 in one memory segment (for performing search and matching operations) and storing synaptic connection parameters in different memory segments to perform synaptic plasticity calculations on matched neurons / synaptic connections and generate outflow pulse messages.

[0093] At item 614, node 100 transmits an outgoing pulse message. For example, output filter 115 can process the output pulse message and transmit the output pulse message to other nodes 100 via mesh architecture 132 and / or transmit the output pulse message internally within the same node 100.

[0094] The following provides an additional example of handling impulse messages generated in response to an inflow impulse message. Upon detecting a generated impulse message, the neuron ID that generated the impulse message is immediately reported to node 100 (e.g., a filter or router associated with the node). Node 100 prepares impulse descriptors for all neurons that generated impulses to produce outflow impulse messages. Node 100 performs a broadcast or multicast operation to transmit the impulse descriptor throughout the SNN system. The output filter can also filter out impulses with local connections within the node and distribute them locally to the relevant delay buckets. Thus, the broadcast or multicast operation can begin (in real-time) for all memory arrays in the SNN network within a fraction of a clock cycle. Because the membrane potential can be calculated in a SIMD manner for the entire memory array, the detection and generation of postsynaptic impulse messages are also performed in parallel for all neurons. After all impulse messages have been sent to the SNN network, node 100 can immediately send a mask message containing several impulses it generated, allowing the receiving router to perform masking, as well as other examples of masking for this component. Such mask messages, along with all impulse messages, may also contain associated identifiers of neurons and / or nodes.

[0095] Some embodiments are directed to a node 100 with a memory structure consisting of multiple layers. Multiple layers can be used to provide parallelization of search and matching operations with neuromorphic computations (e.g., current integration). In some embodiments, presynaptic ID bits can be distributed across several memory arrays by means of multiplexing. This can significantly improve performance. For example, distributing IDs across 37 arrays (to track the size of the human brain, which consists of 86 billion neurons) can result in a 37-fold reduction in latency for search and matching operations. This can be referred to as a multiplexing configuration for achieving high-performance computing (HPC). Synaptic currents and weights, through vertically integrated accumulation, may require floating-point adders distributed across different layers or bonded dies. This eliminates the need for wide input / output. Generally, the entire single-instruction multiple-data (SIMD) processor, consisting of floating-point fused multiply-accumulate (FMA) units, can be distributed across layers, thus implementing a compute-on-a-way paradigm.

[0096] The following embodiments describe the use of a SIMD or MIMD processor to implement SNN operations according to some of the disclosed embodiments. In some embodiments, the controller 109 may be implemented as a SIMD or MIMD processing device and may perform the following methods.

[0097] Figure 7A This is a flowchart illustrating a method for performing pulse delivery in a SIMD or MIMD pipeline according to some embodiments of the present disclosure.

[0098] In block 702a, the method includes loading the first bit of the synaptic ID from a first delay group from memory into one or more SIMD channels. In the illustrated embodiment, the delay group may correspond to a neuronal bucket, such as buckets 505a to 505n. That is, the delay group can be read from the neuronal bucket. In one embodiment, a given delay group of a synaptic identifier is represented as a vector sequence SyID. g , where g represents a given delay group identifier (e.g., from 0 to n). In the illustrated embodiment, the method first loads the first bit of each vector into the corresponding SIMD lane.

[0099] In one embodiment, the synapse ID is stored in a two-dimensional memory cell array. In such an embodiment, the synapse ID for a given delay group can be represented as:

[0100]

[0101] Where s includes a binary value, n represents the synaptic identifier length (in bits), and L represents the total number of synaptic identifiers in a given delay group. In one embodiment, each binary value s may correspond to the intersection of a row and column line of a memory device. Alternatively, the matrix index annotation SyID[g,k][n] is used to refer to individual bits, where g represents the delay group identifier, k represents the synaptic identifier index (e.g., the row of SyID), and n represents the bit position. Thus, s for delay group 1 1,1 It can be equivalently represented as SyID[1,1][1]. In some instances, the delayed group identifier is omitted for brevity.

[0102] Figure 7B The timing block diagram illustrates the sequence of operations in four SIMD pipelines and assumes a synaptic ID vector length of four. Each block in the diagram represents a single operation, and some operations can be reordered in steps relative to each other, as long as they are independent of each other. Specific channel and vector sizes are not limiting. As illustrated, in such an embodiment, in the first operation described by step or block 702a, channel 0 is loaded with SyID[1,1][1] (i.e., the synaptic ID vector of delay group 1). 1,1 ), channel 1 is loaded with SyID[1,2][1] (that is, the delay group 1's s 2,1 ), channel 2 is loaded with SyID[1,3][1] (that is, the delay group 1's s 3,1 ), and channel 3 is loaded with SyID[1,4][1] (i.e., the delay group 1's s 4,1 (702a-1). In other words, the method loads the first column of the SyID matrix into the SIMD pipeline.

[0103] In the illustrated embodiment, the bits loaded at block 702a of the timing block diagram are loaded into a bit vector register (referred to as V2) associated with each track. Therefore, the V2 register of an L SIMD track may contain the following vector:

[0104]

[0105] At the end of box 702a, register V2[0…L] corresponds to the serial bit string associated with the first bit (bit 1) of all synapse ID values ​​in a given delay group. In other words, register V2[0…L] corresponds to the bit value at the first position of all synapse IDs in the matrix.

[0106] In block 704a, the method includes broadcasting the first bit of a pulse identifier to a SIMD channel. In the illustrated embodiment, the pulse identifier can be broadcast from pulse buckets (e.g., buckets 502a to 502n) in a pulse buffer. In one embodiment, a given bucket of a pulse identifier is represented as a vector matrix:

[0107]

[0108] Where sp includes a binary value, n represents the pulse identifier length (in bits) equal to the lengths of both the synaptic identifier and the pulse identifier, and M represents the total number of pulse identifiers in a given delay group. In one embodiment, each binary value sp may correspond to the intersection of a row and column line of a memory device or cache device. Alternatively, the matrix index annotation SpID[g,k][n] is used to refer to individual bits, where g represents the delay group identifier, k represents the pulse identifier index (e.g., the row of SyID), and n represents the bit position. Thus, the s of pulse delay group 1 1,1 It can be equivalently represented as SpID[1,1][1]. In some instances, the delayed group identifier is omitted for brevity.

[0109] In block 704a, the first bit of the first pulse identifier is broadcast to the register (V3) associated with each SIMD channel. Therefore, at the end of block 704, the V3 register of the L SIMD channel may contain the following vector:

[0110]

[0111] As can be seen, vector V3 contains the same bit values ​​for each SIMD channel L.

[0112] exist Figure 7B In an example embodiment, in block 704a, each SIMD track is loaded with the bit value SpID[1,1][1].

[0113] In the illustrated embodiment, the bits loaded in block 704a are loaded into a bit vector register (V3), where each bit is associated with a corresponding SIMD track. Therefore, each track (1 to L) is associated with a single bit in the bit vector register (V3) at the corresponding position. At the end of block 704a, register V3 corresponds to a serial bit string associated with a given position (e.g., position 1) of the pulse ID value. It should be noted that blocks 702a and 704a can be performed in parallel or, due to the fact that these steps are independent of each other (i.e., one operation does not feed data to another operation to begin it), in a different order relative to each other.

[0114] In block 706a, the method compares the synaptic ID bit with the pulse ID bit and caches the result. At the end of blocks 702a and 704a, each SIMD channel contains bits from the synaptic ID (register V2), and all SIMD channels or subsets thereof contain or have the same bits from the pulse ID (register V3). In the illustrated embodiment, block 706a compares these registers and caches the comparison result in cache location C. n In the above, n corresponds to the pulse ID being compared (i.e., 1).

[0115] In one embodiment, the method uses an XNOR (Extra-NOR) operation to compare the synapse ID and the pulse ID. Therefore, the data stored in C... n The result in V2 is equal to V2. n ⊙V3 n The value of n is equal to the vector length of the system, and specifically, equal to the number of SIMD channels.

[0116] Return to Figure 7B During levels 0 to 2, the method calculates C1, where C1 includes an XNOR comparison of all bits of V2 and V3. Therefore, the cached value of C1 is equal to...

[0117]

[0118] Each XNOR operation is performed by the corresponding SIMD channel L.

[0119] exist Figure 7A In box 718a, the method determines whether all pulse identifiers in a given group have been processed. If not, the method continues with boxes 704a and 706a until all the first bits of the pulse identifiers in the given group have been processed.

[0120] Specifically, in the illustrated embodiment, the output of block 718a includes cached vectors C1…C MThe set is given by M, where M corresponds to the number of pulse identifiers. Therefore, each cache vector C is associated with a pulse identifier (0 to M).

[0121] Therefore, the method executes blocks 704a and 706a for each first bit of each pulse vector in a given delay group. More generally, the method computes a bit vector of size M:

[0122]

[0123] Where i increments from 1 to M during processing. Therefore, in Figure 7B In the diagram, boxes 704a-1 / 706a-1 correspond to i=1 (the first digit of the first pulse ID), 704a-2 / 706a-2 correspond to i=2 (the first digit of the second pulse ID), 704a-3 / 706a-3 correspond to i=3 (the first digit of the third pulse ID), and 704a-4 / 706a-4 correspond to i=4 (the first digit of the fourth pulse ID), etc.

[0124] When the method reaches i = M, the method has generated M cache vectors with the above form. The method then proceeds to block 708a.

[0125] In some embodiments, the above pipeline can be further parallelized by introducing more comparator units and thus comparing the first bit from several pulse IDs at once.

[0126] In blocks 708a to 716a, the method performs a cyclic repetition to process all remaining bits (2…n) of each of the remaining pulse identifiers and compares them with the associated synaptic identifier and caches the comparison result in the associated cache bit vector.

[0127] In block 708a, the method loads a cache bit vector that stores the previous comparison result of a specific pulse identifier. In one embodiment, the method will take the cache orientation C... i The bit vector is loaded into bit vector register V1 for use in the L SIMD channel of the first SIMD. In the illustrated embodiment, the method is executed for each pulse identifier in block 708a. In one embodiment, the method may use register V1 as an enable bitmask for the SIMD channel for subsequent operations. In the illustrated embodiment, the method begins in block 708a by loading the value of C1 from memory (the comparison result for the first pulse identifier).

[0128] In block 720a, the method determines whether the next bit from the synaptic identifier should be loaded. As will be discussed, in one embodiment, the method loads the bit vector C of the j-th pulse identifier. j[l] (where 1 < l < L). Subsequently, after processing all the pulse identifiers at the current bit position i, in block 716a, the method updates the bit position i (e.g., from 1 up to n) to the next bit position i + 1. When the method updates the bit position in block 716a, it signals that the next bit position i + 1 from the synaptic identifier should be loaded in block 710a, and for all C j [l] Repeat the sequencing again.

[0129] In block 710a, the method conditionally loads the corresponding bit of the synaptic identifier into register V2 based on the current comparison result. In one embodiment, the method loads the current comparison result in the form of a bit vector C j [l] for the jth pulse ID in block 708a (where l corresponds to the lane number and represents the current comparison result in bit form for a particular synaptic ID) and then loads the value of SyID[1, l][i] when the corresponding bit in C j [l] is set (where l is the lane number between 1 and L). In other words, the method loads the following vector:

[0130]

[0131] That is, the ith bit position of each synaptic ID. By loading this bit position in each SIMD lane, the method can process all synaptic IDs in parallel.

[0132] In one embodiment, the method only loads the corresponding synaptic ID bit when the corresponding V1 value is 1. Thus, if the vector stored in V1 is [0, 0, ٠, 1], then the method will only load the ith bit position of the fourth synaptic ID s 4,i+1 and will not load the bit positions from the first three synaptic IDs. Thus, the method will only selectively load a subset of the ith bit positions of L synaptic identifiers, where L is the number of SIMD lanes.

[0133] In block 712a, the method broadcasts the pulse identifier bit position to register V3. In the illustrated embodiment, the data broadcast in block 712a includes the ith position of a single pulse identifier. Thus, vector V3 can be represented as: <00004,07>

[0135] where n represents the current pulse identifier being analyzed and i represents the current bit position (corresponding to the bit position in V2 i ). In one embodiment, the method can only load the value of the pulse identifier when the corresponding value in C i is one.

[0136] In block 714a, the method compares and caches the values ​​in V2 and V3. In one embodiment, the method computes an XOR operation on V2 and V3 and updates the result in the cached vector received in block 708a. That is:

[0137] C ij =V2 i ⊙V3 i

[0138] Therefore, in box 714a, the method updates the vector C representing the current comparison result of the j-th pulse ID and all synapse IDs based on the values ​​loaded in V2 and V3. j In some embodiments, the method may perform different types of bit-by-bit comparisons:

[0139] C j [i] = C j [i]AND(V2 i ⊙V3 i )

[0140] The above formula will be in C j [i] is always updated to 0 if it is already 0, thus performing the same function as the conditional comparison mentioned above.

[0141] In block 716a, the method determines whether all pulse IDs and their corresponding cache vectors in a given group have been processed. If not, the method loads the next cache vector associated with the corresponding pulse ID and re-executes blocks 708a, 710a, 712a, and 714a for each remaining pulse ID and its corresponding cache vector.

[0142] In some embodiments, blocks 708a, 710a, 712a, and 714a can be further parallelized by introducing more comparator units and thus comparing the first bit from several pulse IDs at a time. In the illustrated embodiment, cache orientation C i The bit vectors in the array become sparser with each repetition (i.e., due to the narrowing of the comparison results and the scarcity of binary 1 values). Therefore, in one embodiment, the method can compress these bit vectors to improve performance.

[0143] Although described sequentially, boxes 708a, 710a, 712a, and 714a can be pipelined to improve performance. For example... Figure 7B As described, in 708a-1 to 714a-1, the SIMD pipeline processes cache vector C1; in 708a-2, 712a-2, and 714a-2, the SIMD pipeline processes cache vector C2, and so on. In the described embodiments, once the final cache vector C... nAfter processing, the method can select the next bit position between the pulse identifier and the synapse identifier and repeat the pipelined method for each pulse identifier.

[0144] like Figure 7B As described above, in block 702A-1, the first SIMD processor loads the first SyID[g,k][1] of the L synaptic IDs SyID[g,k] from the first group (g=1) of several delay groups G in memory into the first SIMD channels 0 to 3 of the first SIMD processor for entry into the bit vector register V2, where k represents the first SIMD channel number. Therefore, in block 702A-1, the first SIMD channel performs the following operations:

[0145]

[0146] Although only four channels and bits are shown in the pipeline diagram, this disclosure is not limited to four bits and in fact, any number of bits can be processed and any number of channels can be used.

[0147] Next, in block 704A-1, the first SIMD broadcasts the first pulse ID SpID[g,i][1] (i=1, because it is the first pulse ID) of the first group (g=1) of the first delay group G from memory to the first SIMD track of the first SIMD to enter the bit vector register V3. Therefore, in block 704A-1, the first SIMD track performs the following operations:

[0148]

[0149] Next, in box 706A-1, the first SIMD compares V2 and V3 and writes the result to the first cache location C1 (e.g., V2 XNOR V3 => C1). Therefore, in box 706A-1, the SIMD performs the following operations:

[0150]

[0151]

[0152] The above process is repeated in a pipeline manner in steps 704A-2, 706A-2, 704A-3, 706A-3, 704A-4, and 706A-4, as described above.

[0153] Specifically, in step 704A-2, the first SIMD performs the following operations:

[0154]

[0155] In step 706A-2, the first SIMD performs the following operations:

[0156]

[0157] In step 704A-3, the first SIMD performs the following operations:

[0158]

[0159] In step 706A-3, the first SIMD performs the following operations:

[0160]

[0161] The above process continues until the last pulse ID is reached. In step 704A-4, the first SIMD broadcasts the first bit of the last pulse ID SpID[g,i][1] (i = the last, because it is the last pulse ID) from the first group (g=1) of the several delay groups G from memory to the first SIMD channels 0 to 3 of the first SIMD to enter the bit vector register V3. Therefore, in step 704A-4, the first SIMD performs the following operation:

[0162]

[0163]

[0164] In step 706A-4, the first SIMD compares V2 and V3 and writes the result to the g1th cache location Cg1 (e.g., V2 XNOR V3 => Cg1), where g1 represents a certain number corresponding to the number of pulse IDs in the first group (g=1) of several delay groups G. Therefore, the first SIMD performs the following operation:

[0165]

[0166] During step 706A-4, the first SIMD can execute step 708A-1 in parallel, wherein the first SIMD loads the bit vector from the first cache location C1 into the bit vector register V1 for use in the LSIMD channel of the first SIMD. In step 708A-1, the first SIMD configures register V1 as the enable bitmask for the first SIMD channel of the first SIMD for subsequent operations. Therefore, the first SIMD performs the following operations:

[0167] Dao 0: C1[0]=>V1[0]

[0168] Tao 1: C1[1]=>V1[1]

[0169] Tao 2: C1[2] => V1[2]

[0170] Tao 3: C1[3]=>V1[3]

[0171] Next, in step 710A-1, the first SIMD loads the second bit SyID[g,k][2] of the L synaptic IDs SyID[g,k] from the first group (g=1) of several delay groups G from memory into the first SIMD channel of the first SIMD to enter the bit vector register V2, where k represents the first SIMD channel number:

[0172] Dao 0: If V[0] == 1: SyID[1,0][2] => V2[0]

[0173] Path 1: If V[1] == 1: SyID[1,1][2] => V2[1]

[0174] Path 2: If V[2] == 1: SyID[1,2][2] => V2[2]

[0175] Path 3: If V[3] == 1: SyID[1,3][2] => V2[3]

[0176] In parallel with step 710A-1, the pipeline executes step 712A-1, where the first SIMD broadcasts the second bit of the first pulse ID SpID[g,i][2] (i=1, because it is the first pulse ID) from the first group (g=1) of several delay groups G from memory to the first SIMD channel of the first SIMD to enter the bit vector register V3:

[0177] Dao 0: If V[0] == 1: SpID[1,1][2] => V3[0]

[0178] Path 1: If V[1] = 1: SpID[1,1][2] => V3[1]

[0179] Path 2: If V[2] = 1: SpID[1,1][2] => V3[2]

[0180] Path 3: If V[3] = 1: SpID[1,1][2] => V3[3]

[0181] Next, after step 712A-1, the first SIMD performs step 714A-1, where the first SIMD compares V2 and V3 and writes the result to the first cache location C1 (e.g., V2 XNOR V3 => C1):

[0182] Dao 0: If V[0]==1: V2[0]XNOR V3[0]=>C1[0]

[0183] Path 1: If V[1]==1:V2[1]XNOR V3[1]=>C1[1]

[0184] Path 2: If V[2]==1:V2[2]XNOR V3[2]=>C1[2]

[0185] Path 3: If V[3]==1:V2[3]XNOR V3[3]=>C1[3]

[0186] As described, the pipeline then repeatedly performs steps 708, 712, and 714 in steps 708A-2, 712A-2, 714A-2, 708A-3, 712A-3, 714A-3, 708A-4, 712A-4, and 714A-4 as described.

[0187] In step 708A-2, the first SIMD is executed:

[0188] Dao 0: C2[0]=>V1[0]

[0189] Tao 1: C2[1]=>V1[1]

[0190] Tao 2: C2[2]=>V1[2]

[0191] Tao 3: C2[3]=>V1[3]

[0192] In step 712A-2, the first SIMD is executed:

[0193] Dao0: SpID[1,2][2]=>V3[0]

[0194] Path 1: SpID[1,2][2]=>V3[1]

[0195] Tao 2: SpID[1,2][2]=>V3[2]

[0196] Tao 3: SpID[1,2][2]=>V3[3]

[0197] In step 714A-2, the first SIMD is executed:

[0198] Dao 0: If V[0]==1: V2[0]XNOR V3[0]=>C2[0]

[0199] Path 1: If V[1]==1:V2[1]XNOR V3[1]=>C2[1]

[0200] Path 2: If V[2]==1:V2[2]XNOR V3[2]=>C2[2]

[0201] Path 3: If V[3]==1:V2[3]XNOR V3[3]=>C2[3]

[0202] In step 708A-3, the first SIMD is executed:

[0203] Dao 0: C3[0]=>V1[0]

[0204] Tao 1: C3[1]=>V1[1]

[0205] Tao 2: C3[2]=>V1[2]

[0206] Tao 3: C3[3]=>V1[3]

[0207] In step 712A-3, the first SIMD is executed:

[0208] Dao0: SpID[1,2][2]=>V3[0]

[0209] Path 1: SpID[1,2][2]=>V3[1]

[0210] Tao 2: SpID[1,2][2]=>V3[2]

[0211] Tao 3: SpID[1,2][2]=>V3[3]

[0212] In step 714A-3, the first SIMD is executed:

[0213] Dao 0: If V[0]==1: V2[0]XNOR V3[0]=>C3[0]

[0214] Path 1: If V[1]==1:V2[1]XNOR V3[1]=>C3[1]

[0215] Path 2: If V[2]==1:V2[2]XNOR V3[2]=>C3[2]

[0216] Path 3: If V[3]==1:V2[3]XNOR V3[3]=>C3[3]

[0217] Although not explicitly stated, the above sequence can continue to be used for all pulse IDs in the first group (g=1) of several delay groups G. In step 708A-4, the first SIMD loads the bit vector from cache position Cg1 of the g1th cache into bit vector register V1 for use in the L SIMD channel of the first SIMD and configures register V1 as the enable bitmask of the first SIMD channel of the first SIMD for subsequent operations; g1 represents a certain number corresponding to the number of pulse IDs in the first group (g=1) of several delay groups G:

[0218] Dao 0: Cg1[0]=>V1[0]

[0219] Dao 1: Cg1[1]=>V1[1]

[0220] Dao 2: Cg1[2]=>V1[2]

[0221] Dao 3: Cg1[3]=>V1[3]

[0222] In step 712A-4, the first SIMD broadcasts the second bit of the last pulse ID SpID[g,i][2] (i = the last one, since it is the last pulse ID) from the first group (g=1) of several delay groups G from memory to the first SIMD channel of the first SIMD to enter the bit vector register V3:

[0223] Tao0: SpID[1, last one][2] => V3[0]

[0224] Tao 1: SpID[1, last one][2] => V3[1]

[0225] Tao 2: SpID[1, last one][2] => V3[2]

[0226] Tao 3: SpID[1, last one][2] => V3[3]

[0227] In step 714A-4, the first SIMD compares V2 and V3 and writes the result to the g1th cache location Cg1 (e.g., V2 XNOR V3 => Cg1), where g1 represents a certain number of pulse IDs corresponding to the first group (g=1) of several delay groups G:

[0228] Dao 0: If V[0]==1: V2[0]XNOR V3[0]=>Cg1[0]

[0229] Path 1: If V[1]==1:V2[1]XNOR V3[1]=>Cg1[1]

[0230] Path 2: If V[2]==1:V2[2]XNOR V3[2]=>Cg1[2]

[0231] Path 3: If V[3]==1:V2[3]XNOR V3[3]=>Cg1[3]

[0232] In a similar manner to that described above, the process continues by comparing the third bit of all pulse IDs in the first group with the third bit of the L synapse IDs also in the first group. The first group is one of several delay groups G. As before, the comparison is conditional on the following: the result of the comparison of the previous bits (first and second bits) is true and recorded in a bit vector stored in cache location Ci. The bit vector in cache location Ci is expected to become increasingly sparse (few 1s) with each repetition; therefore, in some embodiments, the optimization is the compression of these bit vectors.

[0233] Finally, the process continues by comparing the last bit of all pulse IDs in the first group with the last bit of the L synapse IDs that are also in the first group. The first group is one of several delay groups G. As before, the comparison is conditional: the result of the previous bit comparison is true and recorded in the bit vector stored in the cache orientation Ci. By the end of this step, the cache orientation Ci with bit vector 1 will contain matches (which synapse IDs and corresponding synapses received pulses in the first delay group during this time step). At the end of this operation or during this operation, the bitmasks in the cache orientation Ci can be combined (e.g., "ORed") into a single bitmask representing the enabled bits of the L synapse IDs, i.e., which of the L synapse IDs are being targeted.

[0234] Other delay groups can be compared in subsequent steps within this SIMD or can be performed in parallel in other SIMDs. Each group contains synaptic and pulse IDs independent of other groups, so the steps are independent and identical within each group. All delay groups need to undergo this process during each time step.

[0235] In some embodiments, the pipeline will retrieve the bits of the synapse ID from memory only once. This offers an advantage because memory operations are expensive in terms of latency (tens of nanoseconds), especially considering the fact that there are far more synapse IDs than pulses. Therefore, the pulse ID resides in a cache or some other form of fast medium, allowing it to be retrieved quickly (within nanoseconds) and compared rapidly. Thus, in some embodiments, the pipeline can begin reading subsequent bits of the synapse ID while still performing comparisons on previous bits.

[0236] In another embodiment, the pipeline can perform the comparison of the first, second, ... last bits of SyID with the corresponding bits of the pulse ID in parallel, either in a separate SIMD or in a sense amplifier interfacing with the SIMD. This offloads the SIMD from performing these operations, which in this case are performed in the memory / sense amplifier, and the SIMD receives a pulse bitmask ready for further use. Alternatively, raw or simple SIMD operations can be built into each sense amplifier, thus allowing for simple SIMD programming of each sense amplifier array.

[0237] For each delay group of synaptic IDs, a sense amplifier array generates a bitmask indicating which synaptic IDs have matched any of the pulse IDs in the pulse delay bucket. Immediately following the generation of each bitmask, the bitmask can be used for synaptic integration, which in one embodiment is attributed to the summation of charges injected into the neuron's cell body by the synapse making the vesicle membrane permeable. The charge leaks into the neuron proportionally to the synaptic power / weight. This operation (adding charge or current) can be performed in parallel by the same SIMD processor, provided that the SIMD processor's computational power allows it to do so. Synaptic weights can be digital (e.g., 32-bit floating-point numbers) or analog (at least one memory cell has a stable analog value that does not change over time).

[0238] In block 722a, the method determines whether a bit position remains to be analyzed. As discussed, the method processes all cached vectors of all burst identifiers for the current bit position in blocks 708a, 710a, 712a, 714a, 716a, and 720a. In block 722a, once all cached vectors and burst identifiers have been processed, the method moves the bit position until all bit positions have been processed. Therefore, it returns to... Figure 7B The pipeline can be re-executed for each location.

[0239] At the end of the method, the cache orientation C of the bit vector with logic 1 is... i This will contain a match (which synapse IDs and their corresponding synapses received pulses in a given time step within a first delay group). At the end of the method (or during the method), the bitmasks in the cache orientation Ci can be combined (e.g., "ORed") into a single bitmask (SyB) of length n, which represents the enable bits of all L synapse IDs, i.e., which of the L synapse IDs are being targeted. In the illustrated embodiment, the bitmask SyB enables downstream pipeline stages to operate only on those synapse IDs that have been pulsed by the incoming pulse message.

[0240] Other delay groups may be compared in subsequent steps within the SIMD core / processor executing the method, or may be performed in parallel within other SIMD cores / processors. In some embodiments, each group is independent of the synapse and pulse IDs of another group. Therefore, the steps are independent within each group and are identical within each group. In some embodiments, the method is used to process all delay groups during each time step.

[0241] As described, the method retrieves only one bit of the synaptic ID from memory and compares all the bits at once against the corresponding bit positions of all pulse IDs in the SIMD. This offers an advantage because memory operations are costly in terms of latency (typically requiring tens of nanoseconds), especially considering the fact that there are far more synaptic IDs than pulses. Therefore, pulse IDs can reside in a cache or some other form of fast medium, allowing them to be retrieved quickly (e.g., in nanoseconds) and compared rapidly. Thus, the method can begin reading subsequent bits of the synaptic ID while still performing comparisons on previous bits.

[0242] In some embodiments, the method may perform the comparison of the first, second, ... last bits of SyID with the corresponding bits of the pulse ID in parallel, either in a separate SIMD or in a sense amplifier interfaced with the SIMD. This offloads the SIMD from the execution of these operations, which are performed in memory or the sense amplifier, and the SIMD receives a resulting pulse bitmask ready for use. Alternatively, the SIMD operations may be built into each sense amplifier, thus allowing SIMD programming of each sense amplifier array.

[0243] For each delay group of synaptic IDs, a sense amplifier array can generate a bitmask indicating which synaptic IDs have matched any of the pulse IDs in the pulse delay bucket. Immediately following the generation of each bitmask, the bitmask can be used for synaptic integration (as an example) which is the summation of charges injected into the neuron's cell body by synapses making the vesicle membrane permeable. Charge leaks into the neuron proportionally to synaptic potency and / or weight. This operation (adding charge or current) can be performed in parallel by the same SIMD processor. Synaptic weights can be digital (e.g., 32-bit floating-point values) or analog (e.g., at least one memory cell with a stable analog value that does not change over time).

[0244] Figure 7C This is a flowchart illustrating a method for performing synaptic integration in a SIMD or MIMD pipeline according to some embodiments of the present disclosure. In the illustrated embodiments, Figure 7C The method can be executed by a second SIMD array, while Figure 7A The method can be executed by the first SIMD array.

[0245] In block 702b, the method receives a bit vector representing a synaptic bitmask (SyB). In one embodiment, SyB comprises an n-dimensional bitmask vector, where n is the length of the synaptic identifier vector.

[0246] In the illustrated embodiment, the bitmask SyB is as follows: Figure 7A The vector C described in the document is generated and can correspond to a cache. i Each set bit in this type of vector indicates whether the sypanse corresponding to the position of the bit has received a pulse in the current time step.

[0247] Therefore, the bit vector received in block 702b is associated with L synaptic IDs SyID[g,k] of a first group (e.g., g=1) of several delay groups G. In the illustrated embodiment, the method loads the bit vector into register V1 for L SIMD channels of the second SIMD. In some embodiments, the method may configure register V1 as an enable bitmask of the SIMD channels of the second SIMD for subsequent operations.

[0248] In the illustrated embodiment, for each delay bucket of a presynaptic ID, the method generates a bitmask indicating which synaptic IDs have been matched to any of the pulse IDs in the pulse delay bucket, i.e., which synaptic IDs have received a pulse.

[0249] exist Figure 7A The method generates a bit mask (C i )after, Figure 7C The method allows for immediate use of bitmasks for synaptic integration. As an example, the method can perform synaptic integration by summing the charges injected into the neuron's cell body, attributed to the permeability of the synaptic membrane. The charge can leak into the neuron in proportion to the synaptic potency / weight. This operation (adding charge or current) can be performed by... Figure 7A The method can be performed on the same SIMD processor or in parallel by different SIMD processors.

[0250] In block 704b, the method accumulates the synaptic current.

[0251] In the illustrated embodiment, the method accumulates the synaptic currents (i.e., conductances or weights, denoted by the vector SyW[g,k]) from a first group (g=1) of several delay groups G from memory into a SIMD channel of a second SIMD, which is then entered into a vector register V2, where k represents the SIMD channel number and each channel number is associated with a specific neuron ID. In some embodiments, the synaptic currents may be scaled (e.g., multiplied) by a scaled vector V. m In some embodiments, vector Vm It can include the membrane potential of a given neuron.

[0252] Synaptic weights (SyW) can be digital (e.g., 32-bit floating-point values) or analog (e.g., memory cells with stable analog values ​​that do not change over time). In the illustrated embodiment, synaptic weights are assigned in a pattern similar to presynaptic IDs, making their access suitable for SIMD operations predicted by the illustrated bitmask. Thus, after generating the bitmask for the delay bucket, the SIMD processor performing the method can immediately begin accessing the associated weights corresponding to all set bits in the bitmask and accumulating the synaptic currents into a designated register for each neuron. An important observation here is that synaptic weights need to be aligned in the memory column for each neuron to facilitate the accumulation process.

[0253] Synaptic weight vectors can be allocated in a serial manner (similar to the allocation of presynaptic IDs), a bit-parallel manner, or a rectangular manner. In the illustrated embodiments, since the bitmask is typically sparse (e.g., only 5-10% of the set bits in the bitmask), in some embodiments, the method can reduce the area requirement on the SIMD processor by having fewer FMA computation units. Therefore, in some embodiments, bit-parallel allocation can provide improved power benefits and reduce the requirements for area and memory bandwidth. This is especially true where array capabilities allow for offloading or masking certain sections of the array per memory row (e.g., as in HRAM devices). However, in some embodiments, bit-parallel allocation will require a fixed mapping scheme from the bit-serial allocation array of IDs to the bit-parallel allocation array of weights per neuron. In other embodiments, offloading or predictive bit-serial allocation of synaptic weight vectors can also reduce power consumption.

[0254] In the illustrated embodiment, the process in block 704b can be represented as follows:

[0255] V2=V2+(V m ×SyW)

[0256] Where V m SyW represents the scaled vector stored in the SIMD register, and V2 represents the current synaptic weight.

[0257] In some embodiments, when a bitmask for positioning is enabled, the method may only compute the above product. Therefore, the above equation can be more formally expressed as:

[0258]

[0259] Where V2[i] represents the current accumulated weight at position i, V m[i] represents the scaled vector at bit position i, SyW[g,k][i] represents the current synaptic weight of the synaptic identifier k in the delay group g at bit position i, and V1[i] represents the bitmask at bit position i. In some cases, the scaling can be avoided, so this operation is simplified to addition.

[0260] Therefore, each V2[k] will contain the accumulated current of the k-th neuron at the end of the processing of all SyWs for each neuron. In one embodiment, the computation in block 704b can be performed using a Complex Instruction Set Computer (CISC) architecture. Thus, the processing may include a single command of the form: if V[k] == 1: Vm[1,k] x SyW[1,k] + => V2[k], which can be implemented as several Reduced Instruction Set Computer (RISC) micro-operations represented by pseudocode:

[0261] Load Vm[1,k]

[0262] Load SyW[1,k] (in parallel with Vm or subsequently)

[0263] Multiply by Vm[1,k] x SyW[1,k]

[0264] Add the result of Vm[1,k]x SyW[1,k] to V2[k].

[0265] Store the addition result in V2[k].

[0266] In some embodiments, when the method begins at the start of each time step with a given weight vector (SyW[1,k]), V2[k] can be initialized to 0 (706b). Other embodiments may perform other calculations. For example, if SyW[1,k] represents synaptic charge / current, then multiplying by V2[k] can be avoided. m [1,k].

[0267] In some embodiments, the method may restrict each channel number to be associated with a specific neuron ID. In some embodiments, this restriction may be relaxed so that several channels may be associated with a specific neuron ID. In this embodiment, each channel will create a sub-accumulation of current and, at the end of processing all synaptic weights, the results of these sub-accumulations will be summed into a single result per neuron.

[0268] In some embodiments, the order of the received bit vectors is not specified, as long as each bit vector is associated with a specific weight set stored in aligned rows of memory. This is due to the summation of all currents, and therefore the order of this summation within each time step is not critical. Thus, the method implements orderless operation. While the foregoing embodiments receive data from a first SIMD processor, different SIMD processors may execute in different groups. Figure 7A If the method allows data to be received from more than one SIMD processor, then the method can receive data from more than one SIMD processor.

[0269] Figure 7D This is a pipeline timing diagram illustrating a method for performing synaptic integration in a SIMD or MIMD pipeline according to some embodiments of the present disclosure. Figure 7D As explained in the document, it can be executed in a pipelined manner. Figure 7C The method, as described, is for the first bit vector execution blocks 702B-1 and 704B-1, for the second bit vector execution blocks 702B-2 and 704B-2, and so on.

[0270] In step 702B-2, the second SIMD from Figure 7B The first SIMD receiver in the second SIMD receives the first bit vector SyB[g,k] associated with the L synaptic ID SyID[g,k] of the first group (g=1) in a plurality of delay groups G, and loads this bit vector into the bit vector register V1 of the L SIMD channel of the second SIMD. The pipeline configures register V1 as the enable bitmask of the SIMD channel of the second SIMD for subsequent operations:

[0271] Dao 0: SyB[1,0]=>V1[0]

[0272] Dao 1: SyB[1,1]=>V1[1]

[0273] Path 2: SyB[1,2]=>V1[2]

[0274] Path 3: SyB[1,3]=>V1[3]

[0275] In step 704B-1, the second SIMD accumulates the L-synaptic current (i.e., the conductance or weight SyW[g,k] scaled by Vm[k]) from the first group (g=1) of several delay groups G from memory into the SIMD channel of the second SIMD to enter the vector register V2, where k represents the number of SIMD channels, and each channel number is associated with a specific neuron ID:

[0276] Dao 0: If V[0]==1: Vm[0]x SyW[1,0]+=>V2[0]

[0277] Path 1: If V[1]==1: Vm[1]x SyW[1,1]+=>V2[1]

[0278] Path 2: If V[2]==1: Vm[2]x SyW[1,2]+=>V2[2]

[0279] Path 3: If V[3]==1: Vm[3]x SyW[1,3]+=>V2[3]

[0280] Next, in step 702B-2, the second SIMD receives from the first SIMD a second bit vector SyB[g,L+k] associated with the L synaptic ID SyID[g,L+k] of the first group (g=1) in a plurality of delay groups G, and the SIMD loads this bit vector into the bit vector register V1 of the L SIMD channel of the second SIMD and configures register V1 as the enable bitmask of the SIMD channel of the second SIMD for subsequent operations:

[0281] Dao 0: SyB[1,L+0]=>V1[0]

[0282] Path 1: SyB[1,L+1]=>V1[1]

[0283] Path 2: SyB[1,L+2]=>V1[2]

[0284] Path 3: SyB[1,L+3]=>V1[3]

[0285] Next, in step 704B-2, the second SIMD accumulates the L-synaptic currents (i.e., conductances or weighted SyW[g,k] scaled by Vm[k]) from the first group (g=1) of several delay groups G from memory into the SIMD channel of the second SIMD to enter the vector register V2, where k represents the number of SIMD channels, and each channel number is associated with a specific neuron ID. In the illustrated embodiment, Vm[k] remains the same because it is per neuron rather than per synapse:

[0286] Dao 0: If V[0]==1: Vm[0]x SyW[1,L+0]+=>V2[0]

[0287] Path 1: If V[1]==1: Vm[1]x SyW[1,L+1]+=>V2[1]

[0288] Path 2: If V[2]==1: Vm[2]x SyW[1,L+2]+=>V2[2]

[0289] Path 3: If V[3]==1: Vm[3]x SyW[1,L+3]+=>V2[3]

[0290] Steps 702 and 704 described above can continue in batches or increments of L IDs for other synaptic IDs SyID[g,k] in the first group (g=1) of several delay groups G until all bit vectors SyB[g,k] of all IDs in the first group are received. Alternatively, the process can continue for other delay groups G.

[0291] Finally, in step 702B-4, the second SIMD receives from the first SIMD the last bit vector SyB[g,L+k] associated with the last L synaptic IDs SyID[g,lastL+k] of the last group (g=last) in the last of several delay groups G, and the SIMD loads this bit vector into the bit vector register V1 of the L SIMD channel of the second SIMD and configures register V1 as the enable bitmask of the SIMD channel of the second SIMD for subsequent operations:

[0292] Dao 0: SyB[last one, last L+0] => V1[0]

[0293] Path 1: SyB [the last one, the last L+1] => V1[1]

[0294] Path 2: SyB [the last one, the last L+2] => V1[2]

[0295] Path 3: SyB [the last one, the last L+3] => V1[3]

[0296] In step 704B-4, the second SIMD accumulates the last L synaptic current (i.e., the conductance or weight SyW[g, last L+k] scaled by Vm[k]) of the last group (g = last) of several delay groups G from memory into the SIMD channel of the second SIMD and enters it into the vector register V2, where k represents the number of SIMD channels and each channel number is associated with a specific neuron ID. In one embodiment, Vm[k] remains the same because it is per neuron rather than per synapse. In some embodiments, the entire operation is described only for L neurons (i.e., all synaptic weights grouped by delay groups belonging only to L neurons). In one embodiment, the SIMD operation may also overlap memory wait with computation, so multiple batches of L neurons can be processed while waiting for data. Thus, in some embodiments, the pipeline can process another L neurons and restart and perform the same process. Alternatively, another L neurons can be processed in parallel in different SIMDs. Thus, in step 704B-4, the second SIMD performs:

[0297] Dao 0: If V[0]==1: Vm[0]x SyW[the last one, the last L+0]+=>V2[0]

[0298] Path 1: If V[1]==1:Vm[1]x SyW[the last one, the last L+1]+=>V2[1]

[0299] Path 2: If V[2]==1:Vm[2]x SyW[the last one, the last L+2]+=>V2[2]

[0300] Path 3: If V[3]==1:Vm[3]x SyW[the last one, the last L+3]+=>V2[3]

[0301] Additionally, in some embodiments, if, for example, another array (or a set of SIMD arrays) is available and can operate in parallel with the presynaptic ID array, then Figure 7C The method can be with Figure 7A The / 7B method is executed in parallel. In some embodiments, weights and presynaptic identifiers can be co-assigned bit-to-bit, thus allowing bitmasks and weight accumulations to be computed simultaneously and possibly through speculation and prediction.

[0302] In some embodiments, if the SIMD processor has a set of multiplexed arrays with the same bit width as the bit addressing used, then the method can be presumably started in the background and... Figure 7A The matching operation accesses all of the multiplexed array sets in parallel (i.e., without the condition "if V[k] == 1" as previously discussed). The method may then use a wider array to coordinate the allocation of weights and presynaptic identifiers in a bit-serial manner. Alternatively, in some embodiments, when a bitmask is available, the method may begin by accessing only the relevant weights, thus saving power (i.e., with the condition "if V[k] == 1").

[0303] In some embodiments, synaptic current / weight accumulation via vertical integration can be implemented using floating-point adders distributed within a stack of processing elements or via bonded dies. Generally, the entire SIMD processor of a floating-point FMA can be distributed within a stack, thus implementing a unidirectional computing paradigm.

[0304] As previously discussed, bitmasks are typically sparse (i.e., very few binary ones). However, in the illustrated embodiments, each bit of the mask is unique for each neuron. Therefore, in some embodiments, memory may be used that can uniquely access each cell in a column or row. In this embodiment, the method accesses several weights from each column or row at unique locations in the column or row.

[0305] In memory devices where every cell cannot be uniquely accessed (i.e., due to shared word lines), the method can access all bit lines of each word line. In such devices, the method can offload or mask some of the access operations to save power. Alternatively, or in conjunction with the above, the method can utilize these sparse memory accesses for other computations, as discussed herein. Figure 7A Search / match operations and Figure 7C After synaptic integration, the implementation can proceed to solving, such as Figure 7E The neuron model described in the paper (i.e., the membrane equation).

[0306] Figure 7E This is a flowchart illustrating a method for performing neuronal dynamics in a SIMD or MIMD pipeline according to some embodiments of the present disclosure. Figure 7F This is a pipeline diagram illustrating a method for performing neuronal dynamics in a SIMD or MIMD pipeline according to some embodiments of the present disclosure.

[0307] In the illustrated embodiments, during the search / match operation ( Figure 7A ) and synaptic integration operations ( Figure 7C After that, the system can proceed to solve the neuron model (i.e., the membrane equation). As one example, Figure 7E Describe the leaky integral firing (LIF) neuron model described by the following differential equation:

[0308]

[0309] This first-order linear autonomous ordinary differential equation can be solved analytically or through numerical integration in a SIMD array using the numerical form of Equation 1:

[0310]

[0311] In the illustrated embodiment, as will be described below, each processor in the SIMD array runs this numerical equation for its corresponding neuron (e.g., each SIMD channel corresponds to this equation and an active single neuron computes it). In the illustrated embodiment, Δt may be one millisecond and is therefore equal to one and is global for the entire neural network. Other values ​​of Δt may be used.

[0312] As discussed herein, the method performs two multiplications and three additions (subtraction can be replaced by addition of negative variables). In some embodiments, the constant-related parts of the operation can be pre-computed. Therefore, Equation 2 can be calculated as follows. Since Δt is set to -(1), Equation 2 can be rewritten as follows:

[0313]

[0314] Where E L This includes the static membrane potential, also known as the reverse potential of the leakage current, V. m (t) represents the current membrane potential associated with the neuron corresponding to the given neuron identifier, τ m Including membrane time constant, R m Including the simulated membrane resistance of a given neuron, R mt =R m , / τ m I e (t) includes the use of Figure 7C The method calculates and obtains the accumulated synaptic current, E, in the registers of the SIMD array. lt =E L / τ m And τ m =1 / τ m .

[0315] In the illustrated embodiment, the method can pre-calculate the constant portion of Equation 3. Specifically, the method can pre-calculate E at the initial time step (e.g., 702c). Lt R mt and τ inv The value of . Additionally, in some embodiments, the method can also pre-calculate (1-τ). inv The value of ) can also be used in all executions of the method. inv E Lt and R mt The value of .

[0316] In block 702c, the method loads the membrane potential vector (V). m (t)). In one embodiment, a membrane potential vector (V) is loaded from a memory device. m (t)). That is, the membrane potential vector (V m (t) represents the data in a given memory location of the memory device.

[0317] In box 704c, the method broadcasts the first neuron constant (1-τ). inv ), and in box 706c, the method broadcasts the second neuron constant (E). Lt ).

[0318] In block 708c, the method performs a first multiply-accumulate (MAC) operation. In one embodiment, the method calculates V. m (t)×(1-τ inv ))+E Lt The result is then written to register V1.

[0319] In box 710c, the method broadcasts the third neuron constant (R). mt ), and in box 712c, the method broadcasts the fourth neuron constant (V) mt In one embodiment, the fourth neuron constant (V) mt This includes a constant threshold that triggers a reset, in which a delta function pulse occurs and the voltage is reset to its resting potential.

[0320] In block 714c, the method retrieves the accumulated synaptic current (I0) from the vector register of the SIMD processor. E In one embodiment, vector I E Including from Figure 7C The current that is integrated into the synapse during the process.

[0321] In block 716c, the method performs a second MAC operation. In one embodiment, the method calculates (I E ×R mt )+V1 and write the result to register V1.

[0322] like Figure 7E As illustrated, boxes 702c, 704c, 706c, and 708c may overlap with boxes 710c and 712c. Additionally, boxes 714c and 716c may overlap. This is the equation used in the SIMD timing diagram presented here. That is, the first MAC operation (e.g., 708c) can be computed during the first two stages of processing because it has no dependency on other stages. For example, the second SIMD processor may perform this computation during the idle time of waiting for the bitmask in stage 2. Furthermore, in some embodiments, I... e (t) can be zero if the neuron does not receive any pulses during the time step. Therefore, for this type of neuron, once V is calculated... m (t)(1-τ inv )+E Lt Therefore, there is no need to modify V. m (t+1).

[0323] In block 718c, the method performs (as calculated in block 716c) V mt SIMD comparison operation between V1 and V2. In some embodiments, during the solution of V1... m During or after the process, the SIMD processor can test the V pulse. m (That is, the action potential), which is approximately 50 mV in biological neurons (compared to, for example, a resting potential of -60 mV). m(Potential comparison). In one embodiment, this detection is performed via a SIMD comparison operation. In some embodiments, as part of block 716c, the method may transmit the result of the comparison to a pulse router for insertion into a delay group. Specifically, the method determines whether each bit of V1 is greater than a threshold V. mt To calculate the comparison vector.

[0324] In block 720c, the method stores the updated membrane potential (V1) in memory for use with the corresponding neuron. Therefore, in block 720c, the method stores V... m The previous value of (t) is replaced with a calculated value stored in V1. In the illustrated embodiment, the method is re-executed for each neuron / synapse identifier. Additionally, the computation may overlap with memory access portions.

[0325] exist Figure 7F In step 702C-1, the pipeline executes step 702C-1, where the second SIMD loads the L-membrane potential Vm[k] from memory into the SIMD channel of the second SIMD for entry into the vector register V1, where k represents the SIMD channel number, and each channel number is associated with a specific neuron ID. In some embodiments, Vm has already been used by SIMD 2 in stage 2, so it has already been loaded (this optimization can be used to save time and improve performance):

[0326] Dao 0: Vm[0] => V1[0]

[0327] Dao 1: Vm[1]=>V1[1]

[0328] Tao 2: Vm[2]=>V1[2]

[0329] Tao 3: Vm[3]=>V1[3]

[0330] In step 704C-1, the second SIMD broadcasts the neuron constant tau_m_inv_one_minus to the L SIMD channel of the second SIMD to enter the vector register V3, such that each SIMD channel receives tau_m_inv_one_minus. As seen here, this operation can be performed in parallel with Vm loading. The broadcast can occur via a separate bus extending from the SIMD controller or scalar processor to each channel. This operation is much faster than loading from memory. In an alternative, the second SIMD can load the L neuron constant tau_m_inv_one_minus[k] from memory into the SIMD channel of the second SIMD to enter the vector register V3, where k represents the SIMD channel number, and each channel number is associated with a specific neuron ID. This operation is useful when the neuron constant tau_m_inv_one_minus is different for each neuron:

[0331] Channel 0: tau_m_inv_one_minus=>V3[0]

[0332] Track 1: tau_m_inv_one_minus=>V3[1]

[0333] Channel 2: tau_m_inv_one_minus=>V3[2]

[0334] Channel 3: tau_m_inv_one_minus=>V3[3]

[0335] In step 706C-1, the second SIMD broadcasts the neuron constant Elt to the L SIMD channel of the second SIMD for entry into the vector register V4, such that each SIMD channel receives Elt. As illustrated, this operation can be performed in parallel with Vm loading. In some embodiments, the broadcast can occur via a separate bus extending from the SIMD controller or scalar processor to each channel. In some embodiments, this operation is much faster than loading from memory. Alternatively, the second SIMD can load the L neuron constant Elt[k] from memory into the SIMD channel of the second SIMD for entry into the vector register V4, where k represents the SIMD channel number, and each channel number is associated with a specific neuron ID. This operation is useful when the neuron constant Elt is different for each neuron:

[0336] Tao 0: Elt => V4[0]

[0337] Tao 1: Elt => V4[1]

[0338] Tao 2: Elt => V4[2]

[0339] Tao 3: Elt => V4[3]

[0340] In step 708C-1, SIMD performs a vector register multiplication and accumulation: V1 × V3 + V4, and writes the result to V1.

[0341] Channel 0: V1[0]x V3[0]+V4[0]=>V1[0]

[0342] Track 1: V1[1]x V3[1]+V4[1]=>V1[1]

[0343] Track 2: V1[2]x V3[2]+V4[2]=>V1[2]

[0344] Lane 3: V1[3]x V3[3]+V4[3]=>V1[3]

[0345] In step 710C-1, the second SIMD broadcasts the neuron constant Rmt to the L SIMD channels of the second SIMD for entry into vector register V5, such that each SIMD channel receives Rmt. Alternatively, the second SIMD can load the L neuron constant Rmt[k] from memory into the SIMD channels of the second SIMD for entry into vector register V5, where k represents the SIMD channel number, and each channel number is associated with a specific neuron ID. This operation is useful when the neuron constant Rmt is different for each neuron:

[0346] Dao 0: Rmt=>V5[0]

[0347] Dao 1: Rmt=>V5[1]

[0348] Tao 2: Rmt=>V5[2]

[0349] Tao 3: Rmt=>V5[3]

[0350] In step 712C-1, the second SIMD broadcasts the neuron constant Vmt (Vm threshold potential) to the LSIMD channel of the second SIMD for entry into the vector register V4, such that each SIMD channel receives Vmt. Alternatively, the SIMD can load the L neuron constant Vmt[k] from memory into the SIMD channel of the second SIMD for entry into the vector register V4, where k represents the SIMD channel number, and each channel number is associated with a specific neuron ID. This operation is useful when the neuron constant Vmt is different for each neuron:

[0351] Dao 0: Vmt => V4[0]

[0352] Dao 1: Vmt => V4[1]

[0353] Dao 2: Vmt => V4[2]

[0354] Dao 3: Vmt=>V4[3]

[0355] In step 714C-1, the accumulated synaptic current Ie(t)[k] associated with the L neuron is received in the second SIMD receive vector register V2, where k represents the SIMD channel number, and each channel number is associated with a specific neuron ID. In some embodiments, Ie(t)[k] may be received from another SIMD or may be received from another SIMD. Figure 7D The computation is performed via this SIMD during pipelined execution and stored in V2:

[0356] Channel 0: Ie(t)[0]=>V2[0]

[0357] Track 1: Ie(t)[1]=>V2[1]

[0358] Channel 2: Ie(t)[2]=>V2[2]

[0359] Channel 3: Ie(t)[3]=>V2[3]

[0360] In step 716C-1, the second SIMD performs a vector register multiplication and accumulation: V2 × V5 + V1, and writes the result to V1.

[0361] Dao 0: V2[0] x V5[0] + V1[0] => V1[0]

[0362] Tao 1: V2[1]x V5[1]+V1[1]=>V1[1]

[0363] Tao 2: V2[2] x V5[2] + V1[2] => V1[2]

[0364] Tao 3: V2[3]x V5[3]+V1[3]=>V1[3]

[0365] In this operation, the second SIMD updates Vm based on the received synaptic current Ie(t). If the current is 0 (the neuron has not received any impulses), then this operation can be masked or is still correct computationally because it is a 0 update. In some embodiments, the entire operation is performed here only for L neurons (i.e., all Vm updates belong to only L neurons). Therefore, to process another L neurons, the second SIMD will restart and perform the same process. The second SIMD can also overlap memory waits with computation, so multiple batches of L neurons can be processed while waiting for data. Alternatively, another L neurons can be processed in parallel in different SIMDs. In some models, the synaptic current is continuous (the value of the synaptic current from each neuron in the current step is adjusted to serve as the initial value of the synaptic current in the next time step). In such cases, additional computational steps are required to adjust the value of the synaptic current and store it in memory for use in the next time step. In this case, the initial value of the synaptic current can be used in the next step.

[0366] In step 718C-1, the second SIMD compares V1 with V4, and writes a 1 to the bit vector register Vb1 if V1 > V4, otherwise writes a 0. The bit vector register Vb1 now holds the postsynaptic pulses generated by the neuron, with each bit in Vb1 corresponding to the local neuron location (local neuron ID):

[0367] Dao 0: V1[0]>V4[0]=>Vb1[0]

[0368] Tao 1: V1[1]>V4[1]=>Vb1[1]

[0369] Tao 2: V1[2]>V4[2]=>Vb1[2]

[0370] Tao 3: V1[3]>V4[3]=>Vb1[3]

[0371] In some embodiments, after step 718C-1 (not shown in pipeline stage 5), the second SIMD may broadcast the current time step TS_current to memory TS[k] for all neurons to generate pulses, where k represents the SIMD channel number, and each channel number is associated with a specific neuron ID:

[0372] Tao 0: If Vb1[0]: TS_current => TS[0]

[0373] Path 1: If Vb1[1]: TS_current=>TS[1]

[0374] Path 2: If Vb1[2]: TS_current=>TS[2]

[0375] Path 3: If Vb1[3]: TS_current=>TS[3]

[0376] Finally, in step 720C-1, the second SIMD stores the L-membrane potential Vm[k] from the vector register V1 into memory, where k represents the SIMD channel number, and each channel number is associated with a specific neuron ID:

[0377] Dao 0: V1[0] => Vm[0]

[0378] Tao 1: V1[1]=>Vm[1]

[0379] Tao 2: V1[2]=>Vm[2]

[0380] Tao 3: V1[3]=>Vm[3]

[0381] The use of the LIF model is exemplary. In some embodiments, the method may use different neuron models, such as resonate-and-fire, Izhikevich, Hodgkin-Huxley, FitzHugh-Nagumo, Morris-Lecar, Hindmarsh-Rose, and others. In other embodiments, the method may use different models for different neurons or apply even more complex numerical integration methods (e.g., Runge-Kutta, Parker-Sochacki) to solve these equations.

[0382] While the numerical method is publicly available, neuron models can be implemented using simulated values ​​and / or states. For example, a series of leaky DRAM caps (e.g., partial DRAM columns) can be used to store V... m Modeling. The synaptic current can then be injected into that cap array, and the potential can be measured by a sensing amplifier and tested against the pulse. In some embodiments, the SIMD array hardware can be designed to implement both analog and digital versions, and configured to operate in either one.

[0383] In the illustrated embodiment, upon detecting a pulse, the ID of the neuron generating the pulse is immediately reported to a local filter / router, which prepares pulse descriptors for all pulse-generating neurons and participates in a pulse broadcast operation, which is being broadcast throughout the network. The method can also locally filter the pulses within this SNN block and distribute them locally to relevant delay buckets (or delay groups). Therefore, the broadcast operation can begin within a fraction of a 1-millisecond cycle time for all memory arrays in the network. Because V m The equations are computed using SIMD for the entire array, so postsynaptic pulses are detected and generated in parallel for all neurons. After all pulses are sent to the network, the hardware can immediately send a mask message containing the number of pulses generated, allowing all receiving routers to potentially perform masking, as well as other masking of other examples of this component. These mask messages, along with all pulse messages, may also contain the component ID.

[0384] Figure 8This description describes an example networking system 1100 comprising a node cluster 1102 consisting of multiple interconnected nodes 100, according to some embodiments of the present disclosure. A node 100 may include a controller 109 and various memory segments integrated together in a single memory device. The single memory device may be fabricated on a single die or may be a stack of multiple dies. Each node 100 may interface with multiple other nodes in the node cluster 1102 to implement a Memory-based Networking (SNN). An SNN is a computer-implemented memory-based system that is modeled after a Memory-based Networking (BNN) to process information. The node cluster 1102 may be a cluster of nodes 100 within a router 800 or may be an array of routers 800, each containing one or more nodes 100.

[0385] Figure 8 This describes an instance of a computing system 1103, which is part of the networking system 1100. Figure 8 This illustrates how computing system 1103 can be integrated into various machines, devices, and systems, such as Internet of Things (IoT) devices, mobile devices, communication network devices and equipment (e.g., see base station 1130), electrical appliances (e.g., see electrical appliance 1140), and vehicles (e.g., see vehicle 1150)).

[0386] The computing devices of computing system 1103 and networking system 1100 (e.g., see computing devices 1122a, 1122b, 1122c, and 1122d) can be communicatively coupled to one or more communication networks 1120. Computing system 1103 includes, for example, a bus 1106, a controller 1108 (e.g., a CPU), other memory 1110, a network interface 1112, a storage system 1114, other components 1116 (e.g., any type of component found in mobile or computing devices, such as GPS components, input / output (I / O) components (such as various types of user interface components), sensors, cameras, etc.), and a node cluster 1102 implementing a SNN. Other components 1116 may also include one or more user interfaces (e.g., GUI, auditory user interface, haptic user interface, etc.), a display, different types of sensors, haptic, audio, and / or visual input / output devices, additional dedicated memory, one or more additional controllers (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), a neural processor), or any combination thereof. Bus 1106 communicatively couples controller 1108, another memory 1110, network interface 1112, data storage system 1114, and other components 1116, and in some embodiments, such components may be coupled to node cluster 1102. For example, mesh architecture 132 may be coupled to bus 1106.

[0387] The computing system 1103 includes a computer system having a controller 1108, other memories 1110 (e.g., random access memory (RAM), read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (e.g., synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM)), static random access memory (SRAM), cross-point or crossbar memory, crossbar memory, flash NAND or flash NOR, etc.), a cluster of nodes 1102, and a data storage system 1114, which can communicate with each other via a bus 1106 (which may include multiple buses).

[0388] in other words, Figure 8 A block diagram of a computing device 1122 having a computer system in which embodiments of the present disclosure are operable. In some embodiments, the computer system may include a set of instructions for causing the machine, when executed, to perform at least partially any or more of the methods discussed herein. In such embodiments, the machine may be connected (e.g., networked via network interface 1112) to other machines in a local area network (LAN), intranet, extranet, and / or the Internet (e.g., see communications network 1120). The machine may operate as a peer-to-peer (or distributed) network machine in a peer-to-peer (or distributed) network environment or as a server or client machine in a cloud computing infrastructure or environment, operating at the capacity of a server or client machine in a client-server network environment.

[0389] Controller 1108 represents one or more general-purpose processing devices, such as microprocessors, central processing units, etc. More specifically, the processing device may be a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, a Single Instruction Multiple Data (SIMD) microprocessor, a Multiple Instruction Multiple Data (MIMD) microprocessor, or a processor implementing other instruction sets, or a processor implementing a combination of instruction sets. Controller 1108 may also be one or more special-purpose processing devices (e.g., ASICs), programmable logic (e.g., FPGAs), digital signal processors (DSPs), network processors, etc. Controller 1108 is configured to execute instructions for performing the operations and steps discussed herein. Controller 1108 may additionally include network interface devices, such as network interface 1112, for communication via one or more communication networks (e.g., network 1120).

[0390] Data storage system 1114 may include machine-readable storage media (also referred to as computer-readable media) storing one or more instruction sets or software embodying any one or more methods or functions described herein. Data storage system 1114 may have execution capability, for example, it may at least partially execute instructions residing in the storage system. The instructions may also be wholly or at least partially residing in at least one of the other memory 1110 and the node cluster 1102 and / or the controller 1108, which also constitutes the machine-readable storage media, during execution. The other memory 1110 may be or include the main memory or system memory of computing device 1122.

[0391] As mentioned, the networking system 1100 includes computing devices, and each computing device may include one or more buses, controllers, memory, network interfaces, storage systems, and other components. Additionally, Figure 8 Each of the computing devices shown and described herein may include or be part of a mobile device or similar device, such as a smartphone, tablet computer, IoT device, smart TV, smartwatch, glasses or other smart home appliance, in-vehicle infotainment system, wearable smart device, game console, PC, digital camera, or any combination thereof. As shown, the computing device may connect to network 1120, which includes at least a device-local network (e.g., Bluetooth, etc.), a wide area network (WAN), a local area network (LAN), an intranet, a mobile wireless network such as 4G or 5G, an extranet, the Internet, and / or any combination thereof. In some embodiments, as shown under connection 1119, node cluster 1102 may include at least one network interface enabling it to communicate individually with other devices via communication network 1120. For example, mesh architecture 132 may be coupled to communication network 1120. In this respect, memory modules or memory module systems of node cluster 1102 may have their own network interfaces enabling such components to communicate individually with other devices via communication network 1120.

[0392] Each of the computing devices described herein may be, or be replaced by, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network device, a server, a network router, a switch or bridge, or any machine capable of executing a set of instructions (sequentially or otherwise) specifying the actions to be performed by this machine.

[0393] Moreover, although targeting Figure 8The computing system 1103 illustrated herein describes a single machine, but the term "machine" should also be considered to include any collection of machines that individually or collectively execute instruction sets (or multiple instruction sets) to perform one or more of the methods or operations discussed herein. Furthermore, each of the illustrated computing devices and computing systems may each include at least a bus and / or motherboard, one or more controllers (e.g., one or more CPUs), main memory that may include temporary data storage devices, at least one type of network interface, a storage system that may include permanent data storage devices, and / or any combination thereof. In some multi-device embodiments, one device may perform some parts of the methods described herein and then send the completed results via a network to another device, allowing the other device to continue with the remaining steps of the methods described herein.

[0394] Although the memory, controller, and data storage device portions are shown as separate portions in the exemplary embodiments, each portion should be considered to include one or more portions capable of storing instructions and performing their respective operations. The term "machine-readable storage medium" should also be considered to include any medium capable of storing or encoding a set of instructions executable by a machine and causing the machine to perform any one or more of the methods of this disclosure. The term "machine-readable storage medium" should therefore be considered to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0395] Unless otherwise specifically stated, the delimited language such as the phrase “at least one of X, Y or Z” is generally understood, as used in the context, to represent items, terms, etc., that may be X, Y or Z, or any combination thereof (e.g., X, Y and / or Z). Therefore, such delimited language is generally not intended and should not imply that certain embodiments require each of at least one of X, at least one of Y, or at least one of Z to be present.

[0396] Some parts of the previously described algorithms and symbolic representations of operations on data bits within computer memory have been presented. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing most effectively communicate the essence of their work to others skilled in the art. In this document, and generally in general, an algorithm is conceived as a self-consistent sequence of operations that produce a desired result. An operation is an operation that requires physical manipulation of a physical quantity. Typically (but not always), these quantities take the form of electrical or magnetic signals that can be stored, combined, compared, and otherwise manipulated. It has been shown that it is sometimes convenient to refer to these signals as bits, values, elements, symbols, characters, items, numbers, etc., primarily for common use.

[0397] However, it should be remembered that all these and similar terms will be associated with appropriate physical quantities and are merely convenient notations for application to those quantities. This disclosure can refer to the actions and processes of a computer system or similar electronic computing device that manipulate and transform data represented as physical (electronic) quantities in the registers and memories of a computer system into other data similarly represented as physical quantities in the computer system's memory or registers or other such information storage systems.

[0398] This disclosure also relates to apparatus for performing the operations described herein. Such apparatus may be specifically constructed for the desired purpose, or may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in a computer. Such computer programs may be stored in computer-readable storage media, such as any type of disk including floppy disks, optical disks, read-only optical disks (CD-ROMs) and magneto-optical disks, read-only memory (ROMs), random access memory (RAMs), erasable programmable read-only memory (EPROMs), electrically erasable programmable read-only memory (EEPROMs), magnetic cards or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.

[0399] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used with the programs taught herein, or said general-purpose systems may prove convenient for constructing more specialized devices to perform the methods. The structures of various such systems will be presented as described below. Furthermore, this disclosure is described without reference to any particular programming language. It should be understood that the teachings of this disclosure as described herein can be implemented using various programming languages.

[0400] This disclosure may be provided as a computer program product or software, which may include machine-readable media on which instructions are stored for programming a computer system (or other electronic device) to perform processes according to this disclosure. Machine-readable media includes any means for storing information in a machine-readable (e.g., computer-readable) form. In some embodiments, machine-readable (e.g., computer-readable) media includes machine-readable (e.g., computer-readable) storage media, such as read-only memory (“ROM”), random access memory (“RAM”), disk storage media, optical storage media, flash memory components, etc.

[0401] It should be emphasized that the embodiments described above in this disclosure are merely possible examples of implementations set forth for the purpose of clearly understanding the principles of this disclosure. Many variations and modifications can be made to the embodiments described above without substantially departing from the spirit and principles of this disclosure. All such modifications and variations are intended to be included herein, within the scope of this disclosure, and protected by the appended claims.

Claims

1. An apparatus for processing data, comprising: Multiple parallel processors, wherein the multiple parallel processors are configured to: Perform a search and match operation, which loads and compares multiple synaptic identifier bit strings and multiple pulse identifier bit strings to generate multiple bit masks; The synaptic integration phase is performed, in which multiple synaptic current vectors are generated based on the multiple bit masks, and the synaptic current vectors are associated with corresponding synthetic neurons. Solve the neural membrane equations for each of the synthetic neurons; Update one or more state variables associated with the synthetic neuron; and The data values ​​indicating the one or more state variables are written to the memory device.

2. The apparatus according to claim 1, wherein the search and matching operation includes: Perform an XNOR operation between the first bit of multiple synaptic identifier vectors and the first bit of the first pulse identifier; and The result of the XNOR operation is stored in the first position.

3. The apparatus according to claim 2, wherein the search and matching operation includes: Select multiple additional pulse identifiers, and for each additional pulse identifier, perform an XNOR operation between the first bit of the multiple synaptic identifier vectors and the first bit of the additional pulse identifier; The result of the XNOR operation is stored in a corresponding cache location, the number of cache locations being equal to the number of synaptic identifiers and the length of the cache location being equal to the number of synaptic identifier vectors.

4. The apparatus of claim 3, wherein the search and matching operation comprises: Load the remaining bits of the synaptic identifier; An XNOR operation is performed between the remaining bits of the synaptic identifier and the corresponding remaining bits of each of the pulse identifiers; and The result of the XNOR operation is stored in the corresponding cache location.

5. The apparatus of claim 1, wherein performing the synaptic integration phase includes disabling or enabling the execution of the synaptic integration phase based on the plurality of bitmasks.

6. The apparatus of claim 1, wherein performing the synaptic integration phase includes accumulating the synaptic currents associated with each synthetic neuron based on synaptic weights.

7. The apparatus of claim 1, wherein solving the neural membrane equations comprises solving a leaky integral excitation LIF model for each synthetic neuron.

8. The apparatus of claim 1, wherein solving the LIF model for each synthetic neuron includes preloading a plurality of neuron constants.

9. The apparatus of claim 8, wherein solving the LIF model for each synthetic neuron comprises performing a plurality of multiplication-accumulation operations using the neuron constant, a plurality of current membrane potentials, and a plurality of synaptic current vectors.

10. The apparatus of claim 8, wherein the parallel processor comprises a single-instruction multiple-data (SIMD) or multiple-instruction multiple-data (MIMD) processor.

11. A method for processing data, comprising: Search and matching operations are performed by a parallel processor, which load and compare multiple synaptic identifier bit strings and multiple pulse identifier bit strings to generate multiple bit masks. The parallel processor performs a synaptic integration phase, which generates multiple synaptic current vectors based on the multiple bit masks, and the synaptic current vectors are associated with corresponding synthetic neurons. The parallel processor solves the neural membrane equations for each of the synthetic neurons. and The parallel processor updates the state variables associated with the synthetic neuron, and the state variables are stored in a memory device.

12. The method of claim 11, wherein performing the search and matching operations comprises: Perform an XNOR operation between the first bit of multiple synaptic identifier vectors and the first bit of the first pulse identifier; and The result of the XNOR operation is stored in the first position.

13. The method of claim 12, wherein performing the search and matching operations comprises: Select multiple additional pulse identifiers, and for each additional pulse identifier: perform an XNOR operation between the first bit of the multiple synaptic identifier vectors and the first bit of the additional pulse identifier; and The result of the XNOR operation is stored in the corresponding cache location, the number of cache locations being equal to the number of pulse identifiers and the length of the cache location being equal to the number of synaptic identifier vectors.

14. The method of claim 13, further comprising: Load the remaining bits of the synaptic identifier; Perform an XNOR operation between the remaining bits of the synaptic identifier and the corresponding remaining bits of each pulse identifier; and The result of the XNOR operation is stored in the corresponding cache location.

15. The method of claim 11, wherein performing the synaptic integration phase includes disabling or enabling the execution of the synaptic integration phase based on the plurality of bitmasks.

16. The method of claim 11, wherein performing the synaptic integration phase includes accumulating the synaptic currents associated with each synthetic neuron based on a scaled vector and current synaptic weights.

17. The method of claim 11, wherein solving the neural membrane equations comprises solving a leaky integral firing LIF model for each synthetic neuron.

18. The method of claim 11, wherein solving the LIF model for each synthetic neuron includes preloading a plurality of neuron constants.

19. The method of claim 18, wherein solving the LIF model for each synthetic neuron comprises performing a plurality of multiplication-accumulation operations using the neuron constant, a plurality of current membrane potentials, and a plurality of synaptic current vectors.

20. A non-transitory computer-readable storage medium for tangibly storing computer program instructions executable by a parallel processor, the computer program instructions defining the following steps: Perform a search and match operation, which loads and compares multiple synaptic identifier bit strings and multiple pulse identifier bit strings to generate multiple bit masks; The parallel processor performs a synaptic integration phase, which generates multiple synaptic current vectors based on the multiple bit masks, and the synaptic current vectors are associated with corresponding synthetic neurons. The parallel processor solves the neural membrane equations for each of the synthetic neurons. and The parallel processor updates the state variables associated with the synthetic neuron, and the state variables are stored in a memory device.

Images