Network structure configurable brain-like chip

Abstract

The application discloses a kind of network structure configurable brain chip architecture, including M1 macro neural computing core, element-intercross switch array routing structure and output error calculation unit, M1 macro neural computing core is connected with element-intercross switch array routing structure respectively, element-intercross switch array routing structure is connected with output error calculation unit;The pulse event transmitted between the macro neural computing core is encoded as AER data packet of AER format, wherein AER data packet includes the address and time stamp of the neuron of transmitting pulse signal;The pulse data packet is transmitted between the macro neural computing core by element-intercross switch array routing structure, and the pulse data packet emitted from macro neural computing core can be routed to any macro neural computing core.The application can realize high speed, high precision on-chip real-time learning, while reducing pulse routing overhead, also ensure the scalability of on-chip network size and certain network mapping flexibility.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence and brain-like intelligent chip, and particularly relates to a brain-like chip architecture with configurable network structure. BACKGROUND

[0002] At present, a spiking neural network (SNN) with high computing efficiency is attracting more and more attention in the field of artificial intelligence. Compared with an artificial neural network (ANN) involving a large number of dense matrix operations, a spiking neural network model closely simulates the functional mechanism of the human cerebral cortex, represents, transmits and processes sensory data in the form of sparse electrical pulses, and performs operations based on sparse pulse events in space and time, which can greatly improve the energy efficiency of intelligent systems. Since the traditional von Neumann processor cannot efficiently process such irregular pulse events under the conventional instruction flow mechanism, research institutions have designed many pulse-based neuromorphic brain-like chips, including SpiNNaker of the University of Manchester, TmeNorth of IBM, Neurogrid of Stanford University and Loihi chip of Intel, etc. The goal of such chips is to build a large-scale simulation platform of a multi-chip array system to complete neuroscience research and data center cognitive tasks, and the chip area ranges from tens to hundreds of square millimeters, showing high complexity and programmability, but at the cost of large chip area, high cost and relatively high power consumption.

[0003] Since the edge intelligent system has strict requirements on the chip area, cost and energy efficiency, researchers have developed many small neuromorphic brain-like chips for resource-limited edge intelligent systems, including MorphIC and ODIN chips of the University of Leuven and Darwin chip of Zhejiang University, etc. These chip architectures and circuits are highly customized and optimized for specific tasks (such as visual recognition), customized spiking neuron models, SNN topologies and synaptic plasticity, although they achieve minimal area or extremely high energy efficiency, but the scalability and configurability of the chips are severely limited. Among them, some chips cannot implement on-chip learning, while the on-chip learning function is crucial in uncontrolled edge intelligent application scenarios; some chips can only train shallow SNNs through simple on-chip learning rules, with low visual recognition accuracy and very limited application scenarios; some chips implement deep SNN learning algorithms based on layer-by-layer error backpropagation on-chip, greatly improving the visual recognition accuracy, but the chip computing overhead is huge, the energy consumption is high and the learning process has high latency.

[0004] Therefore, it is necessary to develop a brain-like chip architecture with configurable network structure. SUMMARY

[0005] The application aims to provide a network structure configurable brain-like chip architecture which can realize high-speed and high-precision on-chip real-time learning, reduce pulse routing overhead, and ensure the scalability of on-chip network and certain network mapping flexibility.

[0006] The network structure configurable brain-like chip architecture comprises M1 macro neural computing cores, a meta-crossbar array routing structure and an output error calculation unit, the M1 macro neural computing cores are connected with the meta-crossbar array routing structure, and the meta-crossbar array routing structure is connected with the output error calculation unit.

[0007] The pulse events transmitted between the macro neural computing cores are encoded into AER data packets in AER format, wherein the AER data packets contain the address and timestamp of the neuron transmitting the pulse signal.

[0008] The pulse data packets are transmitted between the macro neural computing cores through the meta-crossbar array routing structure, and the pulse data packets emitted from the macro neural computing cores can be routed to any macro neural computing core.

[0009] Optionally, the macro neural computing core comprises M2 parallel micro neural computing cores, and each micro neural computing core can be responsible for the calculation of N L / M2 LIF neurons, for accelerating the calculation and processing tasks of each layer of FC-SNN.

[0010] The micro neural computing core comprises:

[0011] a neural processing unit for being responsible for updating the state of each LIF neuron and emitting a pulse;

[0012] a weight updating unit for being responsible for receiving an error and updating the synaptic weight of the neuron;

[0013] a neuron state and weight memory for being responsible for storing the state information of the neuron and the synaptic weight, the neuron state and weight memory being connected with the neural processing unit and the weight updating unit respectively;

[0014] an output AER buffer for buffering the pulse data emitted by the neural processing unit, the output AER buffer being connected with the neuron state and weight memory.

[0015] Optionally, the macro neural computing core further comprises:

[0016] a register lookup table-based leakage unit for calculating and storing the intermediate variables required in the leakage process of the membrane potential of the computing neuron, the register lookup table-based leakage unit being connected with the neural processing unit of each micro neural computing core in the macro neural computing core respectively.

[0017] an input AER buffer for buffering input spike data, the input AER buffer being connected with the register lookup table based leak unit;

[0018] an output AER dispatcher for dispatching input / output spike data, the output AER dispatcher being connected with the output AER buffer of each micro neural computing core in the macro neural computing core;

[0019] and a routing node connected with the weight update unit, the input AER buffer and the output AER dispatcher of each micro neural computing core in the macro neural computing core respectively.

[0020] Optionally, the whole chip architecture processing procedure comprises a forward phase and a backward phase.

[0021] The macro neural computing core independently completes the calculation in an event-driven manner in the forward phase, i.e. processing upon receiving input spike events without synchronization.

[0022] The macro neural computing core completes the synaptic weight update of neurons in a synchronous array parallel manner in the backward phase.

[0023] The micro neural computing core operates in an array parallel manner in both the forward phase and the backward phase.

[0024] Optionally, the meta-crossbar array routing structure comprises a routing node network, M1+1 AER dispatchers and M1+1 switch connection configuration registers, the routing node network comprises (M1+1)×(M1+1) routing node switches, and the routing node switches in each column are connected in sequence, and the routing node switches in each row are connected in sequence.

[0025] Each routing node switch in each column is connected with the AER dispatcher and the switch connection configuration register of the column respectively through an AND gate, and the routing node switch is controlled to be turned on or off through the switch connection configuration register and the AER dispatcher.

[0026] Optionally, the register lookup table based leak unit comprises a first adder and a register based exponential lookup table, the first adder being connected with the register based exponential lookup table; when AER(i, t) reaches the macro neural computing core, Δt=t-t pre is calculated through the first adder, wherein i represents the source neuron index, t is the spike timestamp, t prc is the spike timestamp of the last event, and the obtained Δt is used as an address to access the register based exponential lookup table initialized in advance to obtain a leak term.

[0027] Optionally, the neural processing unit comprises a first multiplier, a second adder, a first comparator, a fire_flag_regs register and a second comparator;

[0028] The first multiplier is connected with the second adder and a register-based exponential lookup table respectively, the first comparator is connected with the second adder, the fire_flag_regs register is connected with the first comparator, and the second comparator is connected with the second adder.

[0029] The first multiplier is used to multiply the leakage term and the old membrane potential V m，j of the neuron j to obtain the leakage membrane potential of the neuron j; the second adder is used to add the leakage membrane potential of the neuron j and the synaptic weight w ij to obtain the updated membrane potential updated V m，j of the neuron j; the first comparator is used to compare the updated membrane potential updated V m，j with a threshold value V th , and when updated V m，j >V th , a pulse AER_out is emitted; and the address of the neuron emitting the pulse is recorded in the fire_flag_regs register file of the neural processing unit, and the fire_flag_regs register is accessed before each time the pulse is emitted, and if the current neuron j has not emitted a pulse, the pulse is allowed to be emitted.

[0030] When each event arrives and the membrane potential is updated, the second comparator is used to determine whether the current membrane potential is greater than the maximum value V max，j of the previously stored membrane potential, and if yes, V max，j is updated, and the current timestamp is synchronously updated as t max，j and stored in the neuron state and weight memory.

[0031] Optionally, the output error calculation unit comprises an output layer neuron error calculation module and an output error register connected with the output layer neuron error calculation module.

[0032] The output layer neuron error calculation module calculates the error of each output layer neuron and stores it in the output error register, and broadcasts it to the micro neural computing cores of each macro neural computing core after all the output errors are calculated.

[0033] Optionally, the weight update unit comprises a 16-way 2b*1b low-cost multiplier array, a single-cycle 16-to-1 adder tree and a local error accumulation register.

[0034] The 16-way 2bx1b low-cost multiplier array and the single-cycle 16-to-1 adder tree are connected to implement a multiply-add operation.

[0035] The local error accumulation register is connected with the single-cycle 16-to-1 adder tree to store the intermediate result of the multiply-add operation, and the value in the local error accumulation register is the local error local_error(j) of the current neuron j after all multiply-add operations are completed.

[0036] Optionally, the weight update unit further comprises a third adder, a second multiplier, a shifter, an 8-bit linear feedback shift register, a third comparator, a fourth adder and a fifth adder.

[0037] The third adder is connected with a register-based exponential lookup table; the second multiplier is connected with the register-based exponential lookup table, the local error accumulation register and the shifter respectively; the third adder obtains the time stamp t max，j and the pulse time stamp t i subtracts Δt to obtain an exponential term, and multiplies the exponential term with the local error local_error(j) of the hidden layer neuron through the second multiplier, and shifts right by L bits to obtain a 16-bit weight update amount Δw ij .

[0038] The third comparator is connected with the shifter and the 8-bit linear feedback shift register respectively, the fourth adder is connected with the shifter, the third comparator and the fifth adder respectively; after obtaining the 16-bit weight update amount Δw ij , the low 8-bit part Δw ij [7:0] of Δw ij is compared with an 8-bit unsigned random number R generated by the 8-bit linear feedback shift register through the third comparator, the high 8-bit part Δw ij of the updated Δw ij is obtained through the fourth adder and the fifth adder, and the updated weight w is obtained by adding the old weight w read from the neuron state and weight memory, and is written into the neuron state and weight memory.

[0039] The present application has the following advantages:

[0040] (1) The network structure configurable brain-like chip architecture proposed in the application integrates layered multi-core chip architecture, hybrid parallel processing mechanism, error broadcast mechanism, Meta-Crossbar (Meta-Crossbar) pulse routing mechanism, full event-driven mechanism, on-chip weight random update and other technologies, so that the chip realizes the integration of "storage and calculation", can realize high-speed and high-precision on-chip real-time learning, and at the same time reduces the pulse routing overhead, and ensures the scalability and certain network mapping flexibility of the on-chip network size.

[0041] (2) The Meta-Crossbar routing structure proposed in the application realizes configurable implicit connection in the core, only a small number of parameter registers are needed, and full connection (FC) in SNN can be realized, without storing all possible connection relationships and weights, greatly reducing the on-chip storage resource consumption and chip area and energy consumption. At the same time, the Meta-Crossbar pulse routing mechanism enables efficient transmission of pulse events between processing cores, compared with the traditional on-chip network pulse routing method, the huge routing overhead of pulse transmission between chips is relieved. At the same time, the design of the Meta-Crossbar routing architecture also ensures the scalability and certain network mapping flexibility of the inter-chip / on-chip network size.

[0042] (3) The application proposes a full event-driven pulse processing unit, in which the neural processing unit is activated only when the input pulse event arrives, and the pulse neurons responsible for it execute operations in sequence; at the same time, an exponential lookup table based on a register is used instead of complex exponential operation, which can minimize the calculation overhead and energy consumption.

[0043] (4) The on-chip learning circuit module proposed in the application adopts error broadcast, on-chip weight random update learning and other technologies, which greatly reduces the chip area, improves the chip storage resource utilization rate, reduces the chip cost, and enables the chip to realize fast on-chip learning while ensuring that the chip recognition accuracy almost does not decrease.

[0044] (5) The network structure configurable brain-like chip architecture proposed in the application has high-speed and high-precision on-chip learning ability, and has high area and energy efficiency, solves the shortcomings of existing edge small neuromorphic brain-like chips, and has high application potential and value in resource-constrained edge intelligent systems. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation to the scope. Other related drawings can also be obtained by those of ordinary skill in the art without any creative effort.

[0046] Figure 1 The overall scheme flow chart of the network structure configurable brain-like chip architecture in the embodiment;

[0047] Figure 2 The schematic diagram of the network structure configurable brain-like chip architecture in the embodiment;

[0048] Figure 3 The schematic diagram of the Meta-Crossbar routing structure in the embodiment;

[0049] Figure 4 The schematic diagram of the fully event-driven pulse processing unit in the embodiment;

[0050] Figure 5 The schematic diagram of the on-chip learning circuit module in the embodiment. DETAILED DESCRIPTION

[0051] The present application will be described in detail below with reference to the drawings.

[0052] As shown in the overall scheme flow chart of the network structure configurable brain-like chip architecture, it comprises: Figure 1

[0053] 1) Design of the network structure configurable brain-like chip architecture: based on the layered multi-core chip architecture, the hybrid parallel processing mechanism, the error backpropagation mechanism, the Meta-Crossbar routing mechanism, the fully event-driven mechanism, and the on-chip weight random update learning, the chip can realize high-speed and high-precision on-chip real-time learning, and at the same time, the pulse routing overhead is reduced, and the scalability and certain network mapping flexibility of the on-chip network are ensured.

[0054] 2) Design of the Meta-Crossbar routing structure: the proposed Meta-Crossbar routing structure can efficiently transmit pulse events between processing cores, compared with the traditional pulse routing method of the on-chip network (NoC), the huge routing overhead of the inter-chip pulse transmission can be relieved. At the same time, the design of the Meta-Crossbar routing structure also ensures the scalability and certain network mapping flexibility of the inter-chip / on-chip network.

[0055] ​3) Fully event-driven pulse processing unit design: In this circuit unit, the neural processing unit is activated only when an input pulse event arrives, and the spiking neurons it is responsible for execute operations in sequence, which can minimize computational overhead and energy consumption.

[0056] 4) Design of on-chip learning circuit module: The on-chip learning related circuit module adopts error broadcasting, on-chip weight random update learning and other technologies, which greatly improves the utilization rate of chip storage resources, reduces chip area and cost, and enables fast on-chip learning.

[0057] like Figure 2 As shown in this embodiment, a neuromorphic chip architecture with a configurable network structure fully utilizes the structural characteristics and regular connections of FC-SNN (Fully Connected SNN) to reduce hardware design complexity and cost. The proposed in-memory computing neuromorphic chip architecture adopts a dual-level macro-core (MCore)-micro-core (uCore) multi-core architecture, which can simply and efficiently complete the mapping from FC-SNN models to chip computing and storage resources.

[0058] like Figure 2 As shown in the upper right corner, in this embodiment, the configurable neuromorphic chip architecture (hereinafter referred to as the chip architecture) includes M1 macroneural computation cores (MCores), a meta-cross-switch array routing structure, and an output error calculation unit. The M1 macroneural computation cores are each connected to the meta-cross-switch array routing structure, which is connected to the output error calculation unit. Each macroneural computation core can be mapped to a portion of any layer in the FC-SNN, and the maximum number of FC-SNN layers supported by the chip is also M1. Each macroneural computation core can support a maximum of N... L The data computation and storage of LIF neurons are handled by a maximum of N, and each LIF neuron supports a maximum of N. S Each synaptic fan-in. Pulse events transmitted between macroneural computational cores are encoded as AER-formatted AER packets, which contain the address (i.e., index) and timestamp of the neuron emitting the pulse signal. Pulse packets are transmitted between MCores via a meta-crossbar switch array routing structure. Pulse packets originating from an MCore can be routed to any MCore (including itself), thus the chip supports configuring FC-SNN networks with various structures, greatly improving the scalability of on-chip network size and the flexibility of network mapping.

[0059] like Figure 2 As shown in the top left corner, in this embodiment, each MCore contains M2 parallel micro-neural computation cores (uCore), and each uCore is responsible for (N L / M2) LIF neurons are calculated to accelerate the computational processing tasks of each layer of the FC-SNN. The micro neural computing core includes a neural processing unit, a weight update unit, a neuron state and weight memory, and an output AER buffer. The neural processing unit is responsible for updating the state of each LIF neuron and emitting pulses. The weight update unit is responsible for receiving errors and updating the synaptic weights of the neurons. The neuron state and weight memory is responsible for storing neuron state information and synaptic weights. The output AER buffer is used to buffer the pulse data emitted by the neural processing unit. Among them, the neuron state and weight memory inside the uCore are connected with the neural processing unit and the weight update unit respectively, and the output AER buffer is connected with the neuron state and weight memory. In addition, each MCore also contains an input AER buffer for buffering input pulse data, a register lookup table-based leakage unit for calculating and storing intermediate variables required in the leakage process of the membrane potential of the computing neuron, and an output AER scheduler and routing node for scheduling input / output pulse data. Among them, the register lookup table-based leakage unit is connected with the neural processing unit of each micro neural computing core in the macro neural computing core respectively; the input AER buffer is connected with the register lookup table-based leakage unit; the output AER scheduler is connected with the output AER buffer of each micro neural computing core in the macro neural computing core; the routing node is connected with the weight update unit, the input AER buffer and the output AER scheduler of each micro neural computing core in the macro neural computing core respectively.

[0060] As shown in Figure 2 The mixed multi-level parallel processing mechanism of the chip architecture is shown. The processing process of the whole chip architecture includes a forward phase and a reverse phase. At the MCore level, in the forward phase, each MCore independently completes the calculation in an event-driven manner, that is, as long as the input pulse event is received, the processing is started without synchronization; while in the reverse phase, each MCore completes the synaptic weight update of the neurons in a synchronous array parallel manner. At the uCore level, in either the forward phase or the reverse phase, each uCore performs operations in a naive array parallel manner. Through this dynamically reconfigurable mixed multi-level parallel processing mechanism, the computing resources on the chip can be efficiently used for different processing phase requirements, thereby realizing high-speed on-chip learning and inference.

[0061] The data flow in the forward phase and the reverse phase of the processing process of the chip is briefly described as follows:

[0062] In the forward stage, after the input AER reaches the meta-crossbar routing structure, the meta-crossbar routing structure sends the AER to the corresponding MCore according to the configuration of the routing structure. After the MCore receives the AER, the MCore first stores the input AER in the input AER buffer, then sends the AER to the leakage unit based on the register lookup table for operation, and then sends the AER to the neural processing unit in the uCore. The neural processing unit is responsible for updating the LIF neuron state information, including membrane potential, pulse sending time, etc., which is realized by accessing the neuron state and weight memory. If the membrane potential of a certain LIF neuron exceeds the threshold, the neural processing unit will emit an AER into the output AER buffer, and then send the AER to the next MCore through the output AER scheduler and routing node, or directly output to the outside of the chip architecture.

[0063] In the reverse stage, after the output error calculation unit completes the output error calculation, the output error vector is broadcast to each MCore. The weight update unit in the MCore receives the output layer error and starts the neuron synaptic weight update process, that is, the on-chip learning process.

[0064] In the chip architecture in this embodiment, M i = M2= 4, and individually, N L and N S The maximum value of each of N S = 1K, but when N L The maximum value of N L and N S is related, and is limited by the on-chip storage space. The maximum value of N L × N s is 64K. Therefore, the maximum number of neurons supported by the chip architecture is 4 × 1K = 4K, and the maximum number of neuron synapses is 4 × 64 × 1K = 256K.

[0065] As shown in Figure 3 , in this embodiment, the meta-crossbar routing structure is used for efficient transmission of pulse events between MCores in the forward stage. The meta-crossbar routing structure includes a routing node network, M1+1 AER schedulers, and M1+1 switch connection configuration registers. The routing node network includes (M1+1) × (M1+1) routing node switches, and each column of routing node switches is connected in turn, and each row of routing node switches is connected in turn. As shown in Figure 3 (the upper left corner), the input path of the meta-crossbar routing structure includes an input node (Input) and MCore1-MCore4, and the output path includes an output node (Output) and MCore1-MCore4. Figure 3The hollow circles in the diagram represent possible routing paths, and connections are established by linking configuration registers with configuration switches. Pulse packets from input nodes can be routed to MCorel-MCore4, and pulse packets emitted by MCorel-MCore4 can be routed to MCore1-MCore4, or output nodes. For example... Figure 3 As shown in the upper right corner, taking the fourth column of the Meta-Crossbar routing structure as an example, the specific implementation of this Meta-Crossbar routing architecture is illustrated. Figure 3 As shown in the top left corner, the fourth column of the meta-crossover switch array routing structure represents the routing nodes connected to MCore1. Each hollow circle acts as a switch, controlling whether a connection is established. Figure 3 As shown in the upper right corner, the switch is turned on and off via a configuration register. When the value of the configuration register is 1, it indicates that the routing node has a connection. Note that since there may be multiple paths connecting to MCore1, the AER scheduler needs to arbitrate and schedule AER packets to prevent pulse data collisions and conflicts. Input_req, MCore1_req...MCore4_req are the routing path access permission signals output by the AER scheduler. Only when the configuration register is 1 and the routing path access permission signal is high can the AER packet reach MCorel through the AER scheduler.

[0066] One of the core innovations of the meta-crossbar switch array routing structure proposed in this embodiment lies in the "Meta". In early neuromorphic brain-inspired chips, a NoC+Crossbar connection method was typically used to support mapping various network structures, offering high flexibility. However, Crossbar requires allocating storage space for all possible connections and synaptic weights, and weight reuse is not possible. When mapping network structures such as convolutions, it suffers from problems such as high kernel occupancy and low utilization of in-kernel storage resources, resulting in a large chip area. To reduce chip area and power consumption, many small neuromorphic chips adopt fixed connection rules, supporting only specific network structures. However, this approach limits the flexibility and scalability of the chip architecture. In contrast, this embodiment adopts a meta-crossbar switch array routing structure, implementing configurable implicit connections within the kernel. Only a small number of parameter registers (i.e., switch connection configuration registers) are needed to achieve full connection (FC) in SNNs, eliminating the need to store all possible connections and weights. This significantly reduces on-chip storage resource consumption, chip area, and power consumption. Meanwhile, the network structure is also configurable. Combined with the meta-crossbar switch array routing structure, and through the combination of MCore and configuration parameter registers, various network structures can be flexibly implemented on-chip, greatly enhancing the flexibility and scalability of the chip architecture. For example...Figure 3 As shown in the lower part, three examples of configurations are given. Among them, the example in the lower left corner shows how to realize the FC-SNN of 256-256-256-256-256 structure by configuring the parameter registers and the routing connections. In this example, each MCore is responsible for the calculation of one layer (256 neurons) in the FC-SNN. By configuring the parameter registers synapses per neuron and neurons, the fan-in / fan-out number of each MCore can be determined. By configuring the connection relationship of the routing nodes, four MCoress are spliced to realize the network structure of 256-256-256-256-256. The second example shows how to realize the FC-SNN of 512-256-128-64 structure. Different from the first example, MCore1 and MCore2 jointly take charge of one layer in the network in this example. The third example shows how to realize the FC-SNN of 1024-256 structure. In this example, MCore1-MCore4 jointly take charge of one layer in the network.

[0067] In the present embodiment, Figure 3 Taking the first example in the lower left corner as an example, when a certain AER data packet arrives at MCore1, since the connection structure of MCore1 has been determined in advance by configuring the parameter registers, MCore1 can automatically complete the calculation and state update of 256 neurons. That is, one input data packet can trigger the update of 256 neurons, without the need to copy the pulse data packet and deliver the pulse one by one as in the early neuromorphic chips. This data packet is called a “Meta” data packet. Through the above mechanism, the Meta-crossbar array routing structure fully utilizes the regular structure of the deep FC-SNN, improves the utilization rate of the in-core storage resources, realizes the mapping of the deep SNN network with only moderate chip area, and retains the scalability and certain mapping flexibility of the SNN scale. In addition, since there is no need to copy the pulse data packet, the Meta-AER mechanism greatly reduces the number of pulse data packets in the routing network, and alleviates the congestion problem commonly seen in the NoC network.

[0068] As shown in the lower part, three examples of configurations are given. Among them, the example in the lower left corner shows how to realize the FC-SNN of 256-256-256-256-256 structure by configuring the parameter registers and the routing connections. In this example, each MCore is responsible for the calculation of one layer (256 neurons) in the FC-SNN. By configuring the parameter registers synapses per neuron and neurons, the fan-in / fan-out number of each MCore can be determined. By configuring the connection relationship of the routing nodes, four MCoress are spliced to realize the network structure of 256-256-256-256-256. The second example shows how to realize the FC-SNN of 512-256-128-64 structure. Different from the first example, MCore1 and MCore2 jointly take charge of one layer in the network in this example. The third example shows how to realize the FC-SNN of 1024-256 structure. In this example, MCore1-MCore4 jointly take charge of one layer in the network. Figure 4 As shown in the lower part, three examples of configurations are given. Among them, the example in the lower left corner shows how to realize the FC-SNN of 256-256-256-256-256 structure by configuring the parameter registers and the routing connections. In this example, each MCore is responsible for the calculation of one layer (256 neurons) in the FC-SNN. By configuring the parameter registers synapses per neuron and neurons, the fan-in / fan-out number of each MCore can be determined. By configuring the connection relationship of the routing nodes, four MCoress are spliced to realize the network structure of 256-256-256-256-256. The second example shows how to realize the FC-SNN of 512-256-128-64 structure. Different from the first example, MCore1 and MCore2 jointly take charge of one layer in the network in this example. The third example shows how to realize the FC-SNN of 1024-256 structure. In this example, MCore1-MCore4 jointly take charge of one layer in the network. Figure 4 As shown in the lower part, three examples of configurations are given. Among them, the example in the lower left corner shows how to realize the FC-SNN of 256-256-256-256-256 structure by configuring the parameter registers and the routing connections. In this example, each MCore is responsible for the calculation of one layer (256 neurons) in the FC-SNN. By configuring the parameter registers synapses per neuron and neurons, the fan-in / fan-out number of each MCore can be determined. By configuring the connection relationship of the routing nodes, four MCoress are spliced to realize the network structure of 256-256-256-256-256. The second example shows how to realize the FC-SNN of 512-256-128-64 structure. Different from the first example, MCore1 and MCore2 jointly take charge of one layer in the network in this example. The third example shows how to realize the FC-SNN of 1024-256 structure. In this example, MCore1-MCore4 jointly take charge of one layer in the network.

[0069]

[0070] Where: updated V m，j V represents the new membrane potential of the currently processed neuron j. m，j τ represents the old membrane potential of neuron j, and w represents the leakage factor. ij This refers to the synaptic weights between neuron i in the previous layer and neuron j in the current layer. Therefore, updating the membrane potential requires first performing a leakage operation, followed by adding the corresponding membrane potential. Furthermore, the updated membrane potential must be compared with the set neuron threshold V. th For comparison, if it is greater than the neuron threshold V th If so, a pulse is emitted.

[0071] Table 1: Definitions and descriptions of variables in the pulse processing unit

[0072]

[0073]

[0074] Therefore, the calculation process of a fully event-driven pulse processing unit is as follows:

[0075] like Figure 4 As shown in the upper left corner, in this embodiment, the leak unit based on the register lookup table includes a first adder 5 and a register-based exponentiation lookup table, with the first adder 5 connected to the register-based exponentiation lookup table. When AER(i, t) reaches MCore, Δt = tt is first calculated through the first adder 5. pre Then, the obtained Δt is used as the address to access the pre-initialized register-based exponent lookup table, and the leaked term exp(-Δt / τ) can be obtained directly without performing complex exponentiation operations on the chip. Furthermore, since the exponent lookup table is implemented using registers, it occupies very little chip area.

[0076] like Figure 4 As shown, in this embodiment, the neural processing unit includes a first multiplier 1, a second adder 2, a first comparator 3, a fire_flag_regs register, and a second comparator 4. The first multiplier 1 is connected to the second adder 2 and a register-based exponent lookup table, the first comparator 3 is connected to the second adder 2, and the fire_flag_regs register is connected to the first comparator 3. The second comparator 4 is connected to the second adder 2. After obtaining the leaked term exp(-Δt / τ), the leaked term is first multiplied by the old membrane potential V of neuron j through the first multiplier 1. m，jThe leaked neuron membrane potential is obtained by multiplication, and then the leaked neuron membrane potential is multiplied by the synaptic weight w through the second adder 2. ij The summation yields the updated membrane potential V of neuron j. m，j Then, the updated membrane potential V of neuron j is obtained through the first comparator 3. m，j With threshold V th For comparison, if updated V m，j >V th If an event occurs, a pulse AER_out is emitted. Furthermore, to minimize computational overhead and energy consumption, the fully event-driven pulse processing unit only allows each neuron to emit a pulse once. The address of a neuron that has emitted a pulse is recorded in the fire_flag_regs register file of the neural processing unit. Before each pulse emission, the fire_flag_regs register is accessed; if the current neuron j has not yet emitted a pulse, then the pulse is allowed to be emitted. When each event arrives and updates the membrane potential, the second comparator 4 needs to determine whether the current membrane potential is greater than the previously stored maximum membrane potential V. max，j If it is greater than, then V needs to be updated. max，j At the same time, the current timestamp is updated to t. max，j It is stored in neuron state and weight memory for use in the on-chip learning process during the subsequent reverse phase.

[0077] like Figure 4 As shown in the upper right corner, this is a timing diagram illustrating the spiking unit's access to the neuron state and weight memory. Blue represents memory reads, red represents memory writes, and the orange-yellow background represents updates to on-chip learning-related variables. The fully event-driven spiking unit operates on neurons j = 1, 2, 3... sequentially. Because the on-chip memory uses single-port SRAM, it can only read or write per cycle; therefore, it must sequentially update the neuron-related variables V. m，j w ij V max，j t max，j Perform read / write operations.

[0078] like Figure 5 As shown, the on-chip learning circuit module proposed in this embodiment is illustrated. This module employs techniques such as error broadcasting and on-chip weight random update learning, significantly improving chip storage resource utilization, reducing chip area and cost, and enabling rapid on-chip learning. Table 2 shows the definitions and explanations of the variables in the figure. Figure 5The upper part shows the circuit diagram of the output error calculation unit, which includes an output layer neuron error calculation module 14 and an output error register connected to the output layer neuron error calculation module 14. The output layer neuron error calculation module 14 is used to calculate the error of each output layer neuron and store it in the output error register. The first step of the on-chip learning process is to calculate the three-valued output layer neuron error err. For an output layer neuron k, assuming that its assigned label is equal to the sample label and it does not fire a pulse (neuron label == sample label & fire flag == 0), then err(k) = 1; if its assigned label is not equal to the sample label and it fires a pulse (neuron label ≠ sample label & fire flag == 1), then err(k) = -1; otherwise, err(k) = 0. The output error calculation unit calculates the error of each output layer neuron in sequence and then stores it in the output error register. After all output errors have been calculated, they are broadcast to the uCore of each MCore.

[0079] Table 2: Definitions and descriptions of variables in the on-chip learning circuit module

[0080] Variable name Definition Explanation i Source neuron address index - j Neuron index being processed - k Output layer neuron index - neuron label Neuron label - sample label Sample label - B Random projection matrix Binary (-1, 1) err Output layer neuron error Ternary (-1, 0, 1) local_err Hidden layer neuron local error - Δw ij [15:8]] Weight update high 8 bit For random update Δw ij [7:0]] Weight update low 8 bit For random update

[0081] like Figure 5 The lower half shows the circuit diagram of the weight update unit, which also includes a third adder 7, a second multiplier 8, a shifter 9, an 8-bit linear feedback shift register 10, a third comparator 11, a fourth adder 12, and a fifth adder 13. The third adder 7 is connected to a register-based exponent lookup table. The second multiplier 8 is connected to the register-based exponent lookup table, the local error accumulation register, and the shifter 9, respectively. The third comparator 11 is connected to the shifter 9 and the 8-bit linear feedback shift register 10, respectively. The fourth adder 12 is connected to the shifter 9, the third comparator 11, and the fifth adder 13, respectively. After receiving the output error err, the weight update unit in each MCore's uCore needs to calculate the local error local_error according to formula (2):

[0082]

[0083] Where B(j, k) are elements in the random projection matrix B, and all elements in B are binary, having only two values: -1 and 1. As can be seen from formula (2), the calculation of the local error (local_error) requires multiple multiplication and addition operations. For example... Figure 5As shown, in the on-chip learning circuit module proposed in this embodiment, the weight update unit adopts a 16-way 2b x 1b low-cost multiplier array and a single-cycle 16-to-1 adder tree to implement the multiply-add operation, and each uCore can complete 16 multiply-add operations in 2 clock cycles. Through the above design, the storage resources occupied by the multiplier are reduced to the greatest extent, the area and cost of the chip are reduced, and the speed of on-chip learning is ensured. The intermediate result of the multiply-add operation is saved in the local error accumulation register, and when all the multiply-add operations are completed, the value in the local error accumulation register is the local error local_error(j) of the current neuron j. After obtaining the local error, the synaptic weight update is calculated according to formula (3):

[0084]

[0085] where Δw ij represents the synaptic weight update amount, and λ represents the learning rate. The learning rate is set to λ = 2 -U , so in the hardware implementation process, only a shift operation is needed to achieve it, avoiding the use of resource-consuming multipliers. For example Figure 5 As shown in the lowermost circuit, first, t max，j and t i are read from the neuron state and weight memory of the uCore, then Δt is obtained by subtracting t ij , and Δt is used as an address to access the register-based exponential lookup table to obtain the exponential term, then the exponential term is multiplied with local_error(j) through the second multiplier 8, and finally the weight update amount Δw ij is obtained by right shifting Ubit. ij The present application does not directly add the weight update amount Δw ij to the synaptic weight w to obtain the updated weight, because Δw ij is 16 bits, and if it is directly added, the synaptic weight w also needs to be stored in 16-bit format, which will occupy a large chip area. In order to improve the utilization rate of on-chip storage resources and reduce the area and cost of the chip to the greatest extent, the present application proposes an on-chip weight random update technology, which uses low-precision storage of synaptic weight w while ensuring almost no decrease in chip recognition accuracy, greatly reducing the chip area. For example Figure 5 As shown in the lower right corner, after obtaining the 16-bit Δw ij , first, the low 8-bit part Δw ij [7:0] is compared with the 8-bit unsigned random number R generated by the 8-bit linear feedback shift register 10 (8b LFSR) through the third comparator 11, and if Δw [7:0] ≥ R, a carry is generated to the high 8-bit part Δw [15:8] of Δw ij[15:8]; if Δw ij [7:0] < R, then discard Δw. ij [7:0]. Updated Δw ij The high 8-bit portion Δw ij [15:8] The old synaptic weight w, read from the neuron state and weight memory, is then added by the fourth adder 12 and the fifth adder 13 to obtain the updated synaptic weight w, which is then written into the neuron state and weight memory, thus completing the on-chip learning process. Furthermore, due to the pulse timestamp t in this design... i >0, therefore, when updating the weights, if the t of a certain neuron is... i =0 indicates that it has not emitted a pulse, and the relevant synapse will skip the weight update process.

[0086] This invention, by applying the aforementioned on-chip weight random update technology, employs full-precision updates during weight updates and requires only storing the high 8 bits of the weight during storage. This significantly reduces on-chip storage resource consumption, improves chip on-chip storage resource utilization, and thus reduces chip area and cost. Compared to directly storing high-precision 16-bit weights, this random update technology can save 1.27 mm². 2 (17.6%) of the active silicon area. Furthermore, experiments show that applying random updates to on-chip weights causes only a very small loss in recognition accuracy.

[0087] This invention designs a neuromorphic chip architecture for edge intelligent systems, featuring high-speed, high-precision on-chip learning capabilities and a configurable network structure. This architecture integrates hierarchical multi-core processing, hybrid parallel processing, Meta-Crossbar spiking routing, a fully event-driven mechanism, and on-chip weight random update learning to maximize chip processing speed and energy efficiency, enabling the chip to achieve fast, high-precision on-chip deep neural network (SNN) learning. The prototype chip based on this architecture is manufactured using a 65nm CMOS process, with a core area of ​​only 7.2 square millimeters. It supports up to 1,000 spiking neurons and 260,000 synapses, and can complete fast on-chip deep learning at a low power consumption of 61mW. In practical applications, it can achieve on-chip learning and inference speeds of 802 and 2270 frames per second, respectively. The energy consumed by the chip to compute each synaptic operation is only 3.09pJ, and the energy consumed to recognize each image is only 0.43uJ, exhibiting extremely high area and energy efficiency. It has significant application potential and value in resource-constrained edge intelligent systems.

[0088] In this embodiment, the on-chip learning circuit module uses the DeepTempo algorithm [1] for learning.

[0089] [1] C. Shi, T. Wang, J. He, J. Zhang, L. Liu and N. Wu, "DeepTempo: A Hardware-Friendly Direct Feedback Alignment Multi-Layer Tempotron Learning Rule for Deep Spiking Neural Networks," in IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 68, no. 5, pp. 1581-1585, May 2021.

[0090] Therefore, the embodiments of the present application are not limited to the above examples, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application should be equivalent replacement methods, and are included in the protection scope of the present application.

Claims

1. A network-structure-configurable brain-like chip, characterized in that: The macro neural computing core includes M1 macro neural computing cores, a meta-crossbar array routing structure, and an output error calculation unit, the M1 macro neural computing cores are connected with the meta-crossbar array routing structure respectively, and the meta-crossbar array routing structure is connected with the output error calculation unit; The pulse events transmitted between the macro neural computing cores are encoded into AER data packets in an AER format, wherein the AER data packets contain the address and timestamp of the neuron transmitting the pulse signal; The pulse data packets are transmitted between the macro neural computing cores through the meta-crossbar array routing structure, and the pulse data packets emitted from the macro neural computing core can be routed to any macro neural computing core; The meta-crossbar array routing structure includes a routing node network, M1+1 AER schedulers, and M1+1 switch connection configuration registers, The routing node network includes (M1+1) × (M1+1) routing node switches, and the routing node switches in each column are connected in sequence, and the routing node switches in each row are connected in sequence; Each routing node switch in each column is connected with the AER scheduler and the switch connection configuration register of the column through an AND gate respectively, and the opening and closing of the routing node switch are controlled through the switch connection configuration register and the AER scheduler.

2. The network-structure-configurable brain-like chip according to claim 1, wherein: The macro neural computing core includes M2 parallel micro neural computing cores, and each micro neural computing core can be responsible for N L / M2 LIF neuron calculations for accelerating the calculation processing tasks of each layer of the FC-SNN, wherein N L represents the number of LIF neurons that each macro neural computing core can calculate; The micro neural computing core includes: a neural processing unit for updating the state of each LIF neuron and emitting a pulse; a weight update unit for receiving an error and updating the synaptic weight of the neuron; a neuron state and weight memory for storing the neuron state information and the synaptic weight, the neuron state and weight memory being connected with the neural processing unit and the weight update unit respectively; an output AER buffer for buffering the pulse data emitted by the neural processing unit, the output AER buffer being connected with the neuron state and weight memory.

3. The network-structure-configurable brain-like chip according to claim 2, wherein: The macro neural computing core further includes: a register lookup table-based leakage unit for calculating and storing the intermediate variables required in the leakage process of the membrane potential of the computing neuron, the register lookup table-based leakage unit being connected with the neural processing unit of each micro neural computing core in the macro neural computing core respectively; an input AER buffer for buffering input pulse data, the input AER buffer being connected with the register lookup table-based leakage unit; an output AER scheduler for scheduling input / output pulse data, the output AER scheduler being connected with the output AER buffer of each micro neural computing core in the macro neural computing core; and a routing node connected with the weight update unit, the input AER buffer, and the output AER scheduler of each micro neural computing core in the macro neural computing core respectively.

4. The network-structure-configurable brain-like chip according to claim 3, characterized in that: The processing process of the whole chip architecture includes a forward phase and a reverse phase; The macro neural computing core independently completes the calculation in an event-driven manner in the forward phase, that is, the input pulse event is processed as received, without synchronization; The macro neural computing core completes the synaptic weight update of the neuron in a synchronous array parallel manner in the reverse phase; The micro neural computing core operates in an array parallel manner in the forward phase and the reverse phase.

5. The network-structure-configurable brain-like chip according to claim 3, characterized in that: The register-based lookup table-based leak unit comprises a first adder (5) and a register-based exponential lookup table, the first adder (5) being connected with the register-based exponential lookup table; when an AER (action event record) i , t ) reaches a macro neural computing core, calculating by the first adder that Δt = t - t pre , wherein, i indicates a source neuron index, t is a pulse timestamp, t pre is a pulse timestamp of a last event, taking the obtained Δt as an address to access a register-based exponential lookup table initialized in advance to obtain a leak term.

6. The network-structure-configurable brain-like chip according to claim 5, characterized in that: The neural processing unit comprises a first multiplier (1), a second adder (2), a first comparator (3), a fire_flag_regs register and a second comparator (4); The first multiplier (1) is connected with the second adder (2) and a register-based exponential lookup table respectively, the first comparator (3) is connected with the second adder (2), and the fire_flag_regs register is connected with the first comparator (3); and the second comparator (4) is connected with the second adder (2); The first multiplier (1) is used to combine the leaked term with the neuron. j Old membrane potential V m, j The leaked neuron membrane potential is obtained by multiplication; the second adder (2) is used to multiply the leaked neuron membrane potential with the synaptic weights. w ij Adding together yields neurons j Updated membrane potential V m, j The first comparator (3) is used to update the membrane potential. V m, j AND threshold V th Compare and update V m, j > V th When a pulse is emitted, a pulse AER_out is fired; and the address of the neuron that emitted the pulse is recorded in the fire_flag_regs register file of the neural processing unit. This fire_flag_regs register is accessed before each pulse is emitted; if the current neuron... j If no pulse has been emitted yet, then the pulse may be emitted. When each event arrives and updates the membrane potential, it is judged by the second comparator (4) whether the current membrane potential is greater than the previously stored maximum membrane potential V max, j , if greater, it needs to be updated V max, j , and the current timestamp is synchronously updated to t max, j , and stored in the neuron state and weight memory.

7. The network-structure-configurable brain-like chip according to claim 1, wherein: The output error calculation unit comprises an output layer neuron error calculation module (14) and an output error register connected with the output layer neuron error calculation module (14); The output layer neuron error calculation module (14) calculates the error of each output layer neuron and stores it in the output error register, and broadcasts it to the micro neural computing cores of each macro neural computing core after all output errors are calculated.

8. The network-structurally-configurable brain-like chip according to claim 5, characterized in that: The weight updating unit comprises a 16-way 2b×1b low-cost multiplier array, a single-cycle 16-to-1 adder tree and a local error accumulation register; The 16-way 2b×1b low-cost multiplier array and the single-cycle 16-to-1 adder tree are connected to realize multiplication and addition operation; The local error accumulation register is connected with the single-cycle 16-to-1 adder tree to store the intermediate result of multiplication and addition operation, and the value in the local error accumulation register is the local error local_error(j) of the current neuron j after all multiplication and addition operations are completed.

9. The network-structure-configurable brain-like chip according to claim 8, characterized in that: The weight updating unit further comprises a third adder (7), a second multiplier (8), a shifter (9), an 8-bit linear feedback shift register (10), a third comparator (11), a fourth adder (12) and a fifth adder (13); The third adder (7) is connected with a register-based exponential lookup table; the second multiplier (8) is connected with a register-based exponential lookup table, a local error accumulation register and a shifter (9) respectively; the third adder (7) takes the maximum value of the membrane potential of the neuron j at the time stamp t max,j and the pulse time stamp t i Subtracting to get delta t , and delta t is taken as an address to access the register-based exponential lookup table to obtain an exponential term, and the exponential term is multiplied with the local error of the hidden layer neuron local_error(j) through the second multiplier (8), and is shifted right by U bits to obtain the 16-bit weight update amount delta w ij ; wherein U is a learning parameter of lambda, λ , which represents a learning rate, and lambda = 2 -U ; The third comparator (11) is connected to the shifter (9) and the 8-bit linear feedback shift register (10) respectively, and the fourth adder (12) is connected to the shifter (9), the third comparator (11) and the fifth adder (13) respectively; after obtaining the 16-bit weight update amount ∆ w ij Then, use its lower 8 bits ∆ w ij [7:0] is compared with the 8-bit unsigned random number R generated by the 8-bit linear feedback shift register (10) through the third comparator (11), and the updated ∆ w ij The high 8 bits ∆ w ij [15:8] The old weights read from the neuron state and weight memory are added by the fourth adder (12) and the fifth adder (13). w To obtain the updated weights w And write it into the neuron state and weight memory.

Images