Matching feedback integral excitation neuron circuit

Abstract

The invention relates to an integral excitation neuron circuit for symbol processing, which is characterized in that the integral excitation neuron circuit comprises a feedback sub-circuit connected to the positive and negative output ends of the circuit. The feedback sub-circuit is configured to generate a positive charge packet based on each negative spike output signal and the stored weight and output the positive charge packet to a common line of the neuron circuit, and to generate a negative charge packet based on each positive spike output signal and the stored weight and output the negative charge packet to the common line. As the feedback circuit is constructed in the same way as the input weighting circuit, structural matching is realized, so that higher precision and less behavior changing along with PVT change are realized.

Description

Technical Field

[0001] The present invention relates to an integrate-and-fire neuron circuit, comprising: at least one input terminal and at least positive and negative output terminals, wherein the at least one input terminal is used to receive an input signal, and the at least positive and negative output terminals are used to transmit positive and negative spike output signals; a weighted subcircuit, which is configured to generate positive and / or negative charge packets based on the input signal and output the positive and / or negative charge packets to a common line; and an arrangement structure of a capacitor or an integrator and two comparators, which generates a positive spike output signal at the positive output terminal and a negative spike output signal at the negative output terminal.

[0002] Such circuits, which are used to roughly mimic neuron function, are used in artificial neural networks (ANNs), especially artificial spiking neural networks (SNNs). Neuronal dynamics can be viewed as a summation process combined with a mechanism that triggers action potentials above a certain critical voltage. One known representation of a neuronal circuit is the integrate-and-fire neuronal circuit. Information is contained in the presence or absence of spikes generated by such a neuronal circuit. Background Art

[0003] The document "A Neuromorphic Cortical-Layer Microchip for Spike-Based Event Processing Vision Systems," in IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 53, no. 12, pp. 2548-2566, Dec. 2006 by R. Serrano-Gotarredona et al., proposes an integral-spark neuron circuit for signed and unsigned processing. These circuits include a subcircuit for generating positive and / or negative charge packets based on an input signal and outputting these charge packets to a common line. The charge of the positive and negative charge packets is stored in a capacitor, which converts the sum of the charges into a voltage. The voltage is compared with a reference voltage by a comparator, and if the voltage is greater than or less than the corresponding reference voltage, the comparator generates a positive and / or negative spike output signal. When an output spike is generated, the circuit performs a reset to clear all charges on the capacitor. When this reset is performed, continuous inputs may be lost (blind period). Since the reset deletes all information of previous operations, quantization errors are not stored either. Averaging multiple operations of the same type will not show higher resolution and accuracy in the averaged result. Here, the amount by which the voltage exceeds the reference voltage can be considered as a quantization error. The overall gain of such a neuron circuit depends on the absolute value of the capacitor, input current, and reference voltage. Therefore, the results will vary greatly with temperature, process, and power supply variations, also known as PVT variations (PVT: Process Voltage Temperature).

[0004] EP 2877958 B1 describes a spike-domain asynchronous neuron circuit with feedback. The feedback is implemented as proportional feedback, continuous in value and time, and affects an adjustable gain subcircuit of the neuron circuit. Summary of the invention

[0005] An object of the present invention is to provide an integrate-and-fire neuron circuit that provides higher accuracy and less variation over temperature, process and power supply variations (PVT variations).

[0006] This object is achieved by an integrate-and-fire neuron circuit according to claim 1. Advantageous embodiments of the circuit are subject matter of the dependent claims or can be derived from the subsequent description and the exemplary embodiments.

[0007] The proposed circuit comprises at least one input terminal for receiving an input signal and at least positive and negative output terminals for transmitting positive and negative spike output signals. The input signal can be a spike signal, but can also be a pulse width modulated (PWM) or a static signal. The circuit also includes a weighting subcircuit, which is configured to generate positive and / or negative charge packets based on the input signal and the stored weights and output the positive and / or negative charge packets to a common line of the neuron circuit. The capacitance between the common line and the reference potential stores the charge from the positive and negative charge packets and converts the sum of the charges into a voltage. The capacitance can be formed by a capacitor connected between the common line and the reference potential, but can also be an inherent (or parasitic) capacitance between the common line and at least one power supply voltage or ground potential (as a reference potential), which is caused by the components and wiring of the circuit. In addition to the capacitance, an integrator can also be used to sum the charge of the positive and negative charge packets and convert the sum of the charges into a voltage. The first comparator compares the voltage with a higher reference voltage for quantization and is configured to generate a positive spike output signal at the positive output terminal when the voltage is greater than the higher reference voltage. The second comparator compares the voltage with a lower reference voltage for quantization and is configured to generate a negative spike output signal at the negative output terminal when the voltage is less than the lower reference voltage. As known in the art, for example, from the above-mentioned document of R.Serrano-Gotarredona et al., the above-mentioned components enable the circuit to implement signed processing, i.e. processing with positive weights and processing with negative weights. The proposed neuron circuit is characterized by having a feedback subcircuit connected to the positive and negative output terminals. The feedback subcircuit is configured to generate a positive charge packet based on each negative spike output signal (and the stored weight) and output the positive charge packet to a common line, and to generate a negative charge packet based on each positive spike output signal (and the stored weight) and output the negative charge packet to a common line. Therefore, in the proposed neuron circuit, the feedback based on the generated spike output signal is not used to reset the capacitor, but to feedback a charge packet of opposite sign. The feedback subcircuit is constructed in the same way as the input weighting subcircuit. Due to this feedback subcircuit, the input is always acquiring information and there is no blind period. In addition, since the capacitor is not completely reset after each spike output, the accumulation of quantization errors (the difference between the higher / lower reference voltage and the total voltage) and more accurate output compensation are achieved. This, as well as the next stage of integration, the following integration-excitation neuron circuit, will reduce noise and improve accuracy because the average becomes more meaningful. It is possible to wait longer to obtain more accurate calculation results because the quantization errors of previous operations will then be accumulated or compensated over a longer period of time. As will be shown later in conjunction with an exemplary embodiment, the voltage gain between two of these neuron circuits depends only on the weight ratio and the capacitor ratio, and not on the absolute current value and the absolute pulse width.Therefore, using structural matching in a preferred embodiment, the proposed neuron circuit provides higher accuracy and smaller behavioral changes with PVT changes. Structural matching refers to that the functional blocks or functional units intended to perform the same operation are all implemented by the same sub-blocks, components and / or elements in the same structural and physical manner. The functional blocks or functional units that need to perform opposite operations (for example, compensating positive currents by negative currents) are implemented in a complementary structure of complementary elements (for example, replacing PMOS transistors with NMOS in a mirror arrangement structure). That is, in the preferred configuration of the circuit described later, all P-AWEs are constructed as multiple instances of a block, and all N-AWEs are constructed by multiple instances of the second block and its sub-parts, which are mainly internal current sources and switches, wherein the N-AWE contains complementary elements arranged in a structure complementary to the P_AWE. This means that all functional blocks or units will change with PVT in the same way, so the impact of such changes is offset in the overall behavior. Through the present invention, a digitally configurable (leaky) integrate-and-fire neuron that can be used for mixed-signal spiking neuron networks is proposed.

[0008] The proposed feedback enables the use of a local clock created by a small circuit. The highly variable pulse width and period of such a small circuit does not affect the multi-level accuracy, since there is only one clock source when considering the same charge packet in the sending and receiving circuits. The information in the pulse width compensates for the non-ideal circuits. The proposed feedback also provides the opportunity to roughly calculate the result over a short period of time, or to count output spikes over multiple time periods with the same input spike density and obtain a higher accuracy for the same operation, since the rest of the previous calculation is not removed and therefore increases over time. This particular implementation relies on the structural matching of the components and makes the circuit accuracy less dependent on variations in bias currents and capacitances caused by production or environmental effects.

[0009] Below, the preferred embodiment of the proposed neuron circuit is described. These embodiments can also be combined. In a first embodiment, the weighted subcircuit includes at least one, preferably multiple first positive analog weight emulators (P-AWE), each of which is configured to generate a positive charge packet and output the positive charge packet to a common line when enabled by a first enable signal in combination with an additional enable signal. The additional enable signal is provided based on the desired weight. The subcircuit also includes at least one, preferably multiple first negative analog weight emulators (N-AWE), each of which is configured to generate a negative charge packet and output the negative charge packet to a common line when enabled by a first enable signal in combination with an additional enable signal (depending on the desired weight). The input signal interface of the weighted subcircuit receives an input signal and generates a first enable signal for the first positive and negative analog weight emulators based on the input signal.

[0010] In another embodiment, the feedback subcircuit further comprises: at least one, preferably multiple (second) positive analog weight simulators, each (second) positive analog weight simulator is configured to generate a positive charge packet and output the positive charge packet to a common line when enabled by a second enable signal in combination with an additional enable signal; at least one, preferably multiple (second) negative analog weight simulators, each (second) negative analog weight simulator is configured to generate a negative charge packet and output the negative charge packet to a common line when enabled by a second enable signal in combination with an additional enable signal. The feedback signal interface of the feedback subcircuit is connected to the positive and negative output terminals of the neuron circuit, and is configured to generate a second enable signal for the second positive analog weight simulator based on each negative spike output signal, and to generate a second enable signal for the second negative analog weight simulator based on each positive spike output signal. Preferably, the first and / or second positive and negative analog weight simulators of the above one or two embodiments each have a memory or are connected to a memory, the memory stores weights, and provides additional enable signals based on the weights.

[0011] Preferably, the first and / or second positive and negative analog weight emulators of the above embodiments each include a switching current source for generating charge packets. The neuron circuit may also include a common bias circuit to bias these switching current sources of at least one of the first and second positive and negative analog weight emulators.

[0012] In another embodiment, the neuron circuit includes a reference voltage generator to generate a reference potential, an upper reference voltage, and a lower reference voltage, preferably using positive and negative analog weight emulators and capacitors.

[0013] In another embodiment, the proposed neuron circuit includes an offset subcircuit for generating an offset to the generated charge packet. The offset subcircuit includes: at least one, preferably a plurality of third positive analog weight simulators, each of which is configured to generate a positive charge packet and output the positive charge packet to a common line when enabled by a third enable signal in combination with an additional enable signal; and at least one, preferably a plurality of third negative analog weight simulators, each of which is configured to generate a negative charge packet and output the negative charge packet to a common line when enabled by a third enable signal in combination with an additional enable signal. The third enable signal is generated by a digital spike generator connected to the third positive and negative analog weight simulators. The spike generator generates a digital spike signal based on the received digital offset value, wherein the digital spike signal represents the third enable signal for the third positive and negative analog weight simulators.

[0014] The proposed neuron circuit can be advantageously used in spiking neuron networks or in the interface between non-spiking and spiking neuron networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The proposed neuron circuit is described below by way of example in conjunction with the corresponding figures. The figures show:

[0016] Figure 1 is an exemplary embodiment of the proposed neuron circuit;

[0017] Figure 2 is an example of cascading two neuron circuits according to the prior art; and

[0018] Figure 3 is an example of cascading two neuron circuits according to the present invention. DETAILED DESCRIPTION

[0019] Figure 1 An exemplary embodiment of the proposed neuron circuit is schematically depicted, which includes a number of optional components. The left side of the figure shows a weighting subcircuit, a feedback subcircuit, and an offset subcircuit. The right side shows an optional common bias circuit and an optional reference voltage generator, as well as a comparator and a capacitor (Cmem). Figure 1 The different memories shown in are connected to the memory interface, as shown by the thick lines with arrows.

[0020] An input signal is received at a corresponding input terminal Input i (i=1…N) of the input signal interface. Each input signal enables at least one of a plurality of positive analog weight emulators (P-AWE) and a plurality of negative analog weight emulators (N-AWE) via an interface, the interface receiving the input signal and generating an enable signal (P_EN, N_EN). The positive analog weight emulator (P-AWE) generates a positive charge packet (adds charge) from an input spike (input signal) and has a weight memory for storing additional enable data that affects the amount of activation current according to the stored weights. The negative analog weight emulator (N-AWE) is used to generate a negative charge packet (remove charge) from an input spike and also includes a weight memory for storing additional enable data that affects the amount of activation current according to the stored weights. The current or charge packet is generated by a switching current source in the emulator of the weighted subcircuit and fed to a common line of the neuron circuit, such as Figure 1 shown.

[0021] The capacitor Cmem connected in parallel with the parasitic capacitance Cparasitic and connected to the common line stores the charge generated by P-AWE and N-AWE and converts the sum of the charges into a voltage Vmem. The size of the capacitor Cmem can be modified by a digital value stored in a memory (gain memory). The capacitor Cparasitic can also replace the capacitor to store the charge and convert the charge into a voltage Vmem. Alternatively, Figure 1The capacitor Cmem in the circuit can be replaced by an integrator made of an OpAmp and another capacitor to integrate the charge and generate a voltage Vmem. The circuit also includes: a first comparator (CompU) for comparing Vmem with a higher reference voltage VRefU and generating a spike SpikeOUT_P on the positive output terminal when Vmem is greater than VRefU; and a second comparator (CompL) for comparing Vmem with a lower reference voltage VRefL and generating a spike SpikeOUT_N on the negative output terminal when Vmem is less than VRefL.

[0022] The positive feedback signal (Feedback_P) originates from SpikeOUT_N in the feedback signal interface of the feedback subcircuit, which enables another plurality of positive analog weight emulators (P-AWE). The negative feedback signal (Feedback_N) originates from SpikeOUT_P in the feedback signal interface, which enables another plurality of negative analog weight emulators (N-AWE). As in the case of the weighted subcircuit described above, the emulator includes means for storing weights in or near the analog weight emulators (weight memory), which may be SRAM cells or NVM cells.

[0023] Figure 1 The neuron circuit also includes an offset subcircuit having a digital spike generator to generate spikes from a digital offset input (offset memory). The digital spike generator is wired to at least one positive analog weight emulator (P-AWE) having the same or different multiplicity as the input spikes; and to at least one negative analog weight emulator (N-AWE) having the same or different multiplicity as the input spikes. These analog weight emulators generate positive and negative charge packets fed to a common line to be considered as additional input spike signals.

[0024] Figure 1Other (optional) components of the neuron circuit are a common bias circuit, a reference voltage generator, an additional switch SwIC, and a combination of a resistor Rleak and a switch SwLeak. The common bias circuit generates the biases (Biasp, Biasn) and feeds them to the switched current sources of the N-AWE and P-AWE. The reference voltage generator generates an upper reference voltage VRefU and a lower reference voltage VRefL relative to Vref. These reference voltages can be created from currents using the N-AWE and P-AWE circuits and capacitors to make the limits of the comparator depend on the absolute value of the bias current and therefore make the overall gain independent of the absolute value of the bias current. When the Reset input is activated or one of the at least one optional Inhibit input is activated, the switch SwIC is used to set the initial condition on the capacitor Cmem. The implementation of the resistor Rleak and the switch SwLeak is to create a defined leakage integrator behavior, where the resistor can also be implemented as a switched capacitor.

[0025] The most important advantage of the proposed neuronal circuit appears when cascading neurons (neuronal circuit). Figure 2 and Figure 3 The prior art ( Figure 2 ) and the proposed invention ( Figure 3 ), where the inactive part is omitted and the part before or after the focused part is shown with dashed lines. Both examples show a series of two neuron circuits, including capacitors Cmem1 and Cmem2 and corresponding current sources and comparators. In both figures, the excitation feedback of the first stage and the pulse reception of the second stage can be identified. Figure 2 In the system, when the first neuron fires a pulse, the charge removed from the first capacitor Cmem1 in the first stage is at least

[0026] Qremove1=(VrefU-Vref)*Cmem1

[0027] In the second stage, the charge added to the second stage capacitor Cmem2 is

[0028] Qadd2=W2*Iunit*tpulse*Cmem2

[0029] The ratio of Qadd2 / Qremove1, ie, the voltage gain of the circuit from Vmem1 to Vmem2, depends on the absolute weight value W2, the absolute current value Iunit, and the ratio of the absolute pulse width tpulse to the absolute voltage multiplied by the capacitance ratio.

[0030] On the other hand, according to Figure 3In the system according to the invention, when the first neuron fires a pulse, the charge removed from the first capacitor Cmem1 in the first stage is just not more than

[0031] Qremove1=W1*Iunit*tpulse*Cmem1

[0032] In the second stage, the charge added to the second stage capacitor Cmem2 is still

[0033] Qadd2=W2*Iunit*tpulse*Cmem2

[0034] Therefore, the gain Qadd2 / Qremove1 depends on the weight ratio W2 / W1 and the capacitor ratio Cmem2 / Cmem1, which are both quite accurate (e.g. 0.1%) when implemented using the same unit circuit, even though the absolute value of each capacitor varies over a larger range (e.g. 20%). Iunit and tpulse cancel each other out. This achieves very high accuracy and is less dependent on variations in bias current and capacitance.

Claims

1. An integrate-and-fire neuron circuit, comprising at least: - at least one input terminal for receiving an input signal and at least positive and negative output terminals for delivering positive and negative peak output signals; - a weighting subcircuit configured to generate positive and / or negative charge packets based on an input signal and stored weights and output the positive and / or negative charge packets to a common line; - a capacitor (Cmem) or an integrator, located between the common line and a reference potential (Vref), which stores the charges of the positive and negative charge packets and converts the sum of these charges into a voltage (Vmem); a first comparator (CompU) which compares said voltage (Vmem) with a higher reference voltage (VRefU) and is configured to generate a positive peak output signal at a positive output terminal when said voltage (Vmem) is greater than said higher reference voltage (VRefU); a second comparator (CompL) comparing said voltage (Vmem) with a lower reference voltage (VRefL) and configured to generate a negative spike output signal at a negative output terminal when said voltage (Vmem) is less than said lower reference voltage (VRefL); as well as - a feedback subcircuit connected to the positive and negative output terminals and configured to generate a positive charge packet based on each negative spike output signal and the stored weights and output the positive charge packet to a common line, and to generate a negative charge packet based on each positive spike output signal and the stored weights and output the negative charge packet to the common line.

2. The integrate-and-fire neuron circuit according to claim 1, characterized in that: The weighted subcircuit comprises: - at least one or more first positive analog weight emulators (P-AWEs), each first positive analog weight emulator configured to generate positive charge packets and output the positive charge packets to a common line when enabled by the first enable signal in combination with an additional enable signal; - at least one or more first negative analog weight emulators (N-AWE), each first negative analog weight emulator configured to generate a negative charge packet and output the negative charge packet to a common line when enabled by the first enable signal in combination with an additional enable signal; and - an input signal interface receiving an input signal and generating a first enable signal for a first positive and negative analog weight emulator (P-AWE, N-AWE) based on said input signal.

3. The integrate-and-fire neuron circuit according to claim 2, characterized in that: The feedback subcircuit comprises: - at least one or more second positive analog weight emulators (P-AWE), each second positive analog weight emulator configured to generate positive charge packets and output the positive charge packets to the common line when enabled by the second enable signal in combination with the additional enable signal; - at least one or more second negative analog weight emulators (N-AWE), each second negative analog weight emulator being configured to generate and output a negative charge packet to a common line when enabled by the second enable signal in combination with the additional enable signal; and -A feedback signal interface connected to the positive and negative output terminals and configured to generate a second enable signal for a second positive analog weight emulator (P-AWE) based on each negative spike output signal, and to generate a second enable signal for a second negative analog weight emulator (N-AWE) based on each positive spike output signal.

4. The integrate-and-excite neuron circuit according to claim 2 or 3, characterized in that: The first and / or second positive and negative analog weight emulators (P-AWE, N-AWE) each have a memory or are connected to a memory, which stores weights and / or holds additional enabling data.

5. The integrate-and-fire neuron circuit according to one or more of claims 2 to 4, characterized in that The first and / or second positive and negative analog weight emulators (P-AWE, N-AWE) each include a switched current source for generating charge packets.

6. The integrate-and-fire neuron circuit according to claim 5, characterized in that: The circuit includes a common bias circuit to bias the switched current source of at least one of the first and second positive and negative analog weight emulators (P-AWE, N-AWE).

7. An integrate-and-fire neuron circuit according to one or more of claims 1 to 6, characterized in that The circuit includes a reference voltage generator for generating a reference potential (Vref), an upper reference voltage (VRefU), and a lower reference voltage (VRefL) using positive and negative analog weight emulators and capacitors.

8. The integrate-and-fire neuron circuit according to one or more of claims 1 to 7, characterized in that The circuit includes an offset subcircuit, and the offset subcircuit includes: - at least one or more third positive analog weight emulators (P-AWE), each third positive analog weight emulator configured to generate positive charge packets and output the positive charge packets to the common line when enabled by the third enable signal in combination with the additional enable signal; - at least one or more third negative analog weight emulators (N-AWE), each third negative analog weight emulator configured to generate negative charge packets and output the negative charge packets to a common line when enabled by a third enable signal in combination with an additional enable signal; and - A digital spike generator connected to the third positive and negative analog weight emulator (P-AWE, N-AWE) and generating a digital spike signal based on the received digital offset value, wherein the digital spike signal represents a third enable signal for the third positive and negative analog weight emulator (P-AWE, N-AWE).

9. An integrate-and-fire neuron circuit according to one or more of claims 2 to 8, characterized in that The first and second positive analog weight simulators are formed in the same manner, the first and second negative analog weight simulators are formed in the same manner, and the positive and negative analog weight simulators are formed in a complementary manner to achieve structural matching.

10. The integrate-and-fire neuron circuit according to one or more of claims 1 to 9, characterized in that The capacitance (Cmem) is formed by a capacitor connected between the common line and the reference potential (Vref) or at least one power supply line.

11. The integrate-and-fire neuron circuit according to claim 10, characterized in that: The capacitor is configured to be modified in size by a digital value stored in an additional memory.

12. An integrate-and-fire neuron circuit according to one or more of claims 1 to 11, characterized in that A resistor can be switched in parallel with the capacitor (Cmem) to achieve controlled leakage behavior.

13. The integrate-and-fire neuron circuit according to claim 12, characterized in that: The resistors are implemented using switched capacitors.

14. An integrate-and-fire neuron circuit according to one or more of claims 1 to 13, characterized in that A switch is arranged between the common line and the reference potential (Vref) to allow the common line to be set to the reference potential (Vref) when activated by at least one of an inhibit input or a reset input.

Images