LIF digital neuron model based on Flash device

Abstract

The invention discloses an LIF digital neuron model based on a Flash device, and the LIF digital neuron model is matched with a synaptic circuit to complete digital circuit simulation development of accumulation-release of input weight data. The model comprises the following steps: modeling an LIF neuron model, wherein a signal processed by a neuron is a sparse Spike sequence, and multiplication and addition calculation with a weight value needs to be completed on a neural synapse; an LIF neuron model is selected for simplification, and design of a digital neuron circuit model adaptive to the spiking neural network is completed; membrane capacitance and membrane resistance are introduced, and operation conditions of sodium and potassium ions inside and outside a cell membrane are abstracted; and the membrane potential updating module is used for converting an LIF equation into a digital circuit friendly form by using a discrete method, converting the number of floating points into 8 fixed-point decimal numbers, and realizing an accumulation-leakage-distribution function of LIF neurons through a state machine comprising five states. Compared with a traditional neuron circuit, the neuron circuit provided by the invention has a simpler calculation rule and better expandability, and can be compatible with diversified application scenes.

Description

Technical Field

[0001] The invention relates to the technical field of integrated circuits, and in particular to a LIF digital neuron model based on a Flash device. Background Art

[0002] As the basic component of neural networks, neurons are of great significance in any form of artificial network architecture. The design of neuron circuits directly determines whether the entire network architecture can operate correctly and complete the expected tasks. For the third-generation artificial neural network, the spiking neural network, the signal processed by the neuron is a sparse Spike sequence, and the multiplication and addition calculation with the weights needs to be completed on the synapses.

[0003] Through the study of various artificial neuron models, the LIF (Leaky Integrate and Fire) model reduces the accuracy of the description of biological characteristics, but improves the feasibility of the model in practical applications. It is currently the neuron model selected by the vast majority of digital pulse neural network systems. Summary of the invention

[0004] The purpose of the present invention is to provide a LIF digital neuron model based on Flash devices to solve the problems in the background technology.

[0005] In order to solve the above technical problems, the present invention provides a LIF digital neuron model based on Flash devices, which is adapted to the synaptic circuit to complete the digital circuit simulation research and development of the accumulation and release of input weight data; the LIF digital neuron model includes:

[0006] LIF neuron modeling: The signal processed by the neuron is a sparse Spike sequence, and the multiplication and addition calculation with the weights needs to be completed on the synapse; the LIF neuron model is selected for simplification to complete the design of the digital neuron circuit model adapted to the spiking neural network; the membrane capacitance and membrane resistance are introduced to abstract the operation of sodium and potassium ions inside and outside the cell membrane;

[0007] The membrane potential update module converts the LIF equation into a digital circuit-friendly form by using a discretization method, quantizes the floating-point number into an 8-bit fixed-point decimal, and implements the accumulation-leakage-release function of the LIF neuron through a state machine containing 5 states.

[0008] In one embodiment, the LIF neuron modeling includes: the task of LIF modeling is to establish a model that describes the changes in membrane potential over time and with external inputs, without deliberately describing the specific movement of ions during the membrane potential change, and directly focusing on the two major scenarios that cause membrane potential changes: membrane capacitance and membrane resistance; based on the neuron membrane potential iterative equation proposed by the LIF model, the accuracy of the description of biological characteristics is reduced, and the feasibility of the model in practical applications is improved.

[0009] In one embodiment, the membrane potential update module includes: mapping the key parameters of the membrane potential to the digital circuit based on the LIF neuron membrane potential iteration equation; quantizing the membrane potential, input signal, intermediate calculation data and all preset parameters in the LIF circuit. The designed LIF circuit only performs multiplication and addition calculations on the data. Decimals and integers have the same logic in multiplication and addition calculations. Therefore, this rule for quantization will not produce erroneous calculations and meets the circuit design requirements.

[0010] In one embodiment, the LIF neuron membrane potential iterative equation includes: based on the cell membrane potential analysis of biological nerve cells, the charge slowly passes through the cell membrane over time, and the process of cell membrane potential change is modeled using a basic circuit of a capacitor and a resistor in parallel, thereby obtaining a passive membrane equation of the LIF model, which describes the change law of neuron cell membrane potential under current excitation; the equation is discretized by the method of calculus to obtain a digital logic-friendly LIF neuron membrane potential iterative equation.

[0011] In one implementation, the digital circuit is designed using a finite state machine, which includes five state machines: start, mul, add, comp, and mem;

[0012] (1) start: the initialization state of the LIF neuron circuit. The state transition is controlled by the reset signal rst and the input pointer flagb;

[0013] (2) mul: the decay state of the LIF neuron circuit, corresponding to the leakage behavior of real biological neurons, and the state transition is controlled by the reset signal rst;

[0014] (3) add: the iterative state of the LIF neuron circuit, corresponding to the membrane potential update of real biological neurons stimulated by external current, and the state transition is controlled by the reset signal rst;

[0015] (4) comp: the membrane potential comparison state of the LIF neuron circuit, which corresponds to the action potential generation mechanism of real biological neurons. The state transition is controlled by the reset signal rst;

[0016] (5) mem: The temporary state of the membrane potential of the LIF neuron, which corresponds to the resting state of a real biological neuron. The state transition is controlled by the reset signal rst and the input pointer flagb.

[0017] The present invention provides a LIF digital neuron model based on Flash devices, which can be adapted to the synaptic circuit to complete the digital circuit simulation research and development of the accumulation and release of input weight data. The present invention can be integrated with the analog synaptic array or the digital synaptic array with the cooperation of the functional module, and reserves the general communication and control port signals for the outside. Compared with the traditional neuron circuit, it has simpler calculation rules and better scalability, and is compatible with a variety of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a pulse neuron modeling diagram of the LIF digital neuron model based on the Flash device provided by the present invention.

[0019] Figure 2 The invention provides a LIF circuit data quantization rule diagram of a LIF digital neuron model based on a Flash device.

[0020] Figure 3 The invention discloses a LIF state transition diagram of a LIF digital neuron model based on a Flash device.

[0021] Figure 4 It is a schematic diagram of the LIF state transfer timing of the LIF digital neuron model based on the Flash device provided by the present invention.

[0022] Figure 5 The invention discloses a LIF neuron circuit working flow chart of a LIF digital neuron model based on a Flash device. DETAILED DESCRIPTION

[0023] The following is a further detailed description of a LIF digital neuron model based on a Flash device proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer according to the following description. It should be noted that the accompanying drawings are all in a very simplified form and are not in precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0024] like Figure 1 As shown, the present invention provides a LIF digital neuron model based on Flash devices, which can be adapted to the synaptic circuit to complete the digital circuit simulation development of the accumulation-release of input weight data.

[0025] The full name of the LIF model is the Leaky Integrate and Fire Model. It is a bionic neuron model proposed based on neurodynamics. Leaky means that if there is only one neuron input and it is not enough to make the membrane potential exceed the threshold, the membrane potential will automatically leak and gradually fall back to the resting state due to the continuous ion exchange inside and outside the cell membrane; Integrate means that the neuron will receive all pulses sent by the axon terminals connected to it and integrate all received pulse signals; Fire means that when the membrane potential exceeds the threshold, the neuron will send a pulse and the membrane potential will fall back to the resting potential Vrest. The task of the LIF model is to establish a model that describes the changes of membrane potential over time and external input. It is a simplified version of the HH (Hodgkin-Huxley) model. The simplification process is to "solve the main contradiction" and "ignore the secondary contradiction", that is, the model does not deliberately describe the specific movement of ions during the change of membrane potential, but directly focuses on the two major scenarios that cause the change of membrane potential: membrane capacitance and membrane resistance. The introduction of membrane resistance is an abstraction of the movement of sodium, potassium and other ions inside and outside the cell membrane. Compared with the HH model, the LIF model reduces the accuracy of the description of biological characteristics, but improves the feasibility of the model in practical applications and effectively solves the "main contradiction". It is the neuron model selected by the vast majority of digital pulse neural network systems. The following is an analysis of a hardware-friendly LIF model.

[0026] Nerve cells are surrounded by a membrane, a lipid bilayer, that separates the conductive saline solution inside the neuron from the extracellular medium. Figure 1 As shown in A, if a current stimulus I(t) is injected into the neuron, the additional charge q = ∫I(t)dt must go somewhere, which is equivalent to charging the nerve cell. The cell membrane is like an imperfect insulator, and the charge can slowly pass through the cell membrane over time. Therefore, the process of cell membrane potential change can be modeled by a basic circuit of a capacitor in parallel with a resistor. The circuit is driven by I(t), and Urest represents the size of the membrane potential U(t) when there is no external stimulation (resting potential).

[0027] The established circuit is analyzed below. According to the law of conservation of current: I(t) = I R +I C , where I R is the current flowing through the resistor R. According to Ohm’s law, I R =U R / R,U R is the voltage across the resistor R. According to Kirchhoff’s law, U R =U(t)-U rest IC is the current flowing through the capacitor C. From the definition of a capacitor, we can get I C =dq / dt=C[dU(t) / dt]. Therefore:

[0028]

[0029] Multiplying both sides of the equation (1) by the time constant will give the standard form of the LIF model:

[0030]

[0031] Where RC = τ m ; Formula (2) is called the passive membrane equation of the LIF model, which describes the change law of the neuron cell membrane potential under current excitation. However, since digital circuits can only process discrete quantities, this linear differential equation obviously cannot be directly designed by digital circuits. In this regard, the present invention discretizes the above formula using the calculus method, which is more intuitive and accurate than the traditional forward Euler method and CORDIC method, and finally converts the above formula into a digital logic-friendly iterative equation, as follows:

[0032] The rate of change of membrane potential with time at time t, dU(t) / dt, can be expressed as [U(t)-U(t-Δt)] / Δt when Δt approaches 0, where the smaller Δt is, the higher the degree of approximation of the latter to the former. Replace dU(t) / dt with [U(t)-U(t-Δt)] / Δt, and we can get:

[0033]

[0034] In digital circuits, the smallest time interval is one CLK. Therefore, when designing a circuit, Δt in the above formula can be equivalent to a basic time step, and then formula (3) can be transformed into the following form:

[0035]

[0036] Formula (4) is the final iterative equation of neuron membrane potential, which describes the process of continuous accumulation of membrane potential U(t) under the action of stimulus I(t) over time.

[0037] According to the modeling results of the LIF neuron model, the present invention designs a membrane potential update module. Under the premise of fully considering the bionic characteristics, the key parameters in the above formula are R=20, C=1, τ m =20, Urest=0 is mapped to the digital circuit design, and the accumulation-leakage rule of the membrane potential circuit is simplified to the following formula:

[0038] U(t)=0.95U(t-1)+I(t) (5)

[0039] Since digital logic can only process integer data, the calculation process obviously requires the participation of floating-point decimals. In order to prevent data overflow and minimize calculation errors with limited hardware resources, the present invention quantizes the membrane potential, input signal, intermediate calculation data and all preset parameters in the LIF circuit into 16-bit binary integers to represent signed 8-bit fixed-point decimals. The quantization rules are as follows: Figure 2 shown.

[0040] For any 32-bit floating point number, multiply it by 2^8, discard the decimal places of the result and keep only the integer places, and then multiply the integer obtained by Figure 2 The rules shown are converted into 16-bit signed binary numbers to complete the mapping of data to digital circuits. The actual value of the obtained 16-bit signed binary number in the actual calculation of the circuit is an integer equal to q, but the representative value is a signed decimal equal to x. The minimum quantization accuracy is 2^(-8), and the representable range is -255.9921875 to +255.9921875. Since the designed LIF circuit actually only performs multiplication and addition calculations on data, and decimals and integers have the same logic in multiplication and addition calculations, quantization using this rule will not produce incorrect calculations, which meets the circuit design requirements.

[0041] The circuit is designed using a finite state machine, which includes five states: start, mul, add, comp and mem. The state transition is as follows: Figure 3 As shown, the state transfer sequence during normal operation is as follows Figure 4 The specific transfer conditions and internal logic are as follows:

[0042] (1) start: The initialization state of the LIF neuron circuit. The state transition is controlled by the reset signal rst and the input pointer flagb. When rst is pulled low, the circuit returns to the start state from any current state; when rst is high, if flagb is 1, the circuit transfers from the start state to the mul state, otherwise the circuit remains in the start state. The main operation of the start state is to clear the 16-bit membrane potential register, the output Spike_out, and all intermediate registers involved in the circuit calculation;

[0043] (2) mul: The attenuation state of the LIF neuron circuit, corresponding to the leakage behavior of real biological neurons. The state transition is controlled by the reset signal rst. When rst is pulled low, the circuit transitions from the mul state to the start state; when rst is high, the circuit transitions from the mul state to the add state. The main operation of the mul state is to convert the original code of the current membrane potential Ut with an attenuation coefficient (1-τ m) performs signed multiplication calculation and assigns the result to the 16-bit intermediate register midel_mul. The attenuation coefficient of this design is set to 0.95, which is quantized to 16'b0000000011110011, with a quantization error of 0.08%;

[0044] (3) add: the iterative state of the LIF neuron circuit, corresponding to the membrane potential update of real biological neurons stimulated by external current. The state transition is controlled by the reset signal rst. When rst is pulled low, the circuit transfers from the add state to the comp state; when rst is high, the circuit transfers from the mul state to the add state. The add state mainly performs signed addition of the attenuated membrane potential and the input I(t) signal, and updates the result to the membrane potential register;

[0045] (4) comp: The membrane potential comparison state of the LIF neuron circuit corresponds to the action potential generation mechanism of real biological neurons, and the state transition is controlled by the reset signal rst. When rst is pulled low, the circuit transfers from the comp state to the comp state; when rst is high, the circuit transfers from the comp state to the mem state. The main function of the comp state is to judge the pulse release of the LIF neuron, that is, to compare the updated membrane potential Ut with the preset threshold voltage Uth of +1. When Ut>=Uth, the output signal Spike_out of the LIF is set to 1, otherwise Spike_out is set to 0;

[0046] (5) mem: The temporary state of the membrane potential of the LIF neuron, which corresponds to the resting state of a real biological neuron. The state transition is controlled by the reset signal rst and the input pointer flag. When rst is pulled low, the circuit returns from the comp state to the start state; when rst is high, if flag is 1, the circuit transfers from the comp state to the mul state and starts the next membrane potential update. If flag is 0, the circuit remains in the start state waiting for the pointer signal to arrive. In addition, the mem state sends an output flag signal out_flag_LIF that lasts for one beat and an output flag signal out_flag that lasts for n beats, which are used for the interaction between the external circuit and the LIF neuron during system integration.

[0047] The working process of the designed LIF neuron circuit under the continuous input I(t) excitation is as follows Figure 5As shown. When the enable is pulled high, the LIF circuit clears the internal register, completes the initialization and waits for the arrival of the input pointer flagb. When flagb is pulled high, LIF enters the calculation state, and according to the provisions of formula (5), it passes through the mul and add states in sequence to update the membrane potential, and then enters the pulse emission judgment. After generating an output signal and a flag signal to the outside, the resting potential or the updated potential is sent to the mem state cache, waiting for the start of the next calculation.

[0048] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A LIF digital neuron model based on Flash device, characterized in that: Adapt with synaptic circuit to complete the digital circuit simulation development of accumulation and release of input weight data; The LIF digital neuron model includes: LIF neuron modeling: The signal processed by the neuron is a sparse Spike sequence, and the multiplication and addition calculation with the weights needs to be completed on the synapse; the LIF neuron model is selected for simplification to complete the design of the digital neuron circuit model adapted to the spiking neural network; the membrane capacitance and membrane resistance are introduced to abstract the operation of sodium and potassium ions inside and outside the cell membrane; The membrane potential update module converts the LIF equation into a digital circuit-friendly form by using a discretization method, quantizes the floating-point number into an 8-bit fixed-point decimal, and implements the accumulation-leakage-release function of the LIF neuron through a state machine containing 5 states.

2. The LIF digital neuron model based on Flash device as claimed in claim 1, characterized in that: The LIF neuron modeling includes: the task of LIF modeling is to establish a model that describes the changes of membrane potential over time and external inputs, without deliberately describing the specific movement of ions during the membrane potential change, and directly focusing on the two major scenarios that cause the membrane potential change: membrane capacitance and membrane resistance; based on the neuron membrane potential iterative equation proposed by the LIF model, the accuracy of the description of biological characteristics is reduced, and the feasibility of the model in practical applications is improved.

3. The LIF digital neuron model based on Flash device as claimed in claim 2, characterized in that: The membrane potential update module includes: mapping the key parameters of the membrane potential to the digital circuit based on the LIF neuron membrane potential iteration equation; quantizing the membrane potential, input signal, intermediate calculation data and all preset parameters in the LIF circuit. The designed LIF circuit only performs multiplication and addition calculations on the data. Decimals and integers have the same logic in multiplication and addition calculations. Therefore, this rule of quantization will not produce erroneous calculations and meets the circuit design requirements.

4. The LIF digital neuron model based on Flash device as claimed in claim 3, characterized in that: The LIF neuron membrane potential iteration equation includes: based on the cell membrane potential analysis of biological nerve cells, the charge slowly passes through the cell membrane over time, and the process of cell membrane potential change is modeled by using a basic circuit of a capacitor and a resistor in parallel, thereby obtaining a passive membrane equation of the LIF model, which describes the change law of neuron cell membrane potential under current excitation; the equation is discretized by the calculus method to obtain the digital logic-friendly LIF neuron membrane potential iteration equation.

5. The LIF digital neuron model based on Flash device as claimed in claim 4, characterized in that: The digital circuit is designed using a finite state machine, which includes five state machines: start, mul, add, comp and mem; (1) start: the initialization state of the LIF neuron circuit. The state transition is controlled by the reset signal rst and the input pointer flagb; (2) mul: the decay state of the LIF neuron circuit, corresponding to the leakage behavior of real biological neurons, and the state transition is controlled by the reset signal rst; (3) add: the iterative state of the LIF neuron circuit, corresponding to the membrane potential update of real biological neurons stimulated by external current, and the state transition is controlled by the reset signal rst; (4) comp: the membrane potential comparison state of the LIF neuron circuit, which corresponds to the action potential generation mechanism of real biological neurons. The state transition is controlled by the reset signal rst; (5) mem: The temporary state of the membrane potential of the LIF neuron, which corresponds to the resting state of a real biological neuron. The state transition is controlled by the reset signal rst and the input pointer flagb.

Images