A CMOS-based analog pulse neuron circuit

Abstract

The present invention discloses a neuron circuit based on analog pulses, which belongs to the technical field of integrated circuits. The circuit includes: an integral accumulation module, a threshold emission module and a voltage discharge module; the integral accumulation module, the threshold emission module and the voltage discharge module are connected to each other in sequence, and the voltage discharge module discharges the integral voltage to a fixed value after each neuron emits a pulse. The neuron circuit proposed in the present application can prevent the loss of input information, and maximize the integrity of the input information and the working accuracy of the neural network; each module uses only a few MOS tubes and capacitor and resistor devices to realize the basic functions of the neuron, with fewer external input control signals, and the devices used have low power consumption, which is convenient for implantable applications or portable applications; the same reference source can provide discharge current for multiple neurons at the same time, which is conducive to the trend of ultra-large-scale development of neural networks.

Description

Technical Field

[0001] The invention relates to a CMOS-based analog pulse neuron circuit, belonging to the technical field of integrated circuits. Background Art

[0002] Storage and computing integrated neural network technology has been applied to a wide range of artificial intelligence application scenarios, such as image recognition, image processing and other fields. It is considered to be the most effective way to solve the storage wall problem in the von Neumann architecture. It mainly includes convolutional neural networks (CNN), spiking neural networks (SNN), spiking convolutional neural networks (SCNN), etc.

[0003] In a spike-based neural network, data transmission between arrays is achieved by using spikes generated by neuron circuits, replacing the traditional convolutional neural network that uses a large number of ADCs (analog-to-digital converters) to convert the analog output of the array into digital output for transmission between arrays. The working process of neurons is as follows: Figure 1 As shown, the input data is converted into output current I through the storage array OUT , and then converted into an integral voltage V through an integral accumulation circuit OUT After the integration is completed, the comparator is connected to the external reference voltage V TH For comparison, if V OUT Reach V TH , then the comparator output is high, a pulse is issued, and then the integrated voltage is discharged to the ground, waiting for the next integration stage. If V OUT Does not exceed the comparator reference voltage V TH , then the comparator output is low level and the integrated voltage is in the hold state.

[0004] At the end of an integration period, the integrated voltage may have two states: 1. The integrated voltage V OUT <V TH , the output of the comparator does not flip after entering the comparison stage, and the integral voltage remains in the state. Second, the integral voltage V OUT ≥V TH , after the comparator enters the comparison stage, it outputs a pulse with the same pulse width as the comparison stage width, and then discharges the integrated voltage to zero.

[0005] From the working process of the above neurons, it can be seen that the pulse emission frequency is closely related to the input data, and the loss of pulses means the loss of input data. OUTSlightly higher than V TH , then even if the voltage is discharged to zero, there will be no significant loss of information. OUT Much higher than V TH At this time, discharging the voltage to zero will cause excessive voltage discharge, which will cause information loss.

[0006] There is also a limit case where, at the end of a certain integration period, V OUT Slightly lower than V TH , the neuron will not output a pulse. In the next integration period, I OUT is full-scale output, then at the end of this integration period, V OUT ≈2V TH If the integral voltage is still discharged to zero at this time, it will cause the loss of nearly a full-scale information, and with the continuous input of data, this kind of information loss will become more and more, which will affect the working accuracy of the entire neural network. Summary of the invention

[0007] In order to ensure the integrity of input information and improve the working accuracy of a neural network, the present invention provides a neuron circuit based on analog pulses, the circuit comprising: an integral accumulation module, a threshold release module and a voltage discharge module.

[0008] The integral accumulation module, the threshold issuing module and the voltage discharge module are connected to each other in sequence.

[0009] The integral accumulation module integrates and accumulates the input current and converts it into an integral voltage, and sends the integral voltage to the threshold emission module; the threshold emission module compares the integral voltage with the threshold voltage, and if the integral voltage is higher than the threshold voltage, a pulse signal is emitted; if the integral voltage is lower than the threshold voltage, no pulse signal is emitted, and the integral voltage is in a maintained state; the voltage discharge module discharges the integral voltage to a fixed value after each neuron emits a pulse.

[0010] Optionally, the integral accumulation module includes an integral switch, an integral capacitor and a reset switch; one end of the integral switch is connected to the input current, and the other end is connected to the upper plate of the integral capacitor, the lower plate of the integral capacitor is grounded, and the reset switch is connected in parallel with the integral capacitor.

[0011] Optionally, the threshold emission module includes a comparator; the positive input terminal of the comparator is connected to the upper plate of the integrating capacitor in the integral accumulation module, and the reverse input terminal is connected to an external input voltage as the neuron pulse emission threshold.

[0012] Optionally, the voltage discharge module includes a discharge switch and a discharge constant current source; one end of the discharge switch is connected to the upper board of the integral capacitor of the integral accumulation module, and the other end is connected to the discharge constant current source; the state of the discharge switch is controlled by the output of the comparator in the threshold issuance module.

[0013] Optionally, the integral accumulation module includes an integral switch NMOS tube NM2, an integral capacitor C and a reset switch NMOS tube NM1; the threshold issuance module is composed of a comparator; the voltage discharge module is composed of a discharge switch NMOS tube NM3 and a discharge constant current source I LEAK composition.

[0014] The NM1 is connected in parallel with the integrating capacitor C, the drain of the NM2 is connected to the input current, the source is connected to the upper plate of the integrating capacitor C and the drain of the NM3, and is connected to the positive input terminal of the comparator, and the negative input terminal of the comparator is connected to the external input voltage V TH As the pulse emission threshold voltage, the source of NM3 is connected to the discharge constant current source I LEAK , I LEAK The direction is from the drain to the source of NM3, and the gate of NM3 is connected to the comparator output port.

[0015] Optionally, the input current of the integral accumulation module is the current output by the storage and calculation array.

[0016] Optionally, the fixed value of V LEAK The calculation formula is:

[0017]

[0018] Where C is the integrating capacitor, I LEAK is the constant current source current, and t is the pulse duration.

[0019] Optionally, the fixed value is equal to a threshold voltage of a threshold issuance module.

[0020] Optionally, the integral switch, reset switch and discharge switch are CMOS single-tube switches.

[0021] Optionally, the constant current source is a conventional self-biased current source that is independent of the power supply.

[0022] The present invention also provides an application of the above-mentioned neuron circuit based on analog pulses in the field of integrated circuit technology.

[0023] The beneficial effects of the present invention are:

[0024] In view of the problem that input information is easily lost and the network working accuracy is affected in the current pulse neural network, the present application proposes a neuron circuit based on analog pulses. The input information is processed by an integral accumulation module, a threshold emission module and a voltage discharge module to realize functions such as integral accumulation and pulse emission. After the pulse is emitted, the integral voltage is discharged to a fixed value to prevent information loss, ensure the integrity of the input information, and improve the accuracy of the neural network. Each module of the present invention only uses a few CMOS tubes and capacitor and resistor devices to realize the basic functions of neurons, and there are fewer external input control signals. The power consumption of the comparator, self-biased current source and other circuits used is also very low, which is convenient for implantable or portable applications. The voltage discharge module can provide voltage discharge function for multiple neurons at the same time, which is conducive to the trend of neural networks developing in a large-scale direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 This is a schematic diagram of the working process of neurons in a pulse-based neural network.

[0027] Figure 2 It is a schematic diagram of the neuron circuit framework of the present invention.

[0028] Figure 3 It is a schematic diagram of the structure of a specific embodiment of the neuron circuit of the present invention.

[0029] Figure 4 It is a schematic diagram of the structure of a constant current source in a specific embodiment of a neuron circuit of the present invention. DETAILED DESCRIPTION

[0030] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0031] Embodiment 1:

[0032] This embodiment provides a CMOS-based analog pulse neuron circuit for data transmission in a pulse neural network. Figure 2 The circuit includes: an integral accumulation module, a threshold issuance module and a voltage discharge module.

[0033] The integral accumulation module is connected to the current output port of the storage and calculation array, and is used to continuously integrate and accumulate the output current and send the integral voltage signal to the threshold emission module; the threshold emission module is connected to the integral accumulation module, and is responsible for comparing the integral voltage after each integration cycle with the threshold voltage. If the integral voltage is higher than the threshold voltage, the neuron is activated and a pulse signal is emitted. If the integral voltage is less than the threshold voltage, no pulse signal is emitted and the integral voltage is in a holding state; the voltage discharge module is responsible for discharging the integral voltage to a fixed value after each neuron emits a pulse, and the fixed value is equal to the threshold emission voltage of the neuron.

[0034] like Figure 3 As shown, the integral accumulation module includes an integral switch NMOS tube NM2, an integral capacitor C and a reset switch NMOS tube NM1; the threshold release module is composed of a comparator; the voltage discharge module is composed of a discharge switch NMOS tube NM3 and a discharge constant current source I LEAK composition.

[0035] Among them, NM1 is connected in parallel with the integral capacitor C, the drain of NM2 is connected to the current output port of the storage array, the source is connected to the upper board of the integral capacitor C and the drain of NM3, and is connected to the positive input terminal of the comparator, and the reverse input terminal is connected to the external input voltage V TH As the pulse emission threshold voltage, the source of NM3 is connected to the constant current source I LEAK , I LEAK The direction is from the drain of NM3 tube to the source, and the gate of NM3 tube is connected to the output port of the comparator.

[0036] like Figure 4 As shown, the constant current source in this embodiment is a conventional constant current source that is independent of the power supply. The constant current source is composed of PMOS tubes PM1 and PM2, NMOS tubes NM4 and NM5, and resistor R; PMOS tube PM3, NMOS tubes NM6 and NM7 are responsible for changing the current direction, and the drain of NM7 is connected to the source of NM3 in the voltage discharge module. Among them, the width-to-length ratio of PM1, PM2, and PM3 tubes is equal, the width-to-length ratio of NM4 and NM5 tubes is equal, and the width-to-length ratio of NM6 and NM7 tubes is equal. The required current source is obtained by adjusting the value of R. In the figure, VSS is a negative voltage source, the purpose of which is to ensure that the NM4 tube is always in a saturated state during operation to ensure that I REF accuracy.

[0037] The circuit working process of this embodiment is as follows:

[0038] (1) Reset stage: NM1 is turned on, NM2 is turned off, and the integral capacitor C is discharged. At this time, the comparator is in the reset stage, the output is low, and NM3 is in the off state.

[0039] (2) Integration stage: NM2 is turned on, NM1 is turned off, and the integration capacitor C is used to integrate the array output current IOUT Integration is performed, at this time the comparator is still in the reset stage, the output is low, and NM3 remains in the off state.

[0040] (3) Comparison stage: After the integration is completed, the comparator enters the comparison stage. If the integrated voltage does not reach the threshold voltage V TH When the neuron is not activated, the comparator output is still low; when the integrated voltage reaches the threshold voltage V TH When the neuron is activated, the comparator output flips to a high level, and a pulse is emitted. The pulse duration is equal to the duration of the comparison phase of the comparator. In this embodiment, this value is equal to t. During the pulse duration, NM3 is turned on, and the integral capacitor C is connected through the constant current source I in the voltage discharge module. LEAK Discharge to prepare for the next integration stage, discharge voltage When the integral capacitor C is given, it can be adjusted by adjusting I LEAK and the size of t, let · V LEAK =V TH , so that the neuron discharges the integrated voltage to a fixed value in each voltage discharge stage, that is, the threshold discharge voltage V TH , in order to ensure the integrity of the input information and the working accuracy of the neural network.

[0041] If it is necessary to provide a discharge current I for multiple neurons in practical applications LEAK , then in this embodiment, it is only necessary to increase the number of current replicators, such as Figure 4 NMOS tube NM7~NMn tube and constant current source I LEAKn shown.

[0042] Compared with the existing neural circuit, the neural circuit proposed in the present invention can prevent the loss of input information, and maximize the integrity of the input information and the working accuracy of the neural network: after model building and simulation comparison of different voltage discharge methods, it can be concluded that it is consistent with the paper "Y.Xiang et al, Efficient and Robust Spike-Driven Deep Convolutional Neural Networks Based on NOR Flash Computing Array, IEEE Transactions on Electron Devices, vol. 67, no. 6, pp. 2329-2335, June 2020.", "YC Xiang et al, Analog Deep Neural Network Based on NORFlash Computing Array for High Speed / Energy Efficiency Computation, 2019 IEEE International Symposium on Circuits and Systems (ISCAS), 2019, pp. 1-4." and the patent "CN 108446762 Compared with the voltage discharge method in which the neuron integrated voltage is directly discharged to the ground in "A hardware circuit of simulated pulse neurons based on MOS field effect transistors and its application", it can be found that the voltage discharge method in the present invention can improve the accuracy of the neural network by 10% to 20%; the circuit structure is simple, the power consumption is low, and it is convenient for implantable applications and portable applications; the same reference source can provide discharge current for multiple neurons at the same time, which is conducive to the trend of ultra-large-scale development of neural networks.

[0043] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0044] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A neuron circuit based on analog pulses, characterized in that: The circuit comprises: an integral accumulation module, a threshold issuance module and a voltage discharge module; The integral accumulation module, the threshold issuance module and the voltage discharge module are connected to each other in sequence; The integral accumulation module integrates and accumulates the input current into an integral voltage, and sends the integral voltage to the threshold emission module; the threshold emission module compares the integral voltage with the threshold voltage, and if the integral voltage is higher than the threshold voltage, a pulse signal is emitted; if the integral voltage is lower than the threshold voltage, no pulse signal is emitted, and the integral voltage is in a holding state; the voltage discharge module discharges the integral voltage to a fixed value after each neuron emits a pulse; The integral accumulation module includes an integral switch NMOS tube NM2, an integral capacitor C and a reset switch NMOS tube NM1; the threshold issuance module is composed of a comparator; the voltage discharge module is composed of a discharge switch NMOS tube NM3 and a discharge constant current source I LEAK composition; The NM1 is connected in parallel with the integrating capacitor C, the drain of the NM2 is connected to the input current, the source is connected to the upper plate of the integrating capacitor C and the drain of the NM3, and is connected to the positive input terminal of the comparator, and the negative input terminal of the comparator is connected to the external input voltage V TH As the pulse emission threshold voltage, the source of NM3 is connected to the discharge constant current source I LEAK , I LEAK The direction is from the drain to the source of NM3, and the gate of NM3 is connected to the comparator output port.

2. The circuit according to claim 1, characterized in that The input current of the integral accumulation module is the current output by the storage and calculation array.

3. The circuit according to claim 2, characterized in that The integrated voltage is discharged to a fixed value, which is equal to the threshold voltage of the threshold issuing module.

4. The circuit according to claim 3, characterized in that The fixed value V LEAK The calculation formula is: Where C is the size of the integrating capacitor and t is the pulse duration.

5. The circuit according to claim 4, characterized in that The discharge constant current source is a conventional self-biased current source that is independent of the power supply.

6. Application of the analog pulse based neuron circuit described in claims 1-5 in the field of integrated circuit technology.

Images