Product-sum circuit and neural network equipped with the product-sum circuit

Abstract

To provide a product sum circuit capable of being used for neuron computing and a neural network including the product sum circuit.SOLUTION: A product sum circuit 10 is a circuit for performing product-sum operations. The product sum circuit 10 includes a series circuit of a memristor M1 and a capacitator C1. The memristor M1 is an analog memristor whose conductance can change continuously. The capacitator C1 is connected in series after the memristor M1. The product sum circuit 10 may be connected in parallel. The product sum circuit 10 may be integrated into the neural network to be made to function as a synaptic circuit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a product-sum circuit and a neural network including the product-sum circuit. [Background technology]

[0002] While software and hardware related to artificial intelligence are being developed, the software is becoming more complex and large, and the hardware that runs the software is becoming more sophisticated, resulting in a problem of enormous power consumption.

[0003] Furthermore, Non-Patent Document 1 below discloses a circuit equipped with a memristor and a capacitor. The memristor operates by changing the conductance between two states, on and off, and is compatible only with summing functions (sum circuits). It is not compatible with the product-sum operations used in neuron operations. [Prior art documents] [Patent documents]

[0004] [Non-Patent Document 1] Z. Wang, M. Rao, J.-W. Han, J. Zhang, P. Lin, Y. Li, C. Li, W. Song, S. Asapu, R. Midya, Y. Zhuo, H. Jiang, JH Yoon, NK Upadyay, S. Joshi, M. Hu, JP Strachan, M. Barnell, Q. Wu, H. Wu, Q. Qiu, RS Williams, Q. Xia, and JJ Yang, Capacitive Neural Network with Neuro-Transistors, Nat Commun., 9, 3208, 2018. Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to provide a product-sum circuit that can be used for neuron operations and a neural network that uses the product-sum circuit. [Means for solving the problem]

[0006] The product-sum circuit of the present invention includes a memristor whose conductance can be continuously changed and a capacitor connected in series to the downstream of the memristor. A plurality of such series-connected circuits of a memristor and a capacitor may be connected in parallel.

[0007] The neural network of the present invention comprises a synapse circuit having the above-described sum-of-products circuit and a neuron circuit connected to the output of the synapse circuit. The neural network of the present invention may also comprise the above-described sum-of-products circuit and a neuron circuit connected to the output of the sum-of-products circuit. [Effects of the Invention]

[0008] According to the present invention, a product-sum circuit capable of performing product-sum operations can be configured using analog memristors and capacitors, and can be used for neuron operations. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a circuit diagram showing a product-sum circuit according to the present invention; [Figure 2] 2 is a graph showing the output when a spike pulse is input to the circuit of FIG. 1, where (a) shows the timing of inputting the spike pulse, and (b) shows the output of the circuit of FIG. 1. [Figure 3] FIG. 10 is a circuit diagram in which a plurality of product-sum circuits are connected in parallel. [Figure 4] 4 is a graph showing the output when a spike pulse is input to the circuit of FIG. 3, where (a) shows the timing of inputting the spike pulse, and (b) shows the output of the circuit of FIG. 3. [Figure 5] FIG. 1 is a circuit diagram showing a charge pump circuit including a product-sum circuit. [Figure 6] FIG. 1 is a circuit diagram showing an example of a neuron circuit. [Figure 7] FIG. 10 is a diagram showing an example in which the outputs of a plurality of synapse circuits are connected to one neuron circuit. DETAILED DESCRIPTION OF THE INVENTION

[0010] The product-sum circuit and neural network of the present invention will be described with reference to the drawings.

[0011] [Embodiment 1] 1 is a circuit for performing a product-sum operation. The product-sum circuit 10 includes a series circuit of a memristor M1 and a capacitor C1.

[0012] The memristor M1 is a passive element whose resistance changes when a charge passes through it and can maintain that state. It consists of two electrodes and a memristor layer disposed between the electrodes. The memristor layer can be made of an oxide containing the elements Ti, Zr, Hf, V, Nb, Ta, Mo, W, Cr, Fe, Ni, Co, Sc, Y, or Lu, and can have either a crystalline or polycrystalline structure.

[0013] The memristor M1 used in this invention is an analog memristor whose conductance can be continuously changed. Before using the memristor M1, a voltage is applied to the electrodes of the memristor M1 to adjust its conductance to a desired value. The product-sum circuit 10 uses the memristor M1 after its conductance has been adjusted.

[0014] The capacitor C1 is connected in series to the rear of the memristor M1. The capacitance of the capacitor C1 is constant. Alternatively, the capacitance of the variable capacitance capacitor C1 may be fixed.

[0015] A diode D1 is connected in front of the memristor M1. The anode of the diode D1 is connected to the input terminal, and the cathode of the diode D1 is connected to the memristor M1. The diode D1 prevents backflow of current from the memristor M1. If there is no need to prevent backflow of current, the diode D1 may be omitted.

[0016] A capacitor C2 is connected between the output terminal of the product-sum circuit 10 and the ground. This capacitor C2 maintains the potential of the output terminal.

[0017] Figure 2 shows the output voltage when a spike pulse is input to the product-sum circuit 10. In both Figures 2(a) and (b), the horizontal axis represents time and the vertical axis represents voltage, and the times are the same. The spike pulse on / off ratio is 1:9. For example, the spike pulse width is 10 nanoseconds. While a spike pulse is input, the output voltage rises due to the transient operation of the product-sum circuit 10, and when the spike pulse turns off, the voltage is maintained. The more spike pulses are input, the lower the rising voltage becomes. This is because each time a spike pulse is input, charge accumulates in capacitor C1, making it difficult for current to flow through the product-sum circuit 10. Once capacitor C1 is completely charged, no current flows through the product-sum circuit 10.

[0018] The voltage rise at the output terminal varies depending on the conductance of the memristor M1. The higher the conductance of the memristor M1, the higher the voltage rise with the input of a single spike pulse, and the lower the conductance, the lower the voltage rise with the input of a single spike pulse. The output voltage of the product-sum circuit 10 can be changed by adjusting the conductance of the memristor M1.

[0019] The sum-of-products circuit 10 is formed by connecting an analog memristor M1, whose conductance can be continuously changed, in series with a capacitor C1. The sum-of-products circuit 10 can be used for neuron operations. The sum-of-products circuit 10 can also be incorporated into a neural network to function as a synapse circuit.

[0020] Because the conductance of the memristor M1 can be continuously changed, a product-sum circuit 10 can be realized that can obtain a desired output by adjusting that conductance. The range of change in the memristor M1's conductance is very large, and the range of change in the connection strength as a synapse is also very large. The product-sum circuit 10 can also bring out satisfactory performance from spiking neuromorphics. In addition, the circuit configuration is simple, and the product-sum circuit 10 is stopped except when spike pulses are input, resulting in low power consumption. As described above, it can be used for neuron calculations while maintaining the low power consumption of spiking neuromorphics using memcapacitors.

[0021] [Embodiment 2] As shown in Figure 3, the two product-sum circuits 10 described above may be connected in parallel. A diode D1 is connected to the input side of each product-sum circuit 10. The outputs of the two product-sum circuits 10 are connected to form a single output. A capacitor C2 is connected between this output and ground.

[0022] Figure 4 shows the output voltages when spike pulses are input to the two product-sum circuits 10. In both Figures 4(a) and 4(b), the horizontal axis represents time and the vertical axis represents voltage, and the times are the same. In Figure 4(a), the spike pulse input to one product-sum circuit 10 is shown by a solid line, and the spike pulse input to the other product-sum circuit 10 is shown by a dashed-dotted line. Spike pulses are input alternately to each input. As in embodiment 1, the voltage at the output terminal rises each time a spike pulse is input. As the number of spike pulses input increases, the rise in voltage at the output terminal becomes smaller.

[0023] Even if the product-sum circuits 10 described in the first embodiment are connected in parallel, the parallel circuit as a whole performs a product-sum operation. A circuit in which the product-sum circuits 10 are connected in parallel can be used as a synapse circuit for a neural network.

[0024] The number of product-sum circuits 10 connected in parallel is not limited, and multiple product-sum circuits 10 may be connected in parallel. A diode D1 may be connected to the input side of each product-sum circuit 10 to prevent backflow of current. When multiple product-sum circuits 10 are connected in parallel, all of the product-sum circuits 10 do not need to be identical. The conductance of the memristor M1, the capacitance of the capacitor C1, or both may be changed for each product-sum circuit 10. The value of the voltage that rises when a spike pulse is input may be different for each product-sum circuit 10.

[0025] [Embodiment 3] In the circuits of Figures 1 and 3, capacitor C2 is connected between the output terminal and ground, but other circuit elements may be used. For example, a resistor may be connected instead of capacitor C2. In the case of a resistor, the voltage at the output terminal drops when no spike pulse is input.

[0026] [Embodiment 4] The above-described sum-of-products circuit 10 may be incorporated into another circuit. For example, as shown in FIG. 5, the sum-of-products circuit 10 may be incorporated into a charge pump circuit 12. The charge pump circuit 12 includes a first CMOS transistor 14, a second CMOS transistor 16, the sum-of-products circuit 10, a capacitor C3, and two p-type MOSFETs 18 and 20 connected in series.

[0027] The product-sum circuit 10 is connected between the p-type MOSFET of the first CMOS transistor 14 and the n-type MOSFET of the second CMOS transistor 16. A capacitor C3 is connected between the p-type MOSFET of the second CMOS transistor 16 and the n-type MOSFET of the first CMOS transistor 14. A memcapacitor can be used as the capacitor C3.

[0028] The two series-connected p-type MOSFETs 18 and 20 are connected between the n-type MOSFET of the first CMOS transistor 14 and the n-type MOSFET of the second CMOS transistor 16. The connection terminal between the two p-type MOSFETs 18 and 20 is connected to the output terminal. A capacitor C2 is connected between the output terminal and ground.

[0029] The first CMOS transistor 14 and the second CMOS transistor 16 are alternately turned on to operate as a charge pump circuit. This allows for a larger output voltage than the circuits in the above embodiments (widening the dynamic range). The charge pump circuit can be used as a synapse circuit in a neural network, and its output can be input to a neuron circuit.

[0030] [Embodiment 5] The product-sum circuit 10 may be used as part of a neural network. In a general neural network for character recognition or image recognition, the product-sum circuit 10 can be applied to all layers. For example, the product-sum circuit 10 may be incorporated into a full connection of a neural network consisting of multiple layers. The product-sum circuit 10 can also be applied to single-layer and multi-layer perceptrons, convolutional neural networks, Hopfield networks, cellular neural networks, recurrent neural networks, reservoir neural networks, etc.

[0031] [Embodiment 6] The sum-of-products circuit 10 can be used as a synapse circuit, and the output of the sum-of-products circuit 10 may be connected to a neuron circuit. For example, a neuron circuit 22 comprises a circuit in which four inverters (NOT gates) 24 are connected in series (Fig. 6). A capacitor C4 is connected between the input of each inverter 24 and ground. A switch element (FET) 26 is connected between the input of the neuron circuit 22 and ground, and the gate of the switch element 26 is connected to the output of the neuron circuit 22. By connecting multiple inverters 24 in series, time is taken to connect the input of the neuron circuit 22 to ground.

[0032] If the synapse circuit includes the sum-of-products circuit 10, the output of the circuit including the sum-of-products circuit 10 may be input to the synapse circuit. For example, the output of the circuits in FIGS. 1, 3, and 5 may be input to the synapse circuit.

[0033] [Embodiment 7] In the above embodiment, when a spike pulse is input to the product-sum circuit 10, the output voltage rises. However, the product-sum circuit 10 may be weighted negatively instead of positively. When a spike pulse is input to the product-sum circuit 10, the output voltage falls. The synapse circuits included in the product-sum circuit 10 may be weighted negatively.

[0034] Multiple synapse circuits, each including a product-sum circuit 10, may be connected in parallel, with the output of each synapse circuit being input to a single neuron circuit (FIG. 7). Each synapse circuit may be weighted either positively or negatively. For example, by using a synapse circuit with a positive weighting and a synapse circuit with a negative weighting, the positive or negative state of the output voltage can be controlled.

[0035] [Embodiment 8] In the above embodiment, a case where a positive spike pulse is input has been described, but the present invention also applies when a negative spike pulse is input. In that case, the orientation of the diode D1 described above is reversed.

[0036] [Embodiment 9] The circuits described in the above embodiments may be incorporated into IC chips or the like and integrated with various other circuits. The memristor M1, capacitor C1, and diode D1 are integrated on a single substrate. For example, they are fabricated by sequential film deposition and integrated formation in the following order (1) to (3): (1) Laminate the capacitor C1 by sputtering deposition, mist CVD deposition, and anodization of a normal insulator and high-k material; (2) Laminate the memristor by sputtering deposition and mist CVD deposition of an amorphous metal oxide semiconductor; and (3) Laminate the diode (Schottky diode) D1. Because it is an amorphous metal oxide semiconductor, device characteristics comparable to those achieved by sputtering deposition and mist CVD deposition can be reproduced without damaging the underlying structure. Furthermore, sequential deposition is possible, making it advantageous for integration. Because amorphous metal oxide semiconductors are multi-component materials that can be endowed with a variety of functions, two film deposition processes may be combined into one.

[0037] In addition, the present invention can be implemented in various forms with various improvements, modifications, and changes made based on the knowledge of those skilled in the art without departing from the spirit of the present invention. [Explanation of symbols]

[0038] 10: Product-sum circuit 12: Charge pump circuit 14, 16: CMOS transistor 18, 20: p-type MOSFET 22: Neuron Circuit 24: Inverter circuit 26: Switch element M1: Memristor C1, C2, C4: Capacitors C3: Capacitor D1: Diode

Claims

1. A memristor whose conductance can be continuously changed, a capacitor connected in series to a subsequent stage of the memristor; Equipped with The output voltage can be changed by adjusting the conductance; One terminal of the capacitor is connected to the memristor, and the other terminal is an output terminal. A product-sum circuit in which the voltage rise at the output terminal when a spike pulse is input varies depending on the conductance of the memristor.

2. 2. The product-sum circuit according to claim 1, wherein a plurality of circuits each including a memristor and a capacitor connected in series are connected in parallel.

3. a synapse circuit including the product-sum circuit according to claim 1 or 2; a neuron circuit connected to the output of the synapse circuit; Neural network with.

4. a product-sum circuit according to claim 1 or 2; a neuron circuit connected to the output of the sum-of-products circuit; Neural network with.

Images