A variable pulse width input charge accumulation type memristor neural network circuit

Abstract

The present invention discloses a variable pulse width input charge accumulation type memristor neural network circuit, comprising a pulse width modulation circuit, a polarity control circuit, a programming control circuit, a memristor array, a charge accumulation circuit, an activation function circuit, and a comparison control and analog weight update circuit. The present invention converts the input digital logic value into a pulse width modulation signal through a pulse width modulation circuit, thereby facilitating the improvement of the accuracy of the neural network; the pulse width modulation circuit is combined with the polarity control circuit, and the same direction or reverse integration of the charge accumulation circuit is utilized, so that only a single memristor array can be used to achieve positive, zero or negative weights, and the chip area can be significantly saved; in addition, the weight programming of the memristor array and the resetting of the charge accumulation capacitor can be performed synchronously, which is conducive to the improvement of the operation speed of the hardware circuit.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated circuits, and in particular to a variable pulse width input charge accumulation type memristor neural network circuit. Background Art

[0002] Over the past decade, non-volatile storage technology, one of the hot technologies, has been driven by both academia and industry, and has made great progress, especially resistive random access memory represented by memristors. Its outstanding features are three-dimensional high-density integration, ultra-low power consumption, parallel processing, etc., making it one of the best candidate devices for the implementation of big data artificial intelligence hardware circuits. Memristors have a wide range of applications in memory, logic circuits, analog circuits, neuromorphic, etc. Among them, neuromorphic application research based on memristor neural networks is a hot topic among research hotspots and is also the key to the implementation of artificial intelligence hardware.

[0003] The traditional voltage-mode memristor neural network is characterized by an input voltage V j The amplitude is variable, and current accumulation is achieved after passing through the memristor array. That is, for the jth column of the memristor, the output of the current accumulation I j equal Among them, M ij is the resistance of the memristor in the i-th row and j-th column. As a passive component, the resistance of the memristor M ij is a value that is always greater than zero. In order to obtain negative weight values, it is often necessary to add an auxiliary memristor array of the same size, and use the signal operation of the two memristor arrays to obtain negative weight values. When the scale of the neural network is large, the auxiliary memristor array significantly increases the chip area. Secondly, once the programming of the memristor array is completed, it needs to remain stable before the next programming, which requires the input voltage signal V j The magnitude is smaller than the threshold voltage of the memristor. Unfortunately, for non-binary neural networks, the input V j It is a multi-value or analog continuous signal input. To make all input values ​​lower than the threshold voltage of the memristor, it means that the voltage difference between each input value becomes very small. The lower the threshold voltage of the memristor, the smaller the maximum input voltage V jmax The smaller the adjacent input value V j and V j+1 The smaller the voltage difference △V, the more difficult it is to implement in hardware, so that it is impossible to distinguish adjacent input values. In this case, even if the accuracy of the neural network simulated by software is high, the accuracy of the final hardware neural network will be greatly reduced due to the low accuracy or impossibility of hardware implementation. Therefore, in memristor neural networks with a large input value range, a new method is urgently needed to solve the above problems, so as to easily distinguish the polymorphic values ​​of the input and facilitate hardware circuit implementation. Summary of the invention

[0004] The present invention aims to solve the problem that the traditional voltage-mode memristor neural network circuit is difficult to implement in hardware circuit under polymorphic input values ​​and adjacent input values ​​are difficult to distinguish, and provides a variable pulse width input charge accumulation type memristor neural network circuit.

[0005] To solve the above problems, the present invention is achieved through the following technical solutions:

[0006] A variable pulse width input charge accumulation type memristor neural network circuit comprises a comparison control and analog weight update circuit and at least one layer of neural network circuit.

[0007] Each layer of neural network circuit includes an adjustable clock source, a pulse width modulation circuit, a polarity control circuit, a programming control circuit, a memristor array, a charge accumulation circuit and an activation function circuit. The output of the adjustable clock source is connected to the clock input of the pulse width modulation circuit; s The input terminals form the m circuit of this layer of neural network. s Input terminals, pulse width modulation circuit m s The output terminal is connected to the polarity control circuit m s The first input terminal; the polarity control circuit m s The output terminal is connected to the programming control circuit m s A first input terminal; m of the programming control circuit s The output terminals are connected to the m s The memristor array has n input terminals. s The output terminals are connected to the charge accumulation circuit n s The first input terminal of the charge accumulation circuit is n s The output terminals are connected to the activation function circuit n s Input terminals, activation function circuit n s The output terminals form the nth layer of the neural network circuit. s output terminals.

[0008] The input end of the pulse width modulation circuit of the first layer of neural network circuit forms the total input end of the entire variable pulse width input charge accumulation type memristor neural network circuit; the output end of the activation function circuit of the previous layer of neural network circuit is connected to the input end of the pulse width modulation circuit of the next layer of neural network circuit; the output end of the activation function circuit of the last layer of neural network circuit forms the total output end of the entire variable pulse width input charge accumulation type memristor neural network circuit.

[0009] The number of TAR signal input terminals of the comparison control and analog weight update circuit is the same as the number of output terminals of the last layer of the neural network circuit, which is connected to the given target value signal. The number of Z signal output terminals of the comparison control and analog weight update circuit is It is connected to the polarity control circuit of each layer of the neural network circuit m s The number of P signal output terminals of the comparison control and analog weight update circuit is The m of the programming control circuit connected to each layer of the neural network circuit s A second input terminal, comparison control and analog weight update circuit S IN The number of signal output terminals is The m of the programming control circuit connected to each layer of the neural network circuit s A third input terminal, comparison control and analog weight update circuit S WP The number of signal output terminals is The m of the programming control circuit connected to each layer of the neural network circuit s The number of S signal output terminals of the comparison control and analog weight update circuit is The m of the memristor array connected to each layer of the neural network circuit s ×n s Comparison control and analog weight update circuit S GF The number of signal output terminals is It is connected to the charge accumulation circuit of each layer of the neural network circuit. s A second input terminal, comparison control and analog weight update circuit S GB The number of signal output terminals is It is connected to the charge accumulation circuit of each layer of the neural network circuit. s The number of Y signal input terminals of the comparison control and analog weight update circuit is It connects the activation function circuits of each layer of neural network circuits n s output terminals.

[0010] Where m s is the number of input terminals of the s-th layer neural network circuit; n s is the number of output terminals of the s-th layer of the neural network circuit, s = 1, 2, …, S, where S is the number of layers of the neural network.

[0011] In the above scheme, all layers of neural network circuits share the same adjustable clock source.

[0012] In the above scheme, the pulse width modulation circuit of the first layer of the neural network circuit is a digitally controlled pulse width modulation circuit, and the pulse width modulation circuits from the second layer to the last layer are analog pulse width modulation circuits.

[0013] In the above scheme, the polarity control circuit is composed of m s Each polarity control unit includes a PMOS tube T Jpi , NMOS tube TJni , resistor R Ji0 -R Ji2 , and amplifier A 3i ; PMOS tube T Jpi The drain and NMOS tube T Jni After being connected to the drain of the PMOS tube T, the i-th first input terminal of the polarity control circuit is formed; Jpi The gate and NMOS tube T Jni After the gate of the NMOS tube T is connected, the i-th second input terminal of the polarity control circuit is formed; Jni The source is connected via resistor R Ji1 Connect Amplifier A 3i The reverse input terminal of PMOS tube T Jpi The source is divided into two paths, one through the resistor R Ji2 Connect Amplifier A 3i The other is connected to the inverting input of amplifier A 3i The output terminal; the resistor R Ji0 One end of the amplifier is connected to 3i The same-direction input terminal of the amplifier is grounded. 3i The output end of the polarity control circuit forms the i-th output end; where i = 1, 2, ..., m s , m s is the number of input terminals of the s-th layer neural network circuit.

[0014] In the above scheme, the programming control circuit is composed of m s Each programming control unit includes an NMOS transistor T Bi , T Pi ; NMOS tube T Bi The drain of the NMOS tube T forms the i-th first input terminal of the programming control circuit; Pi The drain of the NMOS tube T forms the i-th second input terminal of the programming control circuit; Bi The source and NMOS tube T Pi After being connected to the source of the NMOS tube T, the i-th output terminal of the programming control circuit is formed; Bi The gate of the NMOS tube T forms the i-th third input terminal of the programming control circuit; Pi The gate of forms the i-th fourth input terminal of the programming control circuit; wherein i=1,2,…,m s , m s is the number of input terminals of the s-th layer neural network circuit.

[0015] In the above scheme, the charge accumulation circuit includes n s Each charge accumulation unit includes an NMOS tube T GFj , TGBj , amplifier A 1j , A 2j , and capacitor C j ; NMOS tube T GFj The drain of amplifier A 1j After being connected to the reverse input terminal of the NMOS tube T, the jth first input terminal of the charge accumulation circuit is formed; GFj The gate of forms the jth second input terminal of the charge accumulation circuit; the capacitor C j One end of the amplifier is connected to 1j The other end is connected to the inverting input of amplifier A 1j The output of amplifier A 1j The output of amplifier A 2j The same-direction input terminal and NMOS tube T GBj The drain of NMOS tube T GBj The gate of the charge accumulation circuit forms the jth third input terminal; the amplifier A 2j After the reverse input terminal and output terminal of are connected, the jth output terminal of the charge accumulation circuit is formed; amplifier A 1j The same-direction input terminal, NMOS tube T GFj The source and NMOS tube T GBj The source of is grounded; where j = 1, 2, ..., n s , n s is the number of output terminals of the s-th layer neural network circuit.

[0016] In the above scheme, the activation function circuit is composed of n s The input end of each activation function unit forms the jth input end of the activation function circuit, and the output end of each activation function unit forms the jth output end of the activation function circuit; wherein j = 1, 2, ..., n s , n s is the number of output terminals of the s-th layer neural network circuit.

[0017] The present invention converts the input digital logic value into a pulse width modulation signal through a pulse width modulation circuit. This variable pulse width input is particularly suitable for discrete multi-value or analog continuous input situations, is very easy to distinguish adjacent input values, and has no restrictions on the value range of the input. The reference pulse width can also be controlled by adjusting the frequency of the adjustable clock source. These are all conducive to improving the accuracy of the neural network and solving the problem that the input value range of the traditional voltage-mode memristor neural network circuit is limited by the threshold voltage of the memristor. The pulse width modulation circuit is combined with the polarity control circuit, and the same direction or reverse integration of the charge accumulation circuit is used, so that only a single memristor array can be used to achieve positive, zero or negative weights. Compared with the two memristor arrays of the traditional voltage-mode memristor neural network circuit, this method significantly saves chip area. Moreover, the weight programming of the memristor array and the reset of the charge accumulation capacitor can be performed synchronously, which is conducive to improving the computing speed of the hardware circuit.

[0018] Compared with the prior art, the present invention has the following characteristics:

[0019] 1. The circuit proposed in the present invention can transform the input value of the neural network into a variable pulse width signal whose pulse width is controlled by the input value. This variable pulse width input is particularly suitable for discrete multi-value or analog continuous input. It is very easy to distinguish adjacent input values, and there is no restriction on the value range of the input. The reference pulse width can also be controlled by adjusting the frequency of the adjustable clock source, which is conducive to improving the accuracy of the neural network. The input value range of the traditional voltage-mode memristor neural network circuit is limited by the threshold voltage of the memristor. The larger the input value range, the more difficult it is to distinguish adjacent input values, resulting in lower accuracy of the neural network.

[0020] 2. The present invention transforms the input into a signal whose pulse width is proportional to the input size through the transformation of the pulse width modulation circuit and the polarity control circuit, and the polarity of the signal depends on the sign of the target weight. The column output of the memristor array is integrated in the same direction or in the opposite direction through the capacitor, and the charge accumulation value on the capacitor is the vector matrix multiplication result of the memristor array. This method can easily realize positive, zero or negative weights on the memristor array. There is no need to usually require two memristor arrays like the traditional voltage-mode memristor neural network circuit. This method only requires one memristor array.

[0021] 3. The weight programming of the memristor array and the resetting of the accumulation capacitor can be performed simultaneously, which is beneficial to improving the computing speed of the hardware circuit. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of a variable pulse width input charge accumulation type memristor neural network circuit.

[0023] Figure 2 This is the schematic diagram of the 1M1T structure memristor array.

[0024] Figure 3 Calculate the equivalent circuit diagram for the forward signal of the kth column of the memristor neural network circuit.

[0025] Figure 4 Run the flow chart for the circuit.

[0026] Figure 5 Schematic diagram of voltage waveform at key nodes. DETAILED DESCRIPTION

[0027] The present invention can actually be suitable for memristor neural networks of any depth (i.e., any number of layers). At the beginning of the design, the number of input terminals and the number of output terminals will be given, but the number of layers of the memristor neural network can be directly given at the beginning of the design, or it can be determined by performance optimization during the design. In practical applications, the size of the number of input terminals m of the neural network, the size of the number of output terminals n, the number of layers of the neural network, and the weight update algorithm running in the comparison control and analog weight update circuit depend on the specific application scenario needs. For example, for a simple 3-class pattern recognition of a 3×3 binary image, a single-layer memristor neural network with 10 inputs and 3 outputs can be used, while the handwritten digit recognition of the MNIST handwritten digit recognition library usually requires a single-layer or multi-layer memristor neural network with 784 inputs and 10 outputs; and the recognition of facial images in the Yale face recognition database usually requires a multi-layer memristor neural network with 1024 inputs and 15 outputs. However, based on the similarity between the layers of a multi-layer neural network, this embodiment only describes the features of the present invention through a single-layer neural network.

[0028] Although the present invention is suitable for memristor neural network circuits of any depth, for the convenience of description, this embodiment uses Figure 1 The single-layer memristor neural network circuit shown is used as an example for explanation, namely, a variable pulse width input charge accumulation type memristor neural network circuit, including an adjustable clock source, a pulse width modulation circuit, a polarity control circuit, a programming control circuit, a memristor array, a charge accumulation circuit, an activation function circuit, and a comparison control and analog weight update circuit.

[0029] The ports of the memristor array include input signals XC1, XC2, ..., XC m , S 11 ,…,S mn , output signals YC1, YC2, ..., YC n Input signal XC1, XC2, ..., XC m is the output signal of the programming control circuit. Input signal S 11 ,…,S mn It is the output signal of the comparison control and analog weight update circuit. Output signal YC1,…,YC nis the input signal of the charge accumulation circuit. Taking the nth column circuit of the memristor array as an example, the output YC of the memristor array is n and the charge accumulation circuit T GFn The drain of amplifier A 1n The inverting input terminal and C n One end is connected.

[0030] The size of the memristor array can be selected according to the number of given input signals, the number of output signals and the number of layers. When the given input signals are m and the output signals are n, since this embodiment is a single-layer memristor neural network, the selected memristor array size is m×n. The selected memristor array can adopt a 1M1T structure, a 2M1R structure or a 4M bridge structure, but in this embodiment, the memristor array is as follows: Figure 2 The 1M1T structure shown includes a memristor M 11 ,…,M mn , NMOS tube TC 11 ,…,TC mn The input signals of the memristor array are XC1, XC2, …, XC m is the output of the programming control circuit; output signals YC1, YC2, ..., YC n It is the input of the charge accumulation circuit; NMOS tube TC 11 ,…,TC mn The gates are connected to the S of the comparison control and analog weight update circuits 11 ,…,S mn Taking the circuit of row m and column n as an example, the memristor M mn The positive electrode of the memristor array is connected to the input signal XC m , M mn The negative electrode is connected to the NMOS tube TC mn Drain of NMOS tube TC mn The gate connection S mn ,NMOS tube TC mn The source connection output signal YC n .

[0031] The input end of the pulse width modulation circuit is the signal input end of the variable pulse width input charge accumulation type memristor neural network circuit, and its input signals are m, which are X1, X2, ..., X m , the output of the pulse width modulation circuit is XA1, XA2, ..., XA m , output XA1, XA2, ..., XA m The pulse widths are respectively the same as the inputs X1, X2, ..., X m The output of the pulse width modulation circuit is connected to the input of the polarity control circuit.

[0032] The output CLK of the adjustable clock source is connected to the input end of the pulse width modulation circuit.

[0033] The polarity control circuit includes a PMOS tube T Jp1 ,T Jp2 ,…,T Jpm , NMOS tube T Jn1 ,T Jn2 ,…,T Jnm , resistor R J10 ,R J20 ,…,R Jm0 , R J11 ,R J21 ,…,R Jm1 , R J12 ,R J22 ,…,R Jm2 , and amplifier A 31 ,A 32 ,…,A 3m . Each pair of T Jpm , T Jnm Form a polarity control switch, each group of R Jm0 , R Jm1 , R Jm2 and amplifier A 3m It forms an inverse proportional operational amplifier. Taking the mth row circuit as an example, the output signal XA of the pulse width modulation circuit m Connect PMOS tube T Jpm The drain and NMOS tube T Jnm Drain; compare control and analog weight update circuit output signal Z m Connect to the PMOS tube T of the polarity control circuit Jpm The gate and NMOS tube T Jnm Gate of NMOS tube T Jnm The source connection resistor R Jm1 One end of the resistor R Jm1 The other end of the amplifier A 3m The inverting input terminal and resistor R Jm2 One end of the resistor R Jm2 The other end of the PMOS tube T Jpm The source and amplifier A 3m The output of amplifier A is connected 3m The non-inverting input terminal is connected to a resistor R Jm0 One end of the resistor R Jm0 The other end of the reverse proportional operational amplifier A 3m Output XB m This is the output of the polarity control circuit. The output of the polarity control circuit is XB1, XB2, ..., XB mIt is the input of programming control circuit.

[0034] The programming control circuit includes an NMOS tube T B1 ,T B2 ,…,T Bm , T P1 ,T P2 ,…,T Pm Among them, NMOS tube T B1 ,…,T Bm The gates are connected to the output signal S of the comparison control and analog weight update circuits IN1 ,…,S INm ; Output of polarity control circuit XB1, XB2,…, XB m It is the input of the programming control circuit. P1 ,…,T Pm The gates are connected to the output signal S of the comparison control and analog weight update circuits WP1 ,…,S WPm ; NMOS tube T P1 ,…,T Pm The drains of the comparison control and analog weight update circuits are connected to the output programming pulse signal terminals P1, ..., P m The output signals of the programming control circuit are XC1,…,XC m is the input signal of the memristor array. Taking the mth row circuit as an example, the output XB of the polarity control circuit m Connect the NMOS tube T of the programming control circuit Bm The drain of NMOS tube T Bm The source of NMOS tube T pm The source of the memristor array is connected to the input terminal XC m .

[0035] The charge accumulation circuit includes n s Each charge accumulation unit includes an NMOS tube T GFj , T GBj , amplifier A 1j , A 2j , and capacitor C j ; NMOS tube T GFj The drain of amplifier A 1j After being connected to the reverse input terminal of the NMOS tube T, the jth first input terminal of the charge accumulation circuit is formed; GFj The gate of forms the jth second input terminal of the charge accumulation circuit; the capacitor C j One end of the amplifier is connected to 1j The other end is connected to the inverting input of amplifier A 1j The output of amplifier A1j The output of amplifier A 2j The same-direction input terminal and NMOS tube T GBj The drain of NMOS tube T GBj The gate of the charge accumulation circuit forms the jth third input terminal; the amplifier A 2j After the reverse input terminal and output terminal of are connected, the jth output terminal of the charge accumulation circuit is formed; amplifier A 1j The same-direction input terminal, NMOS tube T GFj The source and NMOS tube T GBj The source of is grounded; where j = 1, 2, ..., n s , n s is the number of output terminals of the s-th layer neural network circuit.

[0036] The activation function circuit is composed of activation functions f1(x),…,f n (x) wherein the output signal Y of the charge accumulation circuit f1 ,Y f2 ,…,Y fn That is, its input signal, its output signal Y1,Y2,…,Y n Connected to the input of the comparison control and analog weight update circuit, the output signal Y1, Y2, ..., Y n It is also the output of the variable pulse width input charge accumulation memristor neural network circuit.

[0037] The comparison control and analog weight updating circuit includes input signals Y1, Y2, ..., Y n ,TAR1,TAR2,…,TAR n ; Output signal Z1, Z2, ..., Z m ,P1,P2,…,P m ,P1,S WP1 ,…,S WPm ,S 11 ,…,S mn ,S GF1 ,…,S GFn ,S IN1 ,…,S INm ,S GB1 ,…,S GBn The input signals are TAR1, TAR2, ..., TAR n is the input signal of the variable pulse width input charge accumulation type memristor neural network circuit; the input signal Y1, Y2, ..., Y n Connected to the output of the activation function circuit; output signals Z1, Z2, …, Z m Connect to the input of the polarity control circuit; output signal S WP1 ,…,S WPm,P1,P2,…,P m ,S IN1 ,…,S INm Connected to the input of the programming control circuit; output signal S GF1 ,…,S GFn ,S GB1 ,…,S GBn Connected to the input of the charge accumulation circuit; output signal S 11 ,…,S mn Connect to the input of the memristor array.

[0038] The internal structure of the comparison control and simulation weight update circuit depends on the algorithm selected for weight update, such as the error back propagation (BP) algorithm, which calculates the gradient of the loss function for all weights in the network, updates the weight value by the gradient descent method to minimize the loss function, and stimulates forward propagation and error back propagation to iterate repeatedly until the system's response to the input reaches a satisfactory predetermined target range. Another example is the random weight update (RWC) algorithm, which generates random weights based on errors. The advantage is that the hardware circuit is much simpler than the BP algorithm, but it requires more iterative calculations and is slower. Of course, the BP algorithm and the RWC algorithm can also be integrated to realize the comparison control and simulation weight update circuit. The comparison control and simulation weight update circuit itself and the algorithm applied to it are not the focus of the present invention.

[0039] The working principle of the present invention is:

[0040] For the convenience of description and in accordance with the practice of neural network circuit research, the input, output and weight are briefly marked with vector matrix symbols. m X, the signal after the pulse width modulation circuit is XA1, XA2, ..., XA m XA, the signal after passing through the polarity control circuit is XB1, XB2, ..., XB m XB, through the programming control circuit signal XC1, XC2, ..., XC m is XC, the weight M of the memristor array 11 ,M 12 ,…,M 1n ,M 12 ,…,M mn is the weight matrix M, the NMOS tube TC of the memristor array 11 ,TC 12 ,…,TC 1n ,TC 12 ,…,TC mn The control signal S 11 ,S 12 ,…,S 1n ,S12 ,…,S mn is S. The output signal Y of the charge accumulation circuit f1 ,Y f2 ,…,Y fn Y f , the output of the activation function circuit is Y1,Y2,…,Y n is Y, target values ​​TAR1, TAR2, …TAR n is TAR. The NMOS gate signal S of the programming control circuit IN1 ,S IN2 ,…,S INm For S IN The NMOS gate signal S of the programming control circuit WP1 ,S WP2 ,…,S WPm For S WP , the input signals P1, P2, ..., P of the programming control circuit m For P, NMOS tube T B1 ,T B2 ,…,T Bm T B . NMOS tube T P1 ,T P2 ,…,T Pm T P The input signals of the memristor array are XC1, XC2, …, XC m is XC, and the output signals of the memristor array are YC1, YC2, ..., YC n YC. NMOS tube T of the charge accumulation circuit GF1 ,T GF2 ,…,T GFn T GF , gate signal S GF1 ,S GF2 ,…,S GFn For S GF ,NMOS tube T GB1 ,T GB2 ,…,T GBn T GB , gate signal S GB1 ,S GB2 ,…,S GBn For S GB .

[0041] The input signal given by the present invention is a digital logic value X. The input of a traditional voltage-mode memristor neural network circuit is usually a normalized voltage signal with different voltage amplitudes. The input digital logic value X of the present invention is proportional to the normalized voltage signal size. The higher the voltage amplitude, the larger the corresponding digital logic value. The pulse width modulation circuit receives the clock CLK of the adjustable clock source, converts the input data X into a signal XA whose pulse width is proportional to the size of the data element of X, and the width of the reference pulse width can be controlled by the frequency of the adjustable clock source CLK. In actual neural network operations, the value of the weight matrix may be positive, zero or negative, but the memristor is a nonlinear passive device, and its resistance M can only be positive. Therefore, the traditional method often configures a memristor array and an auxiliary memristor array, and realizes negative weight values ​​through signal operations of the two arrays.

[0042] In order to effectively reduce the chip area, the present invention uses a polarity control circuit to achieve negative weight values ​​without the need for an auxiliary memristor array. The basic principle is to reverse the polarity of the corresponding input branch voltage that needs to achieve negative weights, thereby obtaining the effect of negative weights being equivalent. Taking the XA1 and XA2 branches as examples, if positive and negative weights need to be achieved respectively, the comparison control and analog weight update circuit sends a low-level instruction Z1 and a high-level instruction Z2. The low-level Z1 will turn off the NMOS tube T Jn1 , turn on PMOS tube T Jp1 The effect of high level Z2 is just the opposite, which will turn off the PMOS tube T Jp2 , turn on NMOS tube T Jn2 Therefore, XA1 passes T Jp1 The signal XB1 after the test is still the original polarity, and XA2 is connected through T Jn2 and the inverse proportional operational amplifier R J21 ,R J22 ,R J20 ,A 32 It becomes the signal XB2 with reverse polarity.

[0043] The programming control circuit realizes the switching control of signal processing and weight programming. IN is low level, programming control matrix S WP is high level, and S GF and S GB When it is high level, the accumulation capacitor array C1, C2, ..., C n Through NMOS tube T GF and T GB The discharge reset and programming pulse P pass through the transistor array T P is sent to the XC input terminal of the memristor array. ij The programming only needs to connect the NMOS TCij The gate S ij Set it to high level. When the signal processing control matrix S IN High level, programming control matrix S WP is low level, and resets NMOS tube T GF and T GB Through the low level S GF and low level S GB When the control is turned off, the signal XB passes through the transistor array T B The signal is sent to the XC end of the memristor array to realize the signal processing function.

[0044] The charge accumulation circuit performs charge accumulation calculation on the output signal YC of the memristor array and sends the calculation result to the activation function circuit. The working modes of the charge accumulation circuit are divided into accumulation capacitor reset mode and charge accumulation mode. In the accumulation capacitor reset mode, the transistor array T GF Through the high level S GF , transistor array T GB Through the high level S GB , the accumulation capacitor array C1, C2, ..., C n Discharge reset. It is worth noting that the weight programming of the memristor array and the reset of the accumulation capacitor can be performed simultaneously, which speeds up the operation of the hardware circuit. GB ,S GF When both are low, it enters the charge accumulation mode, and the output signal YC of the memristor array passes through A 11 ,A 12 ,…,A 1n and the accumulation capacitor array C1, C2, ..., C n and A 21 ,A 22 ,…,A 2n To perform charge accumulation calculation, each column signal passes through the charge accumulation capacitors C1,…,C n Perform the same direction or reverse integration. If all the components of XC are positive polarity signals, reverse integration is performed. Otherwise, the same direction integration or reverse integration may be performed, depending on the relative strength of the same polarity and reverse polarity signals of the column elements of the memristor array. Here, the product of current and time, that is, charge accumulation, is solved for the column output of the memristor. The calculation result Y f Send to the activation function circuit.

[0045] Figure 3 The kth column equivalent circuit diagram for the forward signal calculation of the variable pulse width input charge accumulation memristor neural network circuit, including the memristor M 1k ,…,M mk , amplifier A 1k , A 2k , capacitor C k, activation function f k (x), where the input signal XC m With memristor M mk The positive electrode of the memristor M mk The negative terminal of the amplifier A is connected 1k The inverting input terminal and capacitor C k One end of the capacitor C k The other end is connected to amplifier A 1k The output and amplifier A 2k The same-direction input terminal of amplifier A 1k The same-direction input terminal of amplifier A is grounded; 2k The inverting input of amplifier A is connected 2k The output of , and as the activation function f k (x) input signal; activation function f k The output signal of (x) is Y k .

[0046] according to Figure 3 From the forward signal calculation equivalent circuit diagram of the variable pulse width input charge accumulation memristor neural network circuit, we can know that the transfer function of the system is:

[0047]

[0048] It can be seen that XC in formula (1) j The polarity of determines the weight M jm The symbol of the column current in formula (1) is The sign of determines the direction of integration, and the sum of the column currents depends on the weight value M of the memristor and the input value XC.

[0049] The activation function circuit simulates the activation of neurons. The activation function f(x) can be a sigmoid function, a tanh function, a ReLU function, etc., depending on the application requirements of the neural network circuit.

[0050] The comparison control and analog weight update circuit completes the error calculation and weight change function. The output signal Y is sent to the comparison control and analog weight update circuit, and compared with the target value TAR to obtain the error signal, and the weight is adjusted according to the error signal. That is, it returns to the weight programming and accumulation capacitor reset stage, and then the calculation is repeated again until the minimum error requirement is met. Figure 4 The circuit operation flow chart reflects the above working process of the circuit. Figure 5 This reflects the waveform diagram of the key node of the variable pulse width input charge accumulation memristor neural network circuit at a certain moment. Here, the X2 branch needs to be reversed as an example. Since the size of each element of the input signal X depends on the actual application, it is assumed here that the input digital code DX2 >D X1 >D Xm .from Figure 5 It can be seen that, unlike the traditional voltage-mode memristor neural network, the input signal here is not converted into a signal with variable voltage amplitude, but the size of the input signal is represented by a digital logic value. The larger the input signal, the larger the logic value, and the wider the pulse width of the signal XA after passing through the pulse width modulation circuit. Obviously, due to D X2 The maximum value, therefore the corresponding XA2 pulse width is the widest, and the polarity control circuit changes the signal polarity according to the positive, zero and negative requirements of the weight value. Figure 5 In the case of X2 being negative, after passing through the charge accumulation circuit, the voltage is integrated in the same or opposite direction. f2 The integral direction of is opposite to that of other paths. It is worth mentioning that even if the kth column has Figure 5 The reverse polarity signal XB2,Y f2 The integral direction of is not necessarily opposite to other paths in all cases, because according to Figure 3 The sign of the sum of the column currents in formula (1) determines the direction of integration.

[0051] The present invention proposes a variable pulse width input charge accumulation type memristor neural network circuit, the so-called variable pulse width input is to input the input value of a multi-value or simulated continuous neural network in the form of a variable pulse width, so that it is easy to distinguish the polymorphic values ​​of the input, and no restriction is imposed on the value range of the input, which is conducive to improving the accuracy of the neural network. The input value range of the traditional voltage-mode memristor neural network circuit is limited by the threshold voltage of the memristor, and the more input values, the lower the accuracy of the neural network. The present invention transforms its input into a signal whose pulse width is proportional to the input size through the transformation of the pulse width modulation circuit and the polarity control circuit, and the polarity of the signal depends on the sign of the target weight. The width of the reference pulse width can be adjusted by the frequency of the adjustable clock source. Then, the column output of the memristor array is integrated in the same direction or in the opposite direction through the capacitor, and the charge accumulation value on the capacitor is the vector matrix multiplication result of the memristor array. This method can easily realize positive, zero or negative weights on a hybrid circuit composed of a memristor and CMOS. There is no need to usually require two memristor arrays as in the traditional voltage-mode memristor neural network circuit. This method significantly saves chip area. The weight programming and accumulation capacitance resetting of the memristor array can be performed simultaneously, thereby speeding up the operation of the hardware circuit.

[0052] It should be noted that, although the above embodiments of the present invention are illustrative, they are not intended to limit the present invention, and therefore the present invention is not limited to the above specific embodiments. Without departing from the principles of the present invention, any other embodiments obtained by those skilled in the art under the guidance of the present invention are deemed to be within the protection of the present invention.

[0053] For example, for a multi-layer variable pulse width input charge accumulation type memristor neural network circuit with more than 2 layers (including 2 layers), it is mainly composed of a comparison control and analog weight update circuit and S neural network circuits, where S is the number of layers of the neural network circuit.

[0054] (1) Neural network circuit

[0055] Each layer of neural network circuit is composed of an adjustable clock source, a pulse width modulation circuit, a polarity control circuit, a programming control circuit, a memristor array, a charge accumulation circuit and an activation function circuit. The input end of the first layer of neural network circuit (i.e., the input end of the pulse width modulation circuit) forms the total input end of the entire neural network circuit; the output end of the first layer of neural network circuit (i.e., the output end of the activation function circuit) is connected to the input end of the second layer of neural network circuit (i.e., the input end of the pulse width modulation circuit); the output end of the second layer of neural network circuit (i.e., the output end of the activation function circuit) is connected to the input end of the third layer of neural network circuit (i.e., the input end of the pulse width modulation circuit); ...; the output end of the last layer of neural network circuit (i.e., the output end of the activation function circuit) forms the total output end of the entire neural network circuit.

[0056] The scale of each layer of a multi-layer neural network (i.e., the number of inputs and outputs of each layer of memristor arrays) depends on the type and application field of the neural network itself. The number of inputs of the first layer of memristor arrays is the same as the number of inputs of the entire neural network, and the number of outputs of the last layer of memristor arrays is the same as the number of outputs of the entire neural network, but the number of outputs of the first layer of memristor arrays, the number of inputs and outputs of the middle layer of memristor arrays, and the number of inputs of the last layer of memristor arrays have certain mutual constraints, that is, the number of outputs of the previous layer of memristor arrays is equal to the number of inputs of the next layer of memristor arrays. Taking the memristor array module of a three-layer memristor neural network as an example, assuming that the number of inputs in the first layer is m1 and the number of outputs is n1, the number of inputs in the second layer is m2 and the number of outputs is n2, the number of inputs in the third layer is m3 and the number of outputs is n3. Since the output of the neurons in the previous layer is the input of the neurons in the next layer, m2=n1, m3=n2, the number of memristors in the first layer is m1×n1, the number of memristors in the second layer is n1×n2, and the number of memristors in the third layer is n2×n3.

[0057] Taking into account factors such as cost and circuit simplification, the adjustable clock sources of all neural network layers can be shared. Regardless of whether it is a digitally controlled pulse width modulation circuit or an analog pulse width modulation circuit, the width of the reference pulse width is controlled by the frequency of the adjustable clock source CLK. In addition, since the total input signal is a digital signal, that is, the input of the first layer is a digital signal, the pulse width modulation circuit selected for the first layer of the neural network circuit is a digitally controlled pulse width modulation circuit. Since the output signals of the neural network circuits from the first layer to the last layer are all analog signals, that is, the inputs of the neural network circuits from the second layer to the last layer are analog signals, the pulse width modulation circuits selected for the neural network circuits from the second layer to the last layer are analog pulse width modulation circuits.

[0058] (2) Comparison control and analog weight update circuit

[0059] Based on the error control principle, the comparison control and analog weight update circuits of all neural network layers must be shared. The target value signal given externally by the comparison control and analog weight update circuits, as well as the polarity control circuit, programming control circuit, memristor array, charge accumulation circuit and activation function circuit of each layer of the neural network circuit are connected. As the number of neural network layers increases, the control signals connected to each layer by the comparison control and analog weight update circuits will increase exponentially. Specifically:

[0060] The number of TAR signal input terminals of the comparison control and analog weight update circuit is the same as the number of output terminals of the last layer of the neural network circuit, which is connected to the given target value signal.

[0061] The number of Z signal output terminals of the comparison control and analog weight update circuit is It is connected to the polarity control circuit of each layer of the neural network circuit m s A second input terminal.

[0062] The number of P signal output terminals of the comparison control and analog weight update circuit is The m of the programming control circuit connected to each layer of the neural network circuit s A second input terminal, comparison control and analog weight update circuit S IN The number of signal output terminals is The m of the programming control circuit connected to each layer of the neural network circuit s A third input terminal, comparison control and analog weight update circuit S WP The number of signal output terminals is The m of the programming control circuit connected to each layer of the neural network circuit s A fourth input terminal.

[0063] The number of S signal output terminals of the comparison control and analog weight update circuit is The m of the memristor array connected to each layer of the neural network circuit s ×n s A control terminal.

[0064] Comparison control and analog weight update circuit S GF The number of signal output terminals is It is connected to the charge accumulation circuit of each layer of the neural network circuit. s A second input terminal, comparison control and analog weight update circuit S GB The number of signal output terminals is It is connected to the charge accumulation circuit of each layer of the neural network circuit. s A third input terminal.

[0065] The number of Y signal input terminals of the comparison control and analog weight update circuit is It connects the activation function circuits of each layer of neural network circuits n s output terminals.

[0066] Where m s is the number of input terminals of the s-th layer neural network circuit; n s is the number of output terminals of the s-th layer of the neural network circuit, s = 1, 2, …, S, where S is the number of layers of the neural network.

Claims

1. A variable pulse width input charge accumulation type memristor neural network circuit, comprising a comparison control and analog weight update circuit and at least one layer of neural network circuit, characterized in that: Each layer of the neural network circuit includes an adjustable clock source, a pulse width modulation circuit, a polarity control circuit, a programming control circuit, a memristor array, a charge accumulation circuit and an activation function circuit; the output end of the adjustable clock source is connected to the clock input end of the pulse width modulation circuit; the m of the pulse width modulation circuit s The input terminals form the m circuit of this layer of neural network. s Input terminals, pulse width modulation circuit m s The output terminal is connected to the polarity control circuit m s The first input terminal; the polarity control circuit m s The output terminal is connected to the programming control circuit m s A first input terminal; m of the programming control circuit s The output terminals are connected to the m s input terminals; n of the memristor array s The output terminals are connected to the charge accumulation circuit n s The first input terminal of the charge accumulation circuit is n s The output terminals are connected to the activation function circuit n s Input terminals, activation function circuit n s The output terminals form the nth layer of the neural network circuit. s Output terminals; The polarity control circuit consists of s Each polarity control unit includes a PMOS tube T Jpi , NMOS tube T Jni , resistor R Ji0 -R Ji2 , and amplifier A 3i ; PMOS tube T Jpi The drain and NMOS tube T Jni After being connected to the drain of the PMOS tube T, the i-th first input terminal of the polarity control circuit is formed; Jpi The gate and NMOS tube T Jni After the gate of the NMOS tube T is connected, the i-th second input terminal of the polarity control circuit is formed; Jni The source is connected via resistor R Ji1 Connect Amplifier A 3i The reverse input terminal of PMOS tube T Jpi The source is divided into two paths, one through the resistor R Ji2 Connect Amplifier A 3i The other is connected to the inverting input of amplifier A 3i The output terminal; the resistor R Ji0 One end of the amplifier is connected to 3i The same-direction input terminal of the amplifier is grounded. 3i The output end of forms the i-th output end of the polarity control circuit; The input end of the pulse width modulation circuit of the first layer of neural network circuit forms the total input end of the entire variable pulse width input charge accumulation type memristor neural network circuit; the output end of the activation function circuit of the previous layer of neural network circuit is connected to the input end of the pulse width modulation circuit of the next layer of neural network circuit; the output end of the activation function circuit of the last layer of neural network circuit forms the total output end of the entire variable pulse width input charge accumulation type memristor neural network circuit; The pulse width of the output signal of the pulse width modulation circuit is proportional to the amplitude of the input signal, and the reference pulse width is controlled by the frequency of the adjustable clock source; The number of TAR signal input terminals of the comparison control and analog weight update circuit is the same as the number of output terminals of the last layer of the neural network circuit, which is connected to the given target value signal; the number of Z signal output terminals of the comparison control and analog weight update circuit is , which are connected to the polarity control circuits of each layer of the neural network circuit s The number of P signal output terminals of the comparison control and analog weight update circuit is , which are connected to the programming control circuit of each layer of neural network circuit s A second input terminal, comparison control and analog weight update circuit S IN The number of signal output terminals is , which are connected to the programming control circuit of each layer of neural network circuit s A third input terminal, comparison control and analog weight update circuit S WP The number of signal output terminals is , which are connected to the programming control circuit of each layer of neural network circuit s The number of S signal output terminals of the comparison control and analog weight update circuit is , which are connected to the m of the memristor array of each layer of the neural network circuit s ×n s A control terminal; S of the comparison control and analog weight update circuit GF The number of signal output terminals is , which are connected to the charge accumulation circuits of each layer of the neural network circuit s A second input terminal, comparison control and analog weight update circuit S GB The number of signal output terminals is , which are connected to the charge accumulation circuits of each layer of the neural network circuit s The number of the Y signal input terminals of the comparison control and analog weight update circuit is , which respectively connects the activation function circuits of each layer of the neural network circuit n s Output terminals; where i=1,2,…,m s , m s is the number of input terminals of the s-th layer neural network circuit; n s is the number of output terminals of the s-th layer neural network circuit, s=1,2,…,S, S is the number of layers of the neural network.

2. The variable pulse width input charge accumulation type memristor neural network circuit according to claim 1 is characterized in that: Each layer of neural network circuits shares the same adjustable clock source.

3. The variable pulse width input charge accumulation type memristor neural network circuit according to claim 1, characterized in that: The pulse width modulation circuit of the first layer of the neural network circuit is a digitally controlled pulse width modulation circuit, and the pulse width modulation circuits from the second layer to the last layer are analog pulse width modulation circuits.

4. The variable pulse width input charge accumulation type memristor neural network circuit according to claim 1, characterized in that: The programming control circuit is composed of s The circuit programming control unit is composed of: Each programming control unit includes an NMOS transistor T Bi , T Pi ; NMOS tube T Bi The drain of the NMOS tube T forms the i-th first input terminal of the programming control circuit; Pi The drain of the NMOS tube T forms the i-th second input terminal of the programming control circuit; Bi The source and NMOS tube T Pi After being connected to the source of the NMOS tube T, the i-th output terminal of the programming control circuit is formed; Bi The gate of the NMOS tube T forms the i-th third input terminal of the programming control circuit; Pi The gate of forms the i-th fourth input terminal of the programming control circuit; where i=1,2,…,m s , m s is the number of input terminals of the s-th layer neural network circuit.

5. The variable pulse width input charge accumulation type memristor neural network circuit according to claim 1, characterized in that: The charge accumulation circuit includes n s Road charge accumulation unit; Each charge accumulation unit includes an NMOS tube T GFj , T GBj , amplifier A 1j , A 2j , and capacitor C j ; NMOS tube T GFj The drain of amplifier A 1j After being connected to the reverse input terminal of the NMOS tube T, the jth first input terminal of the charge accumulation circuit is formed; GFj The gate of forms the jth second input terminal of the charge accumulation circuit; the capacitor C j One end of the amplifier is connected to 1j The other end is connected to the inverting input of amplifier A 1j The output of amplifier A 1j The output of amplifier A 2j The same-direction input terminal and NMOS tube T GBj The drain of NMOS tube T GBj The gate of the charge accumulation circuit forms the jth third input terminal; the amplifier A 2j After the reverse input terminal and output terminal of are connected, the jth output terminal of the charge accumulation circuit is formed; amplifier A 1j The same-direction input terminal, NMOS tube T GFj The source and NMOS tube T GBj The source is grounded; where j=1,2,…,n s , n s is the number of output terminals of the s-th layer neural network circuit.

6. The variable pulse width input charge accumulation type memristor neural network circuit according to claim 1, characterized in that: The activation function circuit consists of n s The path activation function unit is composed of The input end of each activation function unit forms the jth input end of the activation function circuit, and the output end of each activation function unit forms the jth output end of the activation function circuit; where j=1,2,…,n s , n s is the number of output terminals of the s-th layer neural network circuit.

Images