A memristive neural network circuit with partially reinforced operant conditioning

Abstract

The present invention discloses a memristive neural network circuit with partially reinforced operant conditioning, comprising a pressure module, a satiety module, a reward module, and a synaptic module; the synaptic module and the reward module are connected, and the reward module includes the pressure module and the satiety module. The technical solution of the present invention enables the circuit to control the rate of reinforcement and inhibition of behavior in operant conditioning with rewards of different frequencies, and also achieves a certain degree of adaptability to multiple reward stimuli during the process of behavioral reinforcement and inhibition.

Description

Technical Field

[0001] The present invention belongs to the technical field of circuit design, and in particular relates to a memristive neural network circuit with partially enhanced operant conditioning. Background Art

[0002] With the rapid development of artificial neural network technology in recent years, research on neuromorphic circuits has become an important area for simulating the neuronal and synaptic structures of the human brain. Traditional artificial neural network circuits built using CMOS technology suffer from numerous challenges, including low integration and high energy consumption. Memristors, as a new type of electronic component, have become a key component in current neural network research due to their significant advantages in memory, low power consumption, and good integration.

[0003] Memristors possess properties highly similar to biological synapses, such as non-volatility, nonlinearity, low power consumption, and excellent scalability. This gives them great potential for simulating biological synapses. Consequently, memristors are widely used in a variety of research fields, including memristive neural networks and associative memory circuits. With the continuous development of memristor technology, its application prospects in neural networks are becoming increasingly broad, making it a key research direction driving neuromorphic computing and brain-inspired computing.

[0004] Associative memory is the process of memorizing information by associating it with other things based on their connections. It can be categorized into two types: classical conditioning and operant conditioning. Operant conditioning, proposed by psychologist B.F. Skinner, explains the learning process of behavior. It emphasizes influencing the frequency of behavior through reinforcement or punishment of consequences. Currently, operant conditioning is a hot topic in artificial neural network research.

[0005] Many memristor-based operant conditioning circuits have been proposed. Publication No. CN202410956651 only considers simple operant conditioning, without incorporating partial reinforcement features or considering the varying effects of different interval lengths on lever-pressing maintenance. Current operant conditioning circuits typically influence behavior maintenance by varying reward intensity or reward type, but lack consideration of the impact of reward intermittentity on behavior maintenance. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a memristive neural network circuit with partially enhanced operant conditioning.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A memristive neural network circuit with partially reinforced operant conditioning includes: a pressing module, a satiety module, a reward module, and a synaptic module; the synaptic module and the reward module are connected, and the reward module includes: a pressing module and a satiety module; wherein,

[0009] The pressing module is composed of a first capacitor C1, a second capacitor C2, a third capacitor C3, a first comparator OP1, a second comparator OP2, a third comparator OP3, a first 555 timer TIMER1, a thirteenth resistor R13, and a fourteenth resistor R14; wherein the analog pressing signal N1 is connected to the inverting input terminal of the first comparator OP1, the output terminal of the first comparator OP1 is connected to the input terminal of the third capacitor C3, the output terminal of the third capacitor C3 is connected to the inverting input port of the second comparator and the input terminal of the thirteenth resistor R13, the output terminal of the thirteenth resistor R13 is connected to the input terminals of the first capacitor C1, the second capacitor C2, and the first port 1 of the first 555 timer TIMER1, the output terminal of the second comparator is connected to the second port 2 of the first 555 timer TIMER1, and the third port 3 of the first 555 timer TIMER1 is connected to the positive input terminal of the third comparator;

[0010] The satiety module is composed of a third NOT gate NOT3, a first NMOS transistor T1, a second adder SUM2, a first memristor M1, a fifth operational amplifier OP5, a first memristor R1, a first digital operation unit ABM1, and a first voltage-controlled switch S1; wherein, the output end of the third NOT gate NOT3 is connected to the G pole of the first NMOS transistor T1, the D pole of the first NMOS transistor T1 is connected to the input end of the second adder SUM2, the output end of the second adder SUM2 is connected to the input end of the first memristor M1 and the 2 input end of the first digital operation unit ABM1, the inverting section of the first memristor M1 is connected to the inverting input end of the fifth operational amplifier OP5, the output end of the fifth operational amplifier OP5 is connected to the 1 input end of the first digital operation unit ABM1, and the output end of the first digital operation unit ABM1 is connected to the input end of the first voltage-controlled switch;

[0011] The reward module is composed of a second AND gate AND2, a third AND gate AND3, a second resistor R2, a third resistor R3, a fourth resistor R4, a fifth resistor R5, a sixth resistor R6, a seventh resistor R7, and a fourth adder OP4; wherein the output end of the second AND gate AND2 is connected to the output end of the second resistor R2, the output end of the third AND gate AND3 is connected to the input end of the third resistor R3, and the output end of the second resistor R2 is connected to the positive input end of the fourth adder OP4;

[0012] The synaptic module is composed of a first NOT gate NOT1, a second NOT gate NOT2, a first AND gate AND1, a seventh resistor R7, an eighth resistor R8, a ninth resistor R9, a tenth resistor R10, an eleventh resistor R11, a twelfth resistor R12, a sixth adder OP6, a seventh operational amplifier OP7, a second memristor M2, and a second digital operation unit ABM2; wherein, the input end of the first NOT gate is connected to the signal N2 that a reward appears when pressing twice, the input end of the second NOT gate is connected to the signal N3 that a reward appears when pressing four times, and the output ends of the first NOT gate NOT1 and the second NOT gate NOT2 are connected to the signal N1 that a reward appears when pressing four times, respectively. The two input terminals of the first AND gate AND1 and the output terminal of the first AND gate AND1 are connected to the input terminal of the seventh resistor R7, the output terminal of the seventh resistor R7 is connected to the positive input terminal of the sixth adder OP6, the output terminal of the sixth adder OP6 is connected to the input terminal of the memristor M2, the output terminal of the memristor M2 is connected to the negative input terminal of the seventh operational amplifier OP7, the output terminal of the seventh operational amplifier OP7 is connected to the 1 input terminal of the second digital operation unit ABM2, the output terminal of the sixth adder OP6 is connected to the 2 input terminal of the second digital operation unit ABM2, and the output terminal of the second digital operation unit is connected to the maintenance rate signal N4.

[0013] Preferably, in the pressing module, when the pressing signal N1 is at a high level and N1>1V, the output of OP1 changes from a high level to a low level. At this time, C3 and R13 generate a falling edge voltage, and OP2 generates a rising edge voltage. The monostable trigger is triggered by the falling edge, and the 3-port of the 555 timer outputs a low level. After passing through the comparator OP3, the signal VOP3 output by the comparator OP3 outputs a low level. When N1 changes from a high level to a low level, it is a falling edge level. OP1 outputs a high level and OP2 outputs a low level. At this time, the monostable trigger PIN3 pin outputs a high level, and the duration of the high level is determined by the size of R14 and C1.

[0014] Preferably, in the satiety module, whenever the reward signal N2 appears after pressing twice or the reward signal N3 appears after pressing four times, the voltage-controlled switch S1 is turned on, and the voltage acts on the memristor M1 and ABM1 after passing through SUM1 and SUM2. The voltage signal VOP5 output by the operational amplifier OP5 acts on the digital operation circuit ABM1, and the output of ABM1 represents the resistance value of M1; when N2 and N3 are both low, there is no food reward at this time, NOT3 is turned on, and outputs a high-level voltage to act on the gate of NMOS. At this time, the MOS tube is turned on and outputs a voltage. The voltage causes the memristor value of M1 to continuously increase and gradually return to the initial value, VABM1 continues to increase, and the satiety gradually returns to the initial state.

[0015] Preferably, in the reward module, the reward signal N2 represents that a food reward is generated every time the lever is pressed twice, and a high-level pulse is output; the reward signal N3 represents that a food reward is generated every time the lever is pressed four times, and a high-level pulse is output; when the output signal VOP3 of the comparator OP3 and the reward signal N2 are at a high level, AND2 is turned on and outputs a 10V voltage, and outputs the reward voltage through the adder; when the output signal VOP3 of the pressing module and the reward signal N3 are at a high level, AND3 outputs a 4V voltage, which also passes through the adder OP4 and is output as the reward voltage.

[0016] Preferably, in the synaptic module, when the voltage signal VOP4 output by the adder OP4 is input to OP6, the output of OP6 causes the memristor value of M2 to decrease, and the output voltage value after passing through OP7 and ABM2 continues to decrease, which indicates that the mouse's pressing frequency continues to increase.

[0017] The memristive neural network circuit of this invention leverages the psychological phenomenon of "partial reinforcement" to control the rate of behavioral reinforcement and inhibition in response to rewards of varying frequencies in operant conditioning. Furthermore, the circuit incorporates a satiety circuit, enabling it to adapt to multiple reward stimuli during both behavioral reinforcement and inhibition. This circuit is closer to practical application, more consistent with biological characteristics, and adaptable to more complex scenarios. It provides further insights for the further development of neural networks for operant conditioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0019] Figure 1 The graph of memristor value change under different voltages;

[0020] Figure 2 Schematic diagram of the structure of a memristive neural network circuit with partially enhanced operant conditioning according to the present invention;

[0021] Figure 3 This is a diagram of the simulation process of operant conditioning in which the user is punished with electrical stimulation every time they press the button twice;

[0022] Figure 4 A diagram showing the simulation process of operant conditioning in which the user is punished with electrical stimulation every four presses. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0024] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Example 1:

[0026] like Figure 1 As shown, Figure 1 The graph below shows the memristor M1 and M2 at different voltages. As can be seen from the graph, the greater the voltage applied to the memristor, the faster the memristor's resistance decreases. Applying a reverse voltage will increase the memristor's resistance.

[0027] like Figure 2 As shown, an embodiment of the present invention provides a memristive neural network circuit with partially reinforced operant conditioning, comprising: a pressing module, a satiety module, a reward module, and a synaptic module; the synaptic module and the reward module are connected, and the reward module comprises: a pressing module and a satiety module.

[0028] As an implementation manner of an embodiment of the present invention, the pressing module is composed of a first capacitor C1, a second capacitor C2, a third capacitor C3, a first comparator OP1, a second comparator OP2, a third comparator OP3, a first 555 timer TIMER1, a thirteenth resistor R13, and a fourteenth resistor R14; wherein the analog pressing signal N1 is connected to the inverting input terminal of the first comparator OP1, the output terminal of the first comparator OP1 is connected to the input terminal of the third capacitor C3, the output terminal of the third capacitor C3 is connected to the inverting input port of the second comparator and the input terminal of the thirteenth resistor R13, the output terminal of the thirteenth resistor R13 is connected to the input terminals of the first capacitor C1, the second capacitor C2 and the first port 1 of the first 555 timer TIMER1, the output terminal of the second comparator is connected to the first port 2 of the first 555 timer TIMER1, and the first port 3 of the first 555 timer TIMER1 is connected to the positive input terminal of the third comparator;

[0029] The satiety module is composed of a third NOT gate NOT3, a first NMOS transistor T1, a second adder SUM2, a first memristor M1, a fifth operational amplifier OP5, a first memristor R1, a first digital operation unit ABM1, and a first voltage-controlled switch S1; wherein, the output end of the third NOT gate NOT3 is connected to the G pole of the first NMOS transistor T1, the D pole of the first NMOS transistor T1 is connected to the input end of the second adder SUM2, the output end of the second adder SUM2 is connected to the input end of the first memristor M1 and the 2 input end of the first digital operation unit ABM1, the inverting section of the first memristor M1 is connected to the inverting input end of the fifth operational amplifier OP5, the output end of the fifth operational amplifier OP5 is connected to the 1 input end of the first digital operation unit ABM1, and the output end of the first digital operation unit ABM1 is connected to the input end of the first voltage-controlled switch;

[0030] The reward module is composed of a second AND gate AND2, a third AND gate AND3, a second resistor R2, a third resistor R3, a fourth resistor R4, a fifth resistor R5, a sixth resistor R6, a seventh resistor R7, and a fourth adder OP4; wherein the output end of the second AND gate AND2 is connected to the output end of the second resistor R2, the output end of the third AND gate AND3 is connected to the input end of the third resistor R3, and the output end of the second resistor R2 is connected to the positive input end of the fourth adder OP4;

[0031] The synaptic module is composed of a first NOT gate NOT1, a second NOT gate NOT2, a first AND gate AND1, a seventh resistor R7, an eighth resistor R8, a ninth resistor R9, a tenth resistor R10, an eleventh resistor R11, a twelfth resistor R12, a sixth adder OP6, a seventh operational amplifier OP7, a second memristor M2, and a second digital operation unit ABM2; wherein, the input end of the first NOT gate is connected to the signal N2 that a reward appears when pressing twice, the input end of the second NOT gate is connected to the signal N3 that a reward appears when pressing four times, and the output ends of the first NOT gate NOT1 and the second NOT gate NOT2 are connected to the signal N1 that a reward appears when pressing four times, respectively. The two input terminals of the first AND gate AND1 and the output terminal of the first AND gate AND1 are connected to the input terminal of the seventh resistor R7, the output terminal of the seventh resistor R7 is connected to the positive input terminal of the sixth adder OP6, the output terminal of the sixth adder OP6 is connected to the input terminal of the memristor M2, the output terminal of the memristor M2 is connected to the negative input terminal of the seventh operational amplifier OP7, the output terminal of the seventh operational amplifier OP7 is connected to the 1 input terminal of the second digital operation unit ABM2, the output terminal of the sixth adder OP6 is connected to the 2 input terminal of the second digital operation unit ABM2, and the output terminal of the second digital operation unit is connected to the maintenance rate signal N4.

[0032] As an implementation method of an embodiment of the present invention, in the pressing module, when the pressing signal N1 is at a high level, since N1>Vth1=1V, the output of OP1 changes from a high level to a low level. At this time, C3 and R13 generate a falling edge voltage, and at the same time, OP2 generates a rising edge voltage. Since the monostable trigger is triggered by a falling edge, the 3-port of the 555 timer outputs a low level. After passing through the comparator OP3, the signal VOP3 output by the comparator OP3 outputs a low level. When N1 changes from a high level to a low level, it is a falling edge level. OP1 outputs a high level and OP2 outputs a low level. At this time, the PIN3 pin of the monostable trigger outputs a high level, and the duration of the high level is determined by the size of R14 and C1.

[0033] In the satiety module, whenever signal N2, indicating a reward for pressing twice, or signal N3, indicating a reward for pressing four times, appears, voltage-controlled switch S1 turns on. This voltage, after passing through SUM1 and SUM2, acts on memristor M1 and ABM1. Operational amplifier OP5 outputs voltage signal VOP5, which is applied to digital arithmetic circuit ABM1. ABM1's output represents M1's resistance. For example, when M1 is 1 kΩ, the output voltage is 1 V. As the SUM2 output signal continues to act on memristor M1, M1's memristance decreases, and VABM1 also decreases, indicating that the food reward's stimulation for the mouse is decreasing. The voltage output by ABM1 serves as the reward voltage. When both N2 and N3 are low, indicating no food reward, NOT3 turns on and outputs a high voltage that acts on the gate of the NMOS transistor. This turns on the MOS transistor and outputs a voltage. This voltage causes M1's memristance to increase and gradually return to its initial value. VABM1 also increases, and satiety gradually returns to its initial state.

[0034] In the reward module, the reward signal N2 represents that a food reward is generated every time the lever is pressed twice, and a high-level pulse is output. The reward signal N3 represents that a food reward is generated every time the lever is pressed four times, and a high-level pulse is output. When the signal VOP3 output by the comparator OP3 and the reward signal N2 are at a high level, AND2 is turned on and outputs a 10V voltage, and outputs the reward voltage through the adder. When the signal VOP3 output by the comparator OP3 and the reward signal N3 are at a high level, AND3 outputs a 4V voltage, which also passes through the adder OP4 and is output as the reward voltage. The voltages of different amplitudes output by AND2 and AND3 respectively indicate that different reward frequencies have different stimulation levels on the mouse, and also reflect the psychological phenomena of some behaviors.

[0035] In the synaptic module, when the voltage signal VOP4 output by adder OP4 is input to OP6, the output of OP6 causes the memristor value of M2 to decrease. After passing through OP7 and ABM2, the output voltage value also continues to decrease, indicating that the mouse's pressing frequency is continuously increasing.

[0036] Furthermore, the present embodiment also considers adaptive regulation, such as the impact of satiety on food reward stimulation. During food reward delivery, the intensity of positive feedback is also affected by satiety. When reward signals N2 and N3 generate a positive pulse, voltage-controlled switch S1 turns on, acting on memristor M1. The resistance of memristor M1 decreases, and the output of VABM1 also decreases, achieving a suppressive effect.

[0037] By simulating the reward signal N2 and the reward signal N3, that is, the reward signal N2 or the reward signal N3 is high, AND2 or AND3 is high, and different voltages are output to act on OP4 to achieve the enhancement function.

[0038] By controlling the changes in the resistance values ​​of memristors M1 and M2, the output of the entire circuit can be controlled, thereby achieving adaptive regulation of the circuit itself, such as the impact of satiety on food reward stimulation, and realizing the psychological effect of partial reinforcement.

[0039] The present invention will conduct case analysis on two scenarios: an operant conditioning simulation in which electrical stimulation is received after every two presses and an operant conditioning simulation in which electrical stimulation is received after every four presses.

[0040] like Figure 3 As shown in FIG, it is a simulation process diagram of operant conditioning reflex with electrical stimulation every time pressing twice. The specific process is as follows:

[0041] The behavioral reinforcement phase of the lever-pressing phase, which involves receiving electrical stimulation twice, lasts from 0 to 48 seconds. When the lever is pressed, N1 generates a 5V high-level pulse. Simultaneously, OP3 is activated and generates a high-level voltage. This voltage indicates that the mouse's neurons are active after pressing the lever. Giving a reward signal at this time has a certain impact on the mouse's behavior. N2 is reward signal 1, which generates a food reward every time the lever is pressed twice. When both N2 and OP3 are high, OP6 generates a reward voltage, which is output to the synaptic module, reducing the memristor value of M2. Since the output voltage of ABM2 is proportional to M2, the output voltage VOUT of ABM2 represents the maintenance rate of the mouse's lever pressing. A lower output voltage indicates a higher maintenance rate of the mouse's lever pressing behavior. During this simulation, VOUT eventually decreases to approximately 3.0V. At the same time, after receiving the food reward, the mouse will feel a certain degree of fullness. Therefore, the memristance of M1 will continue to decrease after receiving the food reward, and the satiety voltage will also decrease. Therefore, after receiving the food reward, the output of OP6 will also decrease due to the influence of satiety. When the mouse does not receive the food reward, the memristance of M1 will continue to rise, and the mouse's satiety will gradually return.

[0042] The behavioral inhibition stage after pressing the lever twice and receiving electrical stimulation: 48-70s is the behavioral inhibition stage of the mouse. In this stage, when the mouse presses the lever, no reward will be generated, which belongs to the natural inhibition stage. When N1 is activated, OP3 generates a high-level voltage, and the neuron is in an active state. Since the reward signal N2 is in an inactive state, the mouse's behavior is also inhibited. At this time, the penalty voltage output by OP6 is -2V. This voltage output value synaptic module causes the resistance of the memristor M2 to continue to increase, and VOUT also continues to increase. The frequency of the mouse pressing the lever continues to decrease. At the same time, because the mouse has not received food rewards, the mouse's sense of fullness decreases, and its sensitivity to food rewards will continue to increase, and gradually return to its initial state.

[0043] like Figure 4 As shown in FIG, it is a simulation process diagram of operant conditioning with electrical stimulation punishment every four presses. The specific process is:

[0044] Behavioral reinforcement stage of pressing the lever four times and receiving electrical stimulation: 0-48s is the process of positive reinforcement of the mouse's lever pressing behavior. In this process, the mouse receives a food reward every time it presses the lever four times. When N1 is activated, OP3 generates a high level, and the neuron is in an active state. After N1 generates four pulses, N3 is activated and generates a high-level pulse, indicating that a food reward is generated. At the same time, OP6 generates a reward voltage and outputs it to the synaptic module, acting on the memristor M2, causing the memristor value of M2 to continuously decrease. Since the output voltage VOUT of ABM2 is proportional to the memristor value of M2, the output voltage VOUT is also continuously decreasing, indicating that the maintenance rate of the mouse's lever pressing behavior is continuously increasing. At 48s, VOUT is reduced to about 2.5V. Due to the psychological effect of partial reinforcement Intermittent rewards often lead mice to exhibit more intense and sustained behaviors. Therefore, pressing the lever four times for a food reward produces a greater reinforcement effect than pressing the lever twice. Consequently, OP6's output voltage is smaller. This smaller reward voltage causes M2 to drop more rapidly, and the behavior maintenance rate VOUT also drops more rapidly. At 48 seconds, VOUT reaches 2.5V, indicating that the mouse presses the lever more frequently, which corresponds to the psychological phenomenon of partial reinforcement. When the mouse receives a food reward, the satiety module is also activated, causing M1's memristance to decrease and the satiety voltage to drop, indicating that the mouse's response to the food reward is also decreasing.

[0045] Behavioral inhibition stage after pressing the lever four times and receiving electrical stimulation: 48-70s is the simulation process of behavioral inhibition. When the mouse presses the lever, N1 is activated and the mouse neurons are in an active state. At this time, OP3 also generates a high-level voltage. However, since N3 does not generate a reward signal, the mouse performs natural behavioral inhibition at this time, causing the memristor value of M2 to continue to increase, and the output voltage of VOUT also continues to increase. At this time, the behavioral frequency continues to decrease, achieving the effect of behavioral inhibition.

[0046] The present invention sets up experimental environments where mice receive food after pressing a lever twice and four times. This invention introduces satiety regulation, making the circuit more biologically consistent and adaptable to different environments. This invention not only simulates adaptive regulation but also implements partial reinforcement, effectively simulating the varying effects of different interval lengths on the retention rate of lever presses. Using two and four intervals as examples, this invention demonstrates a simple structure and comprehensive functionality, and is of great significance to the research of memristive bionic circuits.

[0047] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A memristive neural network circuit with partially reinforced operant conditioning, characterized in that: include: Pressing module, satiety module, reward module and synaptic module; The synapse module is connected to the reward module, which includes a pressing module and a satiety module; In the pressing module, the analog pressing signal N1 is connected to the inverting input terminal of the first comparator OP1, the output terminal of the first comparator OP1 is connected to the input terminal of the third capacitor C3, the output terminal of the third capacitor C3 is connected to the inverting input port of the second comparator and the input terminal of the thirteenth resistor R13, the output terminal of the thirteenth resistor R13 is connected to the input terminals of the first capacitor C1, the second capacitor C2, and the port 1 of the first 555 timer TIMER1, the output terminal of the first capacitor C1 is connected to the input terminal of the fourteenth resistor R14; the output terminal of the second capacitor C2 is connected to the port 5 of the first 555 timer TIMER1; the output terminal of the second comparator is connected to the port 2 of the first 555 timer TIMER1, and the port 3 of the first 555 timer TIMER1 is connected to the positive input terminal of the third comparator; In the satiety module, the output terminal of the third NOT gate NOT3 is connected to the G terminal of the first NMOS transistor T1, the D terminal of the first NMOS transistor T1 is connected to the input terminal of the second adder SUM2, the output terminal of the second adder SUM2 is connected to the input terminal of the first memristor M1 and the 2 input terminal of the first digital operation unit ABM1, the reverse terminal of the first memristor M1 is connected to the reverse input terminal of the fifth operational amplifier OP5, the output terminal of the fifth operational amplifier OP5 is connected to the 1 input terminal of the first digital operation unit ABM1, and the output terminal of the first digital operation unit ABM1 is connected to the input terminal of the first voltage-controlled switch; the voltage signal N2 or N3 passes through SUM1, first, the output voltage of SUM1 will be applied to the positive input terminal of the first voltage-controlled switch S1; when the output voltage of SUM1 is greater than the threshold of S1, the connection port of the first voltage-controlled switch S1 is turned on; the output voltage of SUM1 acts on SUM2, and the output voltage of SUM2 is applied to the first memristor M1; the output voltage VM1 of the first memristor M1 is applied to the fifth operational amplifier OP5; the output voltage VOP5 of the fifth operational amplifier OP5 is applied to the first mathematical operation circuit ABM1; the output voltage VABM1 of ABM1 is output to the input terminal of R4 in the reward module through the connection port of the first voltage-controlled switch S1; the output port of R4 is connected to the positive input terminal of the fourth adder OP4; In the reward module, the output terminal of the second AND gate AND2 is connected to the input terminal of the second resistor R2, the output terminal of the third AND gate AND3 is connected to the input terminal of the third resistor R3, and the output terminal of the second resistor R2 is connected to the positive input terminal of the fourth adder OP4; the second AND gate AND2 has two input ports respectively connected to: the voltage signal N2 and the comparator OP3; the third AND gate AND3 has two input ports respectively connected to: the voltage signal N3 and the comparator OP3; In the synaptic module, the input end of the first NOT gate NOT1 is connected to the signal N2 indicating a reward appears when pressing twice, the input end of the second NOT gate NOT2 is connected to the signal N3 indicating a reward appears when pressing four times, the output ends of the first NOT gate NOT1 and the second NOT gate NOT2 are respectively connected to the two input ends of the first AND gate AND1, the output end of the first AND gate AND1 is connected to the input end of the seventh resistor R7, the output end of the seventh resistor R7 is connected to the positive input end of the sixth adder OP6, the output end of the sixth adder OP6 is connected to the input end of the memristor M2, the output end of the memristor M2 is connected to the negative input end of the seventh operational amplifier OP7, the output end of the seventh operational amplifier OP7 is connected to the 1 input end of the second digital operation unit ABM2, the output end of the sixth adder OP6 is connected to the 2 input end of the second digital operation unit ABM2, and the output end of the second digital operation unit is connected to the maintenance rate signal N4; the input end of R8 is connected to the output end of the fourth adder OP4, and the output end of R8 is connected to the positive input end of the sixth adder OP6.

2. The memristive neural network circuit with partially enhanced operant conditioning according to claim 1, wherein: In the press module, when the press signal N1 is high and N1>1V, the output of OP1 changes from high to low. At this time, C3 and R13 generate a falling edge voltage, and OP2 generates a rising edge voltage. The monostable trigger is triggered by the falling edge. Port 3 of the 555 timer outputs a low level. After passing through the comparator OP3, the signal VOP3 output by the comparator OP3 outputs a low level. When N1 changes from high to low, it is a falling edge level. OP1 outputs a high level and OP2 outputs a low level. At this time, the monostable trigger PIN3 pin outputs a high level, and the duration of the high level is determined by the size of R14 and C1.

3. The memristive neural network circuit with partially enhanced operant conditioning according to claim 2, wherein: In the satiety module, whenever the reward signal N2 for pressing twice or the reward signal N3 for pressing four times appears, the voltage-controlled switch S1 is turned on, and the voltage acts on the memristor M1 and ABM1 after passing through SUM1 and SUM2. The voltage signal VOP5 output by the operational amplifier OP5 acts on the digital operation circuit ABM1, and the output of ABM1 represents the resistance value of M1; when N2 and N3 are both low, there is no food reward at this time, NOT3 is turned on, and outputs a high-level voltage to act on the gate of NMOS. At this time, the MOS tube is turned on and outputs a voltage. The voltage causes the memristor value of M1 to continuously increase and gradually return to the initial value. VABM1 continues to increase, and the satiety gradually returns to its initial state.

4. The memristive neural network circuit with partially enhanced operant conditioning according to claim 3, wherein: In the reward module, the reward signal N2 represents that a food reward is generated every time the lever is pressed twice, and a high-level pulse is output; the reward signal N3 represents that a food reward is generated every time the lever is pressed four times, and a high-level pulse is output; when the output signal VOP3 of the comparator OP3 and the reward signal N2 are at a high level, AND2 is turned on and outputs a 10V voltage, and outputs the reward voltage through the adder; when the output signal VOP3 of the pressing module and the reward signal N3 are at a high level, AND3 outputs a 4V voltage, which also passes through the adder OP4 and is output as the reward voltage.

5. The memristive neural network circuit with partially enhanced operant conditioning according to claim 4, wherein: In the synaptic module, when the voltage signal VOP4 output by adder OP4 is input to OP6, the output of OP6 causes the memristor value of M2 to decrease. After passing through OP7 and ABM2, the output voltage value continues to decrease, indicating that the mouse's pressing frequency is continuously increasing.

Images