Multifunctional operant conditioning neural network circuit with blocking and competition effects

Abstract

The application provides a multi-functional operant conditioning reflex neural network circuit with blocking and competition effect, wherein a stimulating signal is connected with an input end of a hippocampus module, an exciting signal and an output signal are connected with input ends of a cerebellum module, the exciting signal, the hippocampus module and the cerebellum module are connected with an input end of a voltage control module, an output end of the voltage control module is connected with an experience module and a feedback module respectively, an output end of the experience module is connected with a decision module and the feedback module respectively, an output end of the feedback module is connected with the voltage control module and the experience module respectively, and an output end of the decision module is connected with the feedback module and obtains the output signal. The application realizes learning of operant conditioning reflex under multiple input signals, can simulate decision judgment of animals in a complex natural environment, and widens the possibility of the existing operant conditioning reflex circuit.

Description

Technical Field

[0001] This invention relates to the technical field of digital-analog circuits, and more particularly to a multifunctional operant conditioning neural network circuit with blocking and competition effects, which utilizes memristors to further explore and extend Skinner's operant conditioning ideas. Background Technology

[0002] Artificial intelligence (AI) technology is developing rapidly and has promising application prospects, such as image recognition and optimal decision-making. However, most current implementations are primarily software-based, which suffers from the drawback of requiring massive hardware support for complex calculations, making deployment difficult in some industrial applications. A novel in-memory computing neural chip architecture based on memristors has emerged. Its advantages include multi-bit capability, non-volatility, low power consumption, and small size, making it an important research direction for the hardware implementation of AI.

[0003] Professor Cai Shaotang first proposed the concept of memristors in 1971, but it remained largely ignored for many years until 2008 when Hewlett-Packard Labs created a physical memristor that could function properly in a circuit, leading to a rapid development in its applications. Because the resistance of a memristor can change with applied voltage or current and retain its original resistance after power is cut off—similar to the change in synaptic weights under bioelectrical stimulation—memristors can be used to simulate the basic functions of synapses, thereby achieving simulated learning and memory of biological behaviors. Compared to the traditional structure of multiple transistors and capacitors, memristors are characterized by their nanoscale size and low power consumption, thus reducing circuit complexity when used as a substitute. Furthermore, due to their memory properties, memristors have a natural advantage as artificial synapses.

[0004] Operant conditioning is a fundamental theory in artificial intelligence reinforcement learning. American behaviorist psychologist B.F. Skinner proposed the operant conditioning theory through extensive experiments. Some domestic teams have applied operant conditioning to canine training, and some circuits can currently implement basic operant conditioning. However, these are all operant conditioning under a single stimulus, and do not consider operant conditioning under the simultaneous influence of multiple stimuli during learning. In nature and learning, however, multiple signals are often input simultaneously in a complex manner. Therefore, based on the above research findings, this invention realizes operant conditioning under the interaction of multiple stimuli. Summary of the Invention

[0005] To address the technical problem that existing operant conditioning circuits do not consider the influence of multiple stimuli on operant conditioning and have poor applicability, this invention proposes a multifunctional operant conditioning neural network circuit with blocking and competition effects. This circuit enables the learning of operant conditioning under multiple input signals and can simulate the decision-making judgments made by animals in complex natural environments.

[0006] To achieve the above objectives, the technical solution of the present invention is implemented as follows: A multifunctional operant conditioning neural network circuit with blocking and competition effects includes a hippocampal module, a cerebellum module, a voltage control module, an experience module, and a decision module. A stimulus signal is connected to the input terminal of the hippocampal module. One excitation signal and an output signal are both connected to the input terminal of the cerebellum module. Another excitation signal, the output terminal of the hippocampal module, and the output terminal of the cerebellum module are all connected to the input terminal of the voltage control module. The output terminal of the voltage control module is connected to the input terminal of the experience module and one input terminal of the feedback module. The output terminal of the experience module is connected to the input terminal of the decision module and another input terminal of the feedback module. The output terminal of the feedback module is connected to the input terminals of the voltage control module and the experience module. The stimulus signal is connected to the input terminal of the decision module. One output terminal of the decision module is connected to one input terminal of the feedback module. The other output terminal of the decision module receives an output signal.

[0007] Preferably, the stimulation signals include pulse voltage signals N1 and N2, one excitation signal is an action signal N4, and the other excitation signal is a food signal N3. The hippocampal module includes hippocampal module I and hippocampal module II, the voltage control module includes voltage control module I and voltage control module II, and the experience module includes experience module I and experience module II. The pulse voltage signal N1 is connected to the input terminals of the decision-making module and hippocampal module I, respectively. The output terminals of hippocampal module I, food signal N3, and cerebellar module are all connected to the input terminal of voltage control module I. The output terminal of voltage control module I is connected to the input terminal of experience module I, and the output terminal of experience module I is connected to the input terminals of the decision-making module and voltage control module I, respectively. The other input terminal of the feedback module is connected; the pulse voltage signal N2 is connected to the input terminals of the decision module and hippocampus module I respectively; the output terminal of hippocampus module I, food signal N3, and the output terminal of cerebellum module are all connected to the input terminal of voltage control module II; the output terminal of voltage control module II is connected to the input terminal of experience module II; the output terminal of experience module II is connected to the input terminal of decision module, the input terminal of voltage control module II, and the other input terminal of feedback module respectively; the output terminal of feedback module is connected to the input terminals of voltage control module I, voltage control module II, experience module I, and experience module II respectively; the output signal is connected to the input terminal of cerebellum module through diode D46, and the action signal N4 is connected to the input terminal of cerebellum module through diode D47.

[0008] Preferably, both hippocampal module I and hippocampal module II of the hippocampal module include a first-stage charging circuit, a second-stage charging circuit, and a logic judgment circuit. The pulse voltage signal N1 is connected to the input terminals of the first-stage charging circuit, the second-stage charging circuit, and the logic judgment circuit, respectively. The output terminal of the first-stage charging circuit is connected to the input terminal of the second-stage charging circuit, the output terminal of the second-stage charging circuit is connected to the logic judgment circuit, and the output terminal of the logic judgment circuit is connected to the input terminal of voltage control module I or voltage control module II.

[0009] Preferably, the cerebellar module includes resistors R4 and R5, capacitor C3, diode D3, NOT gate D44, and AND gate D45. The output signal is connected to the positive terminal of diode D3 through diode D46, and the action signal N4 is connected to the input terminal of NOT gate D44 through diode D47. The negative terminal of diode D3 is connected to one end of resistor R4, one end of capacitor C3, and one input terminal of AND gate D45. The output terminal of NOT gate D44 is connected to the other input terminal of AND gate D45. The output terminal of AND gate D45 is connected to one end of resistor R5 and the input terminal of voltage control module I or voltage control module II. The other ends of resistor R4, resistor R5, and capacitor C3 are all grounded.

[0010] Preferably, both voltage control module I and voltage control module II include a logic input circuit, a feedback receiving circuit, a learning control circuit, and an output circuit I. The food signal N3, the output terminal of the cerebellum module, and the output terminal of hippocampus module I or hippocampus module II are all connected to the input terminal of the logic judgment circuit. The output terminal of the logic input circuit is connected to one input terminal of the feedback module, one input terminal of the feedback receiving circuit, and the input terminal of the learning control circuit, respectively. The other input terminal of the feedback receiving circuit is connected to one output terminal of experience module I or experience module II. The output terminal of the learning control circuit, the output terminal of the feedback receiving circuit, and the output terminal of the feedback module are all connected to the input terminal of output circuit I. The output terminal of output circuit I is connected to the input terminal of experience module I or experience module II.

[0011] Preferably, both experience module I and experience module II include an experience generation circuit, an experience control circuit, and an experience feedback circuit. The output circuit of voltage control module I or voltage control module II is connected to the input terminal of the experience generation circuit. One output terminal of the experience generation circuit is connected to one input terminal of the experience control circuit, and the other output terminal of the experience generation circuit is connected to the feedback module and the decision module, respectively. The other two input terminals of the experience control circuit are connected to the output terminals of the logic input circuit and the learning control circuit, respectively. The output terminal of the experience control circuit is connected to the input terminal of the experience feedback circuit, and the output terminal of the experience feedback circuit is connected to the other input terminal of the feedback receiving circuit.

[0012] Preferably, the feedback module includes a feedback circuit I, a feedback circuit II, and an experience feedback circuit. One input terminal of feedback circuit I is connected to the other output terminal of the experience generation circuit of experience module I, and the output terminal of feedback circuit I is connected to the input terminal of the output circuit I of voltage control module I. One input terminal of feedback circuit II is connected to the other output terminal of the experience generation circuit of experience module II, and the output terminal of feedback circuit II is connected to the input terminal of the output circuit I of voltage control module II. The output terminal of the decision module, the output terminals of the logic input circuits of voltage control module I and voltage control module II are all connected to the input terminal of the experience feedback circuit. The output terminal of the experience feedback circuit is connected to the other input terminals of feedback circuit I and feedback circuit II, respectively, and the output terminal of the experience feedback circuit is connected to the input terminals of the experience generation circuits of experience module I and experience module II, respectively.

[0013] Preferably, the decision module includes a first delay circuit, a second delay circuit, a first comparison circuit, a second comparison circuit, a trigger circuit, and a logic output circuit. The input terminal of the first comparison circuit is connected to the other output terminal of the experience generation circuit of experience module I, and the input terminal of the second comparison circuit is connected to the other output terminal of the experience generation circuit of experience module II. The output terminals of the first comparison circuit and the second comparison circuit are both connected to the input terminal of the experience feedback circuit of the feedback module. The pulse voltage signal N1 is connected to the input terminal of the first delay circuit, and the pulse voltage signal N2 is connected to the input terminal of the second delay circuit. The output terminals of the first delay circuit, the second delay circuit, the first comparison circuit, and the second comparison circuit are all connected to the input terminal of the logic output circuit. The output terminal of the trigger circuit is connected to the input terminal of the logic circuit, and the output terminal of the logic output circuit outputs an output signal.

[0014] Preferably, the trigger circuit includes a capacitor C7, resistors R31, R32, R33, and R34, diodes D10 and D11, and an operational amplifier OP8. One end of capacitor C7 and one end of resistor R33 are grounded. The other end of one end of capacitor C7 is connected to the inverting input of operational amplifier OP8, one end of resistor R31, and one end of resistor R32, respectively. The other end of resistor R31 is connected to the anode of diode D10, and the other end of resistor R32 is connected to the anode of diode D11. The other end of resistor R33 is connected to one end of resistor R34 and the non-inverting input of operational amplifier OP8, respectively. The cathodes of diodes D10 and D11, and the other end of resistor R34 are all connected to the output of operational amplifier OP8. The output of operational amplifier OP8 is connected to the logic output.

[0015] Preferably, the first-stage charging circuit includes a first resistor, a first capacitor, a first diode, and a first PMOS transistor. The pulse voltage signal N1 is connected to the positive terminal of the first diode and the gate of the first PMOS transistor, respectively. The negative terminal of the first diode is connected to the source of the first PMOS transistor and one end of the first capacitor, respectively. The drain of the first PMOS transistor is connected to the input terminal of the second-stage charging circuit and one end of the first resistor, respectively. The other end of the first capacitor and the other end of the first resistor are both grounded.

[0016] The second-stage charging circuit includes a second resistor, a second capacitor, a second diode, and a second PMOS transistor. The drain of the first PMOS transistor is connected to the gate of the second PMOS transistor. The pulse voltage signal N1 is connected to the positive terminal of the second diode. The negative terminal of the second diode is connected to the source of the second PMOS transistor and one end of the second capacitor. The drain of the second PMOS transistor is connected to the input terminal of the logic judgment circuit and one end of the second resistor. The other ends of the second capacitor and the second resistor are both grounded.

[0017] The logic judgment circuit includes a first NOT gate, a first AND gate, and a third resistor. The pulse voltage signal N1 is connected to the input terminal of the first NOT gate. The output terminal of the first NOT gate and the drain of the second PMOS transistor are both connected to the input terminal of the first AND gate. The output terminal of the first AND gate is connected to one end of the third resistor and the input terminal of voltage control module I or voltage control module II, respectively. The other end of the third resistor is grounded.

[0018] The logic input circuit includes a second NOT gate, a second AND gate, a third AND gate, and a fourth AND gate. The output of the first AND gate of the logic judgment circuit of hippocampal module I or hippocampal module II and the output of AND gate D45 of the cerebellar module are both connected to the input of the fourth AND gate. The output of the fourth AND gate is connected to the input of feedback receiving circuit I, one input of the third AND gate, and one input of the second AND gate. The food signal N3 is connected to the input of the second NOT gate and the other input of the third AND gate. The output of the third AND gate is connected to one input of the feedback module and the input of the learning control circuit I. The output of the second NOT gate is connected to the other input of the second AND gate. The output of the second AND gate is connected to the input of the learning control circuit I and the input of experience module I or experience module II.

[0019] The feedback receiving circuit includes a first voltage-controlled switch, a fourth resistor, a first voltage source, and a first subtractor. The positive input terminal of the first voltage-controlled switch is connected to the output terminal of the fourth AND gate. An output terminal of empirical module I or empirical module II and the positive terminal of the first voltage source are both connected to the first subtractor. The output terminal of the first subtractor is connected to the first contact of the first voltage-controlled switch. The second contact of the first voltage-controlled switch is connected to the input terminal of output circuit I and one end of the fourth resistor, respectively. The negative input terminal of the first voltage-controlled switch, the other end of the fourth resistor, and the negative terminal of the first voltage source are all grounded.

[0020] The learning control circuit includes a second voltage-controlled switch, a second voltage source, a third voltage source, a first NMOS transistor, a fifth resistor, and a sixth resistor. The output of the second AND gate is connected to the positive input of the second voltage-controlled switch. The first contact of the second voltage-controlled switch is connected to the positive terminal of the second voltage source. The second contact of the second voltage-controlled switch is connected to one end of the sixth resistor and the input of either experience module I or experience module II. The positive terminal of the third voltage source is connected to the drain of the first NMOS transistor. The gate of the first NMOS transistor is connected to the output of the third AND gate. The source of the first NMOS transistor is connected to one end of the fifth resistor and the input of output circuit I. The negative input of the second voltage-controlled switch, the other end of the fifth resistor, the other end of the sixth resistor, the negative terminal of the third voltage source, and the negative terminal of the second voltage source are all grounded.

[0021] The output circuit I includes a first adder, a second adder, and a third adder. The input terminal of the first adder is connected to the second contact of the first voltage-controlled switch and the source of the first NMOS transistor, respectively. The output terminal of the first adder and the positive terminal of the fourth voltage source are connected to the two input terminals of the second adder, respectively. The negative terminal of the fourth voltage source is grounded. The output terminal of the second adder and one output terminal of the empirical module I or the empirical module II are connected to the two input terminals of the third adder, respectively. The output terminal of the third adder is connected to the input terminal of the empirical module I or the empirical module II.

[0022] The experience generation circuit includes a third voltage-controlled switch, a first memristor, a third operational amplifier, a seventh resistor, and a mathematical operation unit. The non-inverting input terminal of the third voltage-controlled switch is connected to the output terminal of the feedback module. The first contact of the third voltage-controlled switch is connected to the output terminal of the third adder of the output circuit I. The inverting input terminal of the third voltage-controlled switch is grounded. The second contact of the third voltage-controlled switch is connected to the positive terminal of the first memristor and one input terminal of the mathematical operation unit. The negative terminal of the first memristor is connected to the inverting input terminal of the third operational amplifier and one end of the seventh resistor. The other end of the seventh resistor is connected to the output terminal of the third operational amplifier. The non-inverting input terminal of the third operational amplifier is grounded. The output terminal of the third operational amplifier is connected to the other input terminal of the mathematical operation unit and one input terminal of the experience control circuit. The output terminal of the mathematical operation unit is connected to the other input terminal of the feedback module and the input terminal of the decision module.

[0023] The empirical control circuit includes a second operational amplifier, a fifth voltage source, a sixth voltage source, an eighth resistor, a fifth AND gate, a second NMOS transistor, and a fourth adder. The output of the third operational amplifier is connected to the non-inverting input of the second operational amplifier, and the inverting input of the second operational amplifier is grounded. The output of the second operational amplifier and the output of the second AND gate of the logic input circuit are both connected to the input of the fifth AND gate. The output of the fifth AND gate is connected to the gate of the second NMOS transistor. The drain of the second NMOS transistor is connected to the positive terminal of the fifth voltage source. The source of the second NMOS transistor is connected to one end of the eighth resistor and one input of the fourth adder. The other input of the fourth adder is connected to the second contact of the second voltage-controlled switch of the learning control circuit. The negative terminal of the fifth voltage source and the other end of the eighth resistor are both grounded. The output of the fourth adder is connected to the input of the empirical feedback circuit.

[0024] The empirical feedback circuit includes a second memristor, a first operational amplifier, a ninth resistor, a tenth resistor, and a diode I. The output terminal of the fourth adder is connected to the positive terminal of the second memristor. The negative terminal of the second memristor is connected to one end of the ninth resistor and the inverting input terminal of the first operational amplifier. The non-inverting input terminal of the first operational amplifier is grounded. The other end of the ninth resistor is connected to the output terminal of the first operational amplifier. The output terminal of the first operational amplifier is connected to the negative terminal of the diode I. The positive terminal of the diode is connected to one input terminal of the third adder of the feedback receiving circuit and one end of the tenth resistor. The other end of the tenth resistor is grounded.

[0025] Both feedback circuit I and feedback circuit II include a fourth operational amplifier and a fourth voltage-controlled switch. The non-inverting input terminal of the fourth operational amplifier is grounded, and the inverting input terminal of the fourth operational amplifier is connected to one end of the eleventh resistor and the twelfth resistor, respectively. The other end of the eleventh resistor is connected to the output terminal of the mathematical operation unit of the experience generation circuit of experience module I or experience module II. The other end of the twelfth resistor is connected to the output terminal of the fourth operational amplifier. The output terminal of the fourth operational amplifier is connected to the first contact of the fourth voltage-controlled switch. The non-inverting input terminal of the fourth voltage-controlled switch is connected to one output terminal of the experience feedback circuit. The inverting input terminal of the fourth voltage-controlled switch is grounded. The second contact of the fourth voltage-controlled switch is connected to one end of the thirteenth resistor and one input terminal of the third adder of the output circuit I of voltage control module I or voltage control module II, respectively. The other end of the thirteenth resistor is grounded.

[0026] The experience feedback circuit includes a sixth AND gate, a seventh AND gate, and an eighth AND gate. The input terminal of the sixth AND gate is connected to the output terminal of the second AND gate of the logic input circuit of voltage control module I or voltage control module II. The output terminal of the sixth AND gate is connected to one input terminal of the seventh AND gate, one input terminal of the eighth AND gate, and the positive input terminal of the fourth voltage-controlled switch of feedback circuit I and feedback circuit II. The other input terminal of the seventh AND gate and the other input terminal of the eighth AND gate are both connected to the output terminal of the decision module.

[0027] Both the first delay circuit and the second delay circuit include a third capacitor, a fourteenth resistor, a fifteenth resistor, a diode II, a third NOT gate, and a ninth AND gate. The pulse voltage signal N1 or the pulse voltage signal N2 is connected to the positive terminal of the diode II and the input terminal of the third NOT gate, respectively. The output terminal of the third NOT gate is connected to one input terminal of the ninth AND gate. The negative terminal of the diode II is connected to the other input terminal of the ninth AND gate, one end of the third capacitor, and one end of the fourteenth resistor, respectively. The output terminal of the ninth AND gate is connected to one end of the fifteenth resistor and the input terminal of the logic output circuit, respectively. The other ends of the third capacitor, the fourteenth resistor, and the fifteenth resistor are all grounded.

[0028] Both the first and second comparison circuits include diode III, diode IV, a seventh voltage source, an eighth voltage source, and a fifth operational amplifier. The output terminal of the mathematical operation unit of the experience generation circuit of experience module I or experience module II is connected to the inverting input terminal of the fifth operational amplifier, the positive terminal of diode III, and the positive terminal of diode IV, respectively. The non-inverting input terminal of the fifth operational amplifier is connected to the positive terminal of the eighth voltage source. The negative terminals of diode III, diode IV, and the eighth voltage source are all grounded. The output terminal of the fifth operational amplifier is connected to the input terminal of the logic output circuit and the other input terminal of the seventh AND gate or the other input terminal of the eighth AND gate of the experience feedback circuit of the feedback module, respectively.

[0029] The logic output circuit includes AND gates D33, D36, D37, D39, OR gates D34, D35, and D38. The output of the ninth AND gate of the first delay circuit is connected to one input of AND gate D33 and one input of AND gate D36, respectively. The other input of AND gate D33 is connected to the output of the fifth operational amplifier of the first comparator circuit. The other input of AND gate D36 is connected to the output of the operational amplifier OP8 of the trigger circuit. The outputs of AND gate D33 and AND gate D36 are both connected to the input of OR gate D34. The output of the ninth AND gate in the second delay circuit is connected to one input of AND gate D37 and one input of AND gate D39, respectively. The other input of AND gate D39 is connected to the output of the fifth operational amplifier of the first comparator circuit. The other input of AND gate D37 is connected to the output of operational amplifier OP8 of the trigger circuit. The outputs of AND gate D37 and AND gate D39 are both connected to the input of OR gate D38. The outputs of OR gate D34 and OR gate D38 are both connected to the input of OR gate D35. The output of OR gate D35 outputs signal N5.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] 1) This invention, taking into account the influence of multiple stimulus signals on operant conditioning, realizes the blocking and masking process of operant conditioning, and realizes associative learning of target signals in the presence of interference signals.

[0032] 2) This invention achieves adaptive learning under low reward conditions through negative feedback formed by two memristor learning circuits. This kind of learning has an evolutionary advantage because in nature, behaviors are not absolutely incidental reinforcement stimuli. After a failure, they may be reinforced after multiple attempts. Therefore, the circuit of this invention has more advantages in environmental adaptation.

[0033] 3) This invention uses hippocampal and cerebellar module circuits to receive input signals, thereby realizing the effect of the immediacy of stimulation on operant conditioning, and taking into account the time difference between the circuit receiving external signals and making a response.

[0034] This invention utilizes a composite learning module and a feedback module to realize the influence of multiple external stimuli on the formation of operant conditioning, achieving adaptive learning under low-reward conditions caused by rhythmic learning and forgetting. When multiple stimulus signals form positive feedback with an action, the action will preferentially form an operant conditioning with the strongest signal. This realizes the influence of multiple stimuli on the formation process of operant conditioning, enabling operant conditioning learning under multiple input signals. It can simulate the decision-making and judgment made by animals in complex natural environments, broadening the possibilities of existing operant conditioning circuits and contributing to further research on operant conditioning. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a circuit diagram of the present invention.

[0037] Figure 2 The diagram shows a simulation of the operant conditioning process formed by the learning and forgetting process of operant conditioning in this invention, considering time intervals and operant conditioning. In the diagram, (a) is N1, (b) is N3, (c) is N4, (d) is N5, and (e) is M2.

[0038] Figure 3 The diagram shows the formation process of the operant conditioning reflex in adaptive learning under low reward conditions according to the present invention, where (a) is N1, (b) is M2, (c) is N3, (d) is N5, and (e) is N4.

[0039] Figure 4 The diagram shows a simulation of the formation process of the operational conditioned reflex of the masking process of the present invention, wherein (a) is N3, (b) is N1, (c) is N2, (d) is N5, (e) is N4, and (f) is M2 and M3.

[0040] Figure 5 The diagram shows the formation process of the blocking process operational conditioned reflex of the present invention, wherein (a) is N3, (b) is N1, (c) is N2, (d) is N5, (e) is M2 and M3, and (f) is N4. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] like Figure 1 As shown, a multifunctional operant conditioning neural network circuit with blocking and competition effects includes a hippocampal module, a cerebellar module, a voltage control module, an experience module, a feedback module, and a decision module. A stimulus signal is connected to the input terminals of the hippocampal module and the decision module. One stimulus signal and one output signal are both connected to the input terminal of the cerebellar module. Another stimulus signal, the output terminals of the hippocampal module and the cerebellar module are all connected to the input terminal of the voltage control module. The output terminal of the voltage control module is connected to the input terminal of the experience module and one input terminal of the feedback module. The output terminal of the experience module is connected to the input terminal of the decision module and the other input terminal of the feedback module. The output terminal of the feedback module is connected to the input terminals of the voltage control module and the experience module. The stimulus signal is connected to the input terminal of the decision module. One output terminal of the decision module is connected to one input terminal of the feedback module. The other output terminal of the decision module receives an output signal. The hippocampus module receives and delays the transmission of external stimulus signals; the cerebellum module perceives and delays the transmission of action signals; the voltage control module controls the voltage of the experience modules, including receiving various feedback voltages to control the learning and forgetting of the experience modules; the experience modules are responsible for storing and updating the learning weights to achieve learning, forgetting, and reinforcement learning; the feedback module is responsible for implementing feedback between multiple experience modules to achieve masking and blocking effects; and the decision module weights multiple experience modules and determines whether to output an action signal, as well as the probabilistic response to the input signal.

[0043] The pulse power signals include pulse voltage signals N1, N2, N3, and N4. Pulse voltage signals N1 and N2 both represent external stimulus signals and are connected to their corresponding hippocampal modules. Pulse voltage signal N3 represents food signals and is connected to the voltage control module. Pulse voltage signal N4 represents action signals and is connected to the cerebellum module. The hippocampal module includes hippocampal module I and hippocampal module II; the voltage control module includes voltage control module I and voltage control module II; the experience module includes experience module I and experience module II; pulse voltage signal N1 is connected to the input terminals of the decision module and hippocampal module I, respectively; the output terminal of hippocampal module I, food signal N3, and the output terminal of the cerebellum module are all connected to the input terminal of voltage control module I; the output terminal of voltage control module I is connected to the input terminal of experience module I; the output terminal of experience module I is connected to the input terminal of the decision module, the input terminal of voltage control module I, and another input terminal of the feedback module, respectively; pulse voltage signal N2 is connected to the input terminals of the decision module and hippocampal module I, respectively; the hippocampal... The output terminals of Module I, food signal N3, and cerebellum module are all connected to the input terminal of voltage control module II. The output terminal of voltage control module II is connected to the input terminal of experience module II. The output terminal of experience module II is connected to the input terminal of decision module, the input terminal of voltage control module II, and another input terminal of feedback module. The output terminal of feedback module is connected to the input terminals of voltage control module I, voltage control module II, experience module I, and experience module II. The output signal N5 is connected to the input terminal of cerebellum module through diode D46, and the action signal N4 is connected to diode D47. Diodes D46 and D47 are used to isolate artificial action signals and active output signals.

[0044] like Figure 1 As shown, hippocampal module I includes a first-stage charging circuit, a second-stage charging circuit, and a logic judgment circuit I. A pulse voltage signal N1 is connected to the inputs of both the first-stage charging circuit and the logic judgment circuit I. The output of the first-stage charging circuit is connected to the input of the second-stage charging circuit, and the output of the second-stage charging circuit is connected to the input of the logic judgment circuit I. The output of the logic judgment circuit I is connected to the input of voltage control module I. Hippocampal module II includes a third-stage charging circuit, a fourth-stage charging circuit, and a logic judgment circuit II. A pulse voltage signal N2 is connected to the inputs of both the third-stage charging circuit and the logic judgment circuit II. The output of the third-stage charging circuit is connected to the input of the fourth-stage charging circuit, and the input of the logic judgment circuit is connected to the output of the second-stage charging circuit. The output of the logic judgment circuit II is connected to the input of voltage control module II.

[0045] The first-stage charging circuit includes a resistor R1, a capacitor C1, a diode D1, and a PMOS transistor T1. The anode of diode D1 is connected to the signal source N1, and the cathode of diode D1 is connected to the source of transistor T1 and one end of capacitor C1. The other end of capacitor C1 is grounded. The gate of transistor T1 is connected to the signal source N1, and the drain of transistor T1 is connected to the gate of transistor T2 and one end of resistor R1. The other end of resistor R1 is grounded. When a high-level input signal is received, the signal source N1 charges capacitor C1 through diode D1. When the signal source N1 becomes low-level, diode D1 prevents capacitor C1 from discharging from the input terminal, and capacitor C1 discharges through resistor R1. The values ​​of resistor R1 and capacitor C1 determine the discharge time, i.e., the delay duration. PMOS transistor T1 determines the opening and closing of the discharge path. The function of the first-stage charging circuit is to store the stimulus signal N1 in capacitor C1 and delay its transmission for a certain period of time. Because there is a time difference between the animal receiving the external stimulus and reacting, this signal needs a delay to synchronize with other signals.

[0046] The second-stage charging circuit includes a resistor R2, a capacitor C2, a diode D2, and a PMOS transistor T2. The anode of diode D2 is connected to signal source N1, and the cathode of diode D2 is connected to the source of transistor T2 and one end of capacitor C2. The other end of capacitor C2 is grounded. The gate of transistor T2 is connected to the drain of transistor T1, and the drain of transistor T2 is connected to one end of resistor R2. The other end of resistor R2 is grounded. When a high-level input signal is received, signal source N1 charges capacitor C2 through diode D2. When signal source N2 becomes low-level, diode D2 prevents capacitor C2 from discharging from the input terminal, and capacitor C2 discharges through resistor R2. The values ​​of resistor R2 and capacitor C2 determine the discharge time, i.e., the delay duration. PMOS transistor T2 determines the opening and closing of the discharge path; the path is open when the voltage on C1 is below 1.3V. The second-stage charging circuit can maintain the output of the stimulus signal N1 for a controllable period of time, so that the stimulus signal can be synchronized with the food signal. This is because after the animal receives the signal and makes the correct response, it will take some time before the external world generates a reward food signal.

[0047] The logic circuit I includes NOT gate D16, AND gate D17, and resistor R3. The input of NOT gate D16 is connected to signal source N1, and the output of NOT gate D16 is connected to one input of AND gate D17. The other input of AND gate D17 is connected to the drain of transistor T2, and the output of AND gate D17 is connected to voltage control module I and one end of resistor R3. The other end of resistor R3 is grounded. Logic circuit I is responsible for the output of hippocampus module I. It will only generate an output when there is no stimulus signal N1, capacitor C2 of the second-stage charging circuit is charged, and the transistor of the second-stage charging circuit is turned on.

[0048] The input of hippocampal module I is connected to the stimulation signal N1. First, the stimulation signal charges capacitor C1 in the first-stage charging circuit through diode D1. Simultaneously, the stimulation signal also acts on the gate of transistor T1, causing the drain and source of transistor T1 to de-conduct, thus retaining the charge on capacitor C1. At the same time, the stimulation signal N1 also charges capacitor C2 in the similarly structured second-stage charging circuit. When the stimulation signal N1 disappears, transistor T1 conducts, but due to the presence of diode D1, capacitor C1 can only discharge through resistor R1. While capacitor C1 is discharging, capacitor C2 cannot discharge because the source of transistor T1 is connected to the gate of transistor T2. After capacitor C1 finishes discharging, capacitor C2 begins to discharge through transistor T2. Due to the presence of diode D2, capacitor C2 discharges through resistor R2. When the voltage across resistor R2 is greater than 2.4V, AND gate D17 detects a high level. When the stimulus signal N1 is present or when capacitor C2 is discharging, the second input of AND gate D17 is at a high level. However, there is an NOT gate D16 between the first input of AND gate D17 and the stimulus signal N1, so AND gate D17 only generates an output for a short period of time before capacitor C2 discharges.

[0049] The third-stage charging circuit includes resistor R6, capacitor C4, diode D4, and PMOS transistor T5. The anode of diode D4 is connected to signal source N2, and the cathode of diode D4 is connected to the source of transistor T5 and one end of capacitor C4. The other end of capacitor C4 is grounded. The gate of transistor T5 is connected to signal source N2, and the drain of transistor T5 is connected to the gate of transistor T6 and one end of resistor R6. The other end of resistor R6 is grounded. The fourth-stage charging circuit includes resistor R7, capacitor C5, diode D5, and PMOS transistor T6. The anode of diode D5 is connected to signal source N2, and the cathode of diode D5 is connected to the source of transistor T6 and one end of capacitor C5. The other end of capacitor C5 is grounded. The gate of transistor T6 is connected to the drain of transistor T5, and the drain of transistor T6 is connected to one end of resistor R7. The other end of resistor R7 is grounded.

[0050] The logic judgment circuit II includes NOT gate D42, AND gate D43, and resistor R8. The input of NOT gate D42 is connected to signal source N2, the output of NOT gate D42 is connected to one input of AND gate D43, the other input of AND gate D43 is connected to the drain of transistor T6, the output of AND gate D43 is connected to voltage control module II and one end of resistor R8, and the other end of resistor R8 is grounded.

[0051] like Figure 1As shown, the cerebellum module includes resistor R4, resistor R5, capacitor C3, diode D3, NOT gate D44, and AND gate D45. The anode of diode D3 is connected to the action signal N4, and the cathode of diode D3 is connected to one end of capacitor C3, one end of resistor R4, and one input of AND gate D45. The other end of capacitor C3 and resistor R4 are grounded. The input of NOT gate D44 is connected to the action signal N4, and the output of NOT gate D44 is connected to the other input of AND gate D45. The output of AND gate D45 is connected to one end of resistor R5, one input of AND gate D21 in voltage control module I, and one input of AND gate D26 in voltage control module II. The other end of resistor R5 is grounded. When the input is a high-level signal, the signal charges capacitor C3 through diode D3. When the signal source becomes low-level, diode D3 prevents capacitor C3 from discharging from the input terminal, and capacitor C3 discharges through resistor R1. The values ​​of resistor R4 and capacitor C3 determine the discharge time, i.e., the delay duration. The role of the cerebellar module is to delay action signals for a period of time so that they are consistent with food signals.

[0052] Action signal N4 charges capacitor C3 through diode D3. Simultaneously, when action signal N4 is present, the second input of AND gate D45 is at a high level. The first input of AND gate D45 is connected to action signal N4 through NOT gate D44. When action signal N4 is present, AND gate D45 will not produce an output. When action signal N4 disappears, capacitor C3 begins to discharge, and AND gate D45 begins to produce an output. The duration of the output is determined by capacitor C3 and resistor R4.

[0053] like Figure 1 As shown, voltage control module I includes logic input circuit I, feedback receiving circuit I, learning control circuit I, and output circuit I. The input terminals of voltage control module I are connected to the output of hippocampus module I, the output of cerebellum module I, the food reward signal N3, and the input of feedback module I, respectively. The output terminal of voltage control module I is connected to experience module I. The output terminals of food signal N3, cerebellum module, and hippocampus module I are all connected to the input terminals of logic judgment circuit I. The output terminal of logic input circuit I is connected to one input terminal of feedback module I, one input terminal of feedback receiving circuit I, and the input terminal of learning control circuit I, respectively. The other input terminal of feedback receiving circuit I is connected to one output terminal of experience module I. The output terminals of learning control circuit I, feedback receiving circuit I, and feedback module are all connected to the input terminals of output circuit I. The output terminal of output circuit I is connected to the input terminal of experience module I.

[0054] The logic input circuit I includes NOT gate D18, AND gate D19, AND gate D20, and AND gate D21. The input of NOT gate D18 is connected to the food reward signal N3. The output of NOT gate D18 is connected to one input of AND gate D19. The other input of AND gate D19 is connected to the output of AND gate D21. The output of AND gate D19 is connected to the positive input of the voltage-controlled switch S2 in the learning control circuit I and one input of AND gate D22 in the experience module I. One input of AND gate D20 is connected to the food reward signal N3. The other input of AND gate D20 is connected to the output of AND gate D21. The output of AND gate D20 is connected to the gate of transistor T3 in the learning control circuit I and one input of AND gate D24 in the feedback module. One input of AND gate D21 is connected to the output of AND gate D17 in the hippocampus module I. The other input of AND gate D21 is connected to the output of AND gate D45 in the cerebellum module. The output of AND gate D21 is connected to the other input of AND gate D19, the input of AND gate D20, and the positive input of the voltage-controlled switch S1 in the feedback receiving circuit I. This part of the circuit processes the synchronized stimulus signal, action signal, and food signal. When only the stimulus signal and action signal are present, a forgetting voltage is generated. When all three signals are present, a learning signal is generated. Its output also acts on the feedback module.

[0055] Feedback receiving circuit I includes a voltage-controlled switch S1, a resistor R9, a voltage source V4, and a subtractor SUB1. The positive input of voltage-controlled switch S1 is connected to the output of AND gate D21 in logic input circuit I, and the negative input of voltage-controlled switch S1 is grounded. The first contact of voltage-controlled switch S1 is connected to the output of subtractor SUB1, and the second contact of voltage-controlled switch S1 is connected to one input of adder SUM1 in output circuit I and one end of resistor R9. The other end of resistor R9 is grounded. The positive terminal of voltage source V4 is connected to the positive input of subtractor SUB1, and the negative terminal of voltage source V4 is grounded. The positive input of subtractor SUB1 is connected to the positive terminal of voltage source V4, and the negative input of subtractor SUB1 is connected to the positive terminal of diode D6 in empirical module I. The output of subtractor SUB1 is connected to the first contact of voltage-controlled switch S1. Voltage source V4 is the active forgetting voltage, which is combined with the feedback voltage through subtractor SUB1. The output of subtractor SUB1 is the active forgetting voltage applied to experience module I. When voltage-controlled switch S1 is not closed, the first input of adder SUM1 is connected to ground through resistor R9, achieving a 0V input. When voltage-controlled switch S2 is closed, the output of subtractor SUB1 directly applies to the first input of adder SUM1. This part of the circuit is mainly used to receive the feedback voltage from experience module I, and to reduce the natural forgetting voltage through this feedback voltage, thereby achieving reinforcement learning and adaptive learning under low-reward conditions.

[0056] The learning control circuit I includes voltage source V1, voltage source V2, NMOS transistor T3, resistor R10, resistor R11, and voltage-controlled switch S2. The positive terminal of voltage source V1 is connected to the first contact of voltage-controlled switch S2, and the negative terminal of voltage source V1 is grounded. The positive terminal of voltage source V2 is connected to the drain of transistor T3, and the negative terminal of voltage source V2 is grounded. The gate of transistor T3 is connected to the output of AND gate D20 in logic input circuit I. The drain of transistor T3 is connected to the positive terminal of voltage source V2, and the source of transistor T3 is connected to the output circuit I. One input of the adder SUM1 is connected to one end of resistor R10. The other end of resistor R10 is grounded. One end of resistor R11 is grounded. The other end of resistor R11 is connected to the first contact of voltage-controlled switch S2. The positive input of voltage-controlled switch S2 is connected to the output of AND gate D19 in logic input circuit I. The negative input of voltage-controlled switch S2 is grounded. The second contact of voltage-controlled switch S2 is connected to the positive terminal of voltage source V1. The first contact of voltage-controlled switch S2 is connected to the other end of resistor R11 and one input of adder SUM4 in empirical module I. Voltage source V1 is the learning voltage of memristor M1 in experience module I, and voltage source V2 is the learning voltage of memristor M2 in experience module I. Resistors R10 and R11 are responsible for the ground connection of the two outputs respectively. When voltage-controlled switch S2 and NMOS transistor T3 are not closed or conducting, these two outputs are 0V. When AND gate D19 outputs a high level, voltage-controlled switch S2 is closed, and learning voltage V1 flows to experience module I through voltage-controlled switch S2. When AND gate D20 outputs a high level, voltage-controlled switch S2 is closed, and learning voltage V12 flows to adder SUM2 through NMOS transistor T3. V3 is the natural forgetting voltage, which always acts on adder SUM3.

[0057] Output circuit I includes adders SUM1, SUM2, and SUM3, and voltage source V3. One input of adder SUM1 is connected to the first contact of voltage-controlled switch S1 in feedback receiving circuit I. The other input of adder SUM1 is connected to the source of transistor T3 in learning control circuit I. The output of adder SUM1 is connected to one input of adder SUM2. The other input of adder SUM2 is connected to the positive terminal of voltage source V3. The output of adder SUM2 is connected to one input of adder SUM3. The other input of adder SUM3 is connected to the first contact of voltage-controlled switch S5 in feedback module. The output of adder SUM3 is connected to the first contact of voltage-controlled switch S3 in experience module I. The positive terminal of voltage source V3 is connected to one input of SUM2, and the negative terminal of voltage source V3 is grounded. Adder SUM1 processes the learning and forgetting voltages and passes the results to adder SUM2. Adder SUM2 processes the natural forgetting voltage and passes the results to adder SUM3. Adder SUM3 processes the feedback voltage and passes the results to experience module I. These three adders combine the three control voltages—active forgetting voltage, learning voltage, and natural forgetting voltage—into a single output. Since the learning voltage is greater than the forgetting voltage, the output is positive when a learning voltage is present and negative when no learning voltage is present. This part primarily controls the learning and forgetting voltage values ​​of experience module I, including the control of learning experience and feedback experience.

[0058] When the hippocampus module I and the cerebellum module output signals, the two inputs of AND gate D21 are high, and AND gate D21 generates an output. This causes the voltage-controlled switch S1 to close, and the forgetting voltage V4 is processed by the subtractor SUB1 along with the feedback voltage from experience module I. Then, it is transmitted to adder SUM1 via voltage-controlled switch S1. If there is a food signal N3, AND gate D20 generates an output, representing the generation of the learning voltage. The output of AND gate D20 is connected to the gate of PMOS transistor T3, and the learning voltage source V2 is transmitted to adder SUM1 via transistor T3. If there is a stimulus signal N1 and an action signal N4, but no food signal N3, AND gate D20 outputs a low level, transistor T3 is not turned on, and the voltage flowing from the source of transistor T3 to the first input of adder SUM1 is 0V. In this case, adder SUM1 only outputs the forgetting voltage. Simultaneously, the food signal N3 flows through NOT gate D18 to the first input terminal of AND gate D19. AND gate D19 generates an output, closing voltage-controlled switch S2. The forgetting voltage V1 of the experience feedback circuit I flows through voltage-controlled switch S2 to adder SUM4, and the forgetting signal output by AND gate D19 also flows to experience module I. Adders SUM1, SUM2, and SUM3 combine the learning voltage, forgetting voltage, natural forgetting voltage, and feedback voltage and output them to voltage-controlled switch S3 in experience module I.

[0059] Voltage control module II includes logic input circuit II, feedback receiving circuit II, learning control circuit II, and output circuit II. The input terminals of voltage control module II are connected to the outputs of hippocampus module II, cerebellum module II, and food reward signal N3, respectively. The output terminals of voltage control module II are connected to feedback receiving circuit II, learning control circuit II, experience module II, and feedback module, respectively. The food signal N3, the output of cerebellum module II, and the output of hippocampus module II are all connected to the input terminals of logic judgment circuit II. The output terminals of logic input circuit II are connected to one input terminal of the feedback module, one input terminal of feedback receiving circuit II, and the input terminal of learning control circuit II, respectively. The other input terminal of feedback receiving circuit II is connected to one output terminal of experience module II. The output terminals of learning control circuit II, feedback receiving circuit II, and feedback module II are all connected to the input terminals of output circuit II. The output terminal of output circuit II is connected to the input terminal of experience module II.

[0060] The logic input circuit II includes NOT gate D29, AND gate D28, AND gate D27, and AND gate D26. The input of NOT gate D29 is connected to the food reward signal N3. The output of NOT gate D29 is connected to one input of AND gate D28. The other input of AND gate D28 is connected to the output of AND gate D26. The output of AND gate D28 is connected to the positive input of the voltage-controlled switch S7 in the learning control circuit II and one input of AND gate D30 in the experience module II. One input of AND gate D27 is connected to the food reward signal N3. The other input of AND gate D27 is connected to the output of AND gate D26. The output of AND gate D27 is connected to the gate of transistor T7 in the learning control circuit II and one input of AND gate D24 in the feedback module. One input of AND gate D26 is connected to the output of AND gate D43 in the hippocampus module II. The other input of AND gate D26 is connected to the output of AND gate D45 in the cerebellum module. The output of AND gate D26 is connected to the input of AND gate D28, the input of AND gate D27, and the positive input of the voltage-controlled switch S6 in the feedback receiving circuit II.

[0061] Feedback receiving circuit II includes a voltage-controlled switch S6, a resistor R22, a voltage source V8, and a subtractor SUB2. The positive input of voltage-controlled switch S6 is connected to the output of AND gate D26 in logic input circuit II, and the negative input of voltage-controlled switch S6 is grounded. The first contact of voltage-controlled switch S6 is connected to the output of subtractor SUB2, and the second contact of voltage-controlled switch S6 is connected to one input of adder SUM5 in output circuit II and one end of resistor R22. The other end of resistor R22 is grounded. The positive terminal of voltage source V8 is connected to the positive input of subtractor SUB2, and the negative terminal of voltage source V8 is grounded. The positive input of subtractor SUB2 is connected to the positive terminal of voltage source V8, and the negative input of subtractor SUB2 is connected to the positive terminal of diode D12 in empirical module II. The output of subtractor SUB2 is connected to the first contact of voltage-controlled switch S6.

[0062] The learning control circuit II includes voltage source V10, voltage source V9, NMOS transistor T7, resistors R24 and R23, and voltage-controlled switch S7. The positive terminal of voltage source V10 is connected to the first contact of voltage-controlled switch S7, and the negative terminal of voltage source V10 is grounded. The positive terminal of voltage source V9 is connected to the drain of transistor T7, and the negative terminal of voltage source V9 is grounded. The gate of transistor T7 is connected to the output of AND gate D27 in logic input circuit II, the drain of transistor T7 is connected to the positive terminal of voltage source V9, and the source of transistor T7 is connected to output circuit II. One input of adder SUM5 is connected to one end of resistor R24. The other end of resistor R24 ​​is grounded. One end of resistor R23 is grounded. The other end of resistor R23 is connected to the first contact of voltage-controlled switch S7. The positive input of voltage-controlled switch S7 is connected to the output of AND gate D28 in logic input circuit II. The negative input of voltage-controlled switch S7 is grounded. The first contact of voltage-controlled switch S7 is connected to the positive terminal of voltage source V10. The second contact of voltage-controlled switch S7 is connected to one end of resistor R23 and one input of adder SUM8 in empirical module II.

[0063] Output circuit II includes adders SUM5, SUM6, and SUM7, and voltage source V11. One input of adder SUM5 is connected to the first contact of voltage-controlled switch S6 in feedback receiving circuit II. The other input of adder SUM5 is connected to the source of transistor T7 in learning control circuit II. The output of adder SUM5 is connected to one input of adder SUM6. The other input of adder SUM6 is connected to the positive terminal of voltage source V11. The output of adder SUM6 is connected to one input of adder SUM7. The other input of adder SUM7 is connected to the first contact of voltage-controlled switch S4 in feedback module. The output of adder SUM7 is connected to the first contact of voltage-controlled switch S8 in experience module II. The positive terminal of voltage source V11 is connected to one input of SUM6, and the negative terminal of voltage source V11 is grounded.

[0064] like Figure 1 As shown, the experience module I includes an experience generation circuit I, an experience control circuit I, and an experience feedback circuit I. The input of the experience circuit I is connected to the output of the output circuit I of the voltage control module I. The output of the experience generation circuit I is connected to the input of the decision module, the feedback module, and the experience control circuit I. Another input of the experience control circuit I is connected to the output of the logic input circuit I and the learning control circuit I of the voltage control module I. The output of the experience control circuit I is connected to the experience feedback circuit I. The output of the experience feedback circuit I is connected to the subtractor SUB1 of the feedback receiving circuit I of the voltage control module I.

[0065] The experience generation circuit I includes a voltage-controlled switch S3, a memristor M2, an operational amplifier OP3, a resistor R13, and a mathematical operation unit ABM1. The positive input of the voltage-controlled switch S3 is connected to the output of the NAND gate D23 in the feedback module, and the negative input of the voltage-controlled switch S3 is grounded. The first contact of the voltage-controlled switch S3 is connected to the output of the adder SUM3 in the voltage control module. The second contact of the voltage-controlled switch S3 is connected to the positive terminal of the memristor M2 and the IN2 input of the operational unit ABM1. The negative terminal of the memristor M2 is connected to the negative input of the operational amplifier OP3 and one end of the resistor R13. The other end of the resistor R13 is connected to the output of the operational amplifier OP3 and the positive input of the operational amplifier OP2 in the experience control circuit I. The positive input of the operational amplifier OP3 is grounded. The negative input of the operational amplifier OP3 is connected to the negative terminal of the memristor M2 and one end of the resistor R13. The output of the operational amplifier OP3 is connected to the other end of the resistor R13 and the IN1 input of the operational unit ABM1. The IN2 input of BM1 is connected to the positive terminal of memristor M2, and the output of operational unit ABM1 is connected to one end of R16 in the feedback module and the negative input of operational amplifier OP7 in the decision module. Memristor M2 stores the weights of experience module I. Voltage-controlled switch S3 controls whether the control voltage of experience generation circuit I is transmitted. Operational amplifier OP3, memristor M2, and resistor R13 form an inverting amplifier circuit, expressed as Vout = -Vin*R13 / M2. The expression for mathematical operation unit ABM1 is Vout = -Vin1 / Vin2. Combining these two equations, the output of mathematical operation unit ABM1 is M2 / R13, which is the weight of memristor M2. This part of the circuit applies the output voltage from voltage control module I to memristor M1, and through the inverting amplifier formed by operational amplifier OP3, it reads the resistance value of memristor M1, i.e., the weight, in conjunction with mathematical operation unit ABM1. Simultaneously, voltage-controlled switch S3 receives signals from the feedback module, achieving a blocking effect.

[0066] The empirical control circuit I includes operational amplifier OP2, voltage source V5, voltage source V6, resistor R12, AND gate D22, NMOS transistor T4, and adder SUM4. The positive input of operational amplifier OP2 is connected to the output of operational amplifier OP3 in empirical generation circuit I, and the negative input of operational amplifier OP2 is connected to the positive terminal of voltage source V6. The output of operational amplifier OP2 is connected to one input of AND gate D22, and the other input of AND gate D22 is connected to the output of AND gate D19 in voltage control module I. The output of AND gate D22 is connected to... The gate of transistor T4 is connected to the voltage source V5. The drain of transistor T4 is connected to the positive terminal of voltage source V5. The source of transistor T4 is connected to one input of adder SUM4 and one end of resistor R12. The other end of resistor R12 is grounded. The negative terminals of voltage sources V5 and V6 are grounded. One input of adder SUM4 is connected to the source of transistor T4. The other input of adder SUM4 is connected to the first contact of voltage-controlled switch S2 in voltage control module I. The output of adder SUM4 is connected to the positive terminal of memristor M1 in empirical feedback circuit I. Operational amplifier OP2 is used as a voltage comparator. NMOS transistor T4 is used to control the learning voltage V5. If transistor T4 is turned on, voltage V5 will be applied to memristor M1 through adder SUM4. This part implements the learning-forgetting voltage for the experience feedback memristor M1. When memristor M1 is in a forgetting state but has not yet completed forgetting, the output of operational amplifier OP3 will be greater than 1.2V, operational amplifier OP2 will generate an output, and the learning voltage on the voltage source V5 controlled by it will be applied to adder SUM4. If memristor M2 is in a learning state or has already forgotten, operational amplifier OP2 will not generate an output. The recovery voltage of memristor M1 generated by voltage control module I will be transmitted to the memristor through adder SUM4.

[0067] The experience feedback circuit I includes memristor M1, operational amplifier OP1, resistors R14 and R15, and diode D6. The positive terminal of memristor M2 is connected to the output of adder SUM4 in experience control circuit I. The negative terminal of memristor M2 is connected to the negative input of operational amplifier OP1 and one end of resistor R14. The other end of resistor R14 is connected to the output of operational amplifier OP1. The positive input of operational amplifier OP1 is grounded. The output of operational amplifier OP1 is connected to the negative terminal of diode D6. The positive terminal of diode D6 is connected to the negative input of subtractor SUB1 in voltage control module I and one end of resistor R15. The other end of resistor R15 is grounded. Operational amplifier OP1, memristor M1, and resistor R14 form an inverting amplifier circuit. Its output expression is Vout = -Vin * R14 / M1, meaning the smaller the memristor M1, the smaller the output voltage. When memristor M1 is in learning mode, the input is positive and the output is negative. The output feedback signal can be transmitted to voltage control module I through diode D6. When the output is a positive voltage, diode D6 conducts unidirectionally, and the feedback signal cannot be transmitted. This part implements the output section of the experience feedback. When in a state of forgetting, the resistance of memristor M1 decreases; when completely forgotten, the resistance of memristor M1 begins to rise again. Its weighted performance is output to voltage control module I through operational amplifier OP1 of the inverting amplifier circuit, realizing training in the early stage of forgetting, which can achieve the effect of long-term memory.

[0068] When the voltage control module I outputs the learning voltage, it is applied to the memristor M2 via the voltage-controlled switch S3. After passing through the inverting amplifier composed of operational amplifiers OP3, the output voltage is the input voltage multiplied by the reciprocal of the memristor's resistance value. Then, through the mathematical operation unit ABM1, the resistance value of memristor M2 is obtained by dividing the input voltage of memristor M2 by the output voltage of the inverting amplifier. When a forgetting voltage is input, the operational amplifier OP3 of the inverting amplifier outputs a positive voltage. When the resistance value of memristor M2 is not greater than 3.3KΩ, the output voltage of operational amplifier OP3 is greater than 1.2V, which is greater than the voltage source V6. At this time, the operational amplifier OP2, used for judgment, outputs a high level. This high level is then combined with the forgetting signal through AND gate D22 for judgment, ultimately controlling the feedback learning voltage V5. This voltage then passes through transistor T4 and enters adder SUM4, ultimately acting on the experience feedback memristor M1. The voltage is then output from operational amplifier OP1 to the voltage control circuit I through the inverting amplifier circuit.

[0069] Experience Module II includes Experience Generation Circuit II, Experience Control Circuit II, and Experience Feedback Circuit II. The input of Experience Generation Circuit II is connected to the output of Output Circuit II of Voltage Control Module II. The output of Experience Generation Circuit II is connected to the input of Decision Module, Feedback Module, and Experience Control Circuit II. Another input of Experience Control Circuit II is connected to the output of Logic Input Circuit II and Learning Control Circuit II of Voltage Control Module II. The output of Experience Control Circuit II is connected to Experience Feedback Circuit II. The output of Experience Feedback Circuit II is connected to the input of Subtractor of Feedback Receiver Circuit II of Voltage Control Module II.

[0070] The experience generation circuit II includes a voltage-controlled switch S8, a memristor M3, an operational amplifier OP4, a resistor R27, and a mathematical operation unit ABM2. The positive input of the voltage-controlled switch S8 is connected to the output of the NAND gate D25 in the feedback module, and the negative input of the voltage-controlled switch S8 is grounded. The first contact of the voltage-controlled switch S8 is connected to the output of the adder SUM7 in the voltage control module. The second contact of the voltage-controlled switch S8 is connected to the positive terminal of the memristor M3 and the IN2 input of the operational unit ABM2. The negative terminal of the memristor M3 is connected to the negative input of the operational amplifier OP4 and one end of the resistor R27. The resistor R27... The other end is connected to the output of operational amplifier OP4 and the positive input of operational amplifier OP5 in empirical control circuit II. The positive input of operational amplifier OP4 is grounded. The negative input of operational amplifier OP4 is connected to the negative terminal of memristor M3 and one end of resistor R27. The output of operational amplifier OP4 is connected to the other end of resistor R27 and the IN1 input of operational unit ABM2. The IN2 input of operational unit ABM2 is connected to the positive terminal of memristor M3. The output of operational unit ABM2 is connected to one end of R21 in the feedback module and the negative input of operational amplifier OP9 in the decision module.

[0071] The experience control circuit II includes operational amplifier OP5, voltage source V12, voltage source V13, resistor R25, AND gate D30, NMOS transistor T8, and adder SUM8. The positive input of operational amplifier OP5 is connected to the output of operational amplifier OP4 in experience generation circuit II, and the negative input of operational amplifier OP5 is connected to the positive terminal of voltage source V13. The output of operational amplifier OP5 is connected to one input of AND gate D30, and the other input of AND gate D30 is connected to the output of AND gate D28 in voltage control module II. The output of AND gate D30 is connected to... The gate of transistor T8 is connected to the gate of transistor T8. The drain of transistor T8 is connected to the positive terminal of voltage source V12. The source of transistor T8 is connected to one input of adder SUM8 and one end of resistor R25. The other end of resistor R25 is grounded. The negative terminal of voltage source V12 is grounded. The negative terminal of voltage source V13 is grounded. One input of adder SUM8 is connected to the source of transistor T8. The other input of adder SUM8 is connected to the first contact of voltage-controlled switch S7 in voltage control module II. The output of adder SUM8 is connected to the positive terminal of memristor M4 in empirical feedback circuit II.

[0072] The experience feedback circuit II includes a memristor M4, an operational amplifier OP6, resistors R26 and R28, and a diode D12. The positive terminal of the memristor M4 is connected to the output of the adder SUM8 in the experience control circuit II. The negative terminal of the memristor M4 is connected to the negative input of the operational amplifier OP6 and one end of the resistor R26. The other end of the resistor R26 is connected to the output of the operational amplifier OP6. The positive input of the operational amplifier OP6 is grounded. The output of the operational amplifier OP6 is connected to the negative terminal of the diode D12. The positive terminal of the diode D12 is connected to the negative input of the subtractor SUB2 in the voltage control module II and one end of the resistor R28. The other end of the resistor R28 is grounded.

[0073] like Figure 1 As shown, the feedback module includes feedback circuit I, feedback circuit II, and an empirical feedback circuit. One input of feedback circuit I is connected to the output of AND gate D20 in voltage control module I, and the other input of feedback circuit I is connected to the positive terminal of diode D6 in empirical module I. The output of feedback circuit I is connected to one input of adder SUM7 in voltage control module II. One input of feedback circuit II is connected to the output of AND gate D27 in voltage control module II, and the other input of feedback circuit II is connected to the positive terminal of diode D12 in empirical module II. The output of feedback circuit II is connected to one input of adder SUM3 in voltage control module I. The output of the decision module, the output of the logic input circuits of voltage control module I and voltage control module II are all connected to the input of the empirical feedback circuit. The output of the empirical feedback circuit is connected to the other input of feedback circuit I and feedback circuit II, respectively. The output of the empirical feedback circuit is also connected to the input of the empirical generation circuit of empirical module I and empirical module II, respectively.

[0074] Feedback circuit I includes resistors R16 and R17, operational amplifier OP10, voltage-controlled switch S4, NAND gate D23, and AND gate D24. One end of resistor R16 is connected to the output of the mathematical operation unit ABM1, and the other end is connected to the negative input of operational amplifier OP10. The positive input of operational amplifier OP10 is grounded. The output of operational amplifier OP10 is connected to the first contact of voltage-controlled switch S4 and one end of resistor R17. The other end of resistor R17 is connected to the negative input of operational amplifier OP10. One end of voltage-controlled switch S4 is connected to the output of operational amplifier OP10, and the other end of voltage-controlled switch S4... Connect one input of adder SUM7 in voltage control module II and one end of resistor R18. The other end of resistor R18 is grounded. The negative input of voltage-controlled switch S4 is grounded. The positive input of voltage-controlled switch S4 is connected to the output of AND gate D24. One input of AND gate D24 is connected to the output of AND gate D20 in voltage control module I. The other input of AND gate D24 is connected to the output of AND gate D27 in voltage control module II. The output of NAND gate D23 is connected to the positive input of voltage-controlled switch S3 in empirical module I. One input of NAND gate D23 is connected to the output of AND gate D24. The other input of NAND gate D23 is connected to the output of operational amplifier OP7 in decision module II. Operational amplifier OP10 is used as a proportional amplifier circuit to adjust the magnitude of the feedback signal. Voltage-controlled switch S4 is responsible for whether to transmit the feedback signal to voltage control module II. When AND gate D24 outputs a common learning signal, voltage-controlled switch S4 is closed, and the feedback signal is applied to voltage control module II. This part of the circuit enables the memristor M2 to output its resistance value to the voltage-controlled switch S4 via the proportional amplifier circuit and operational amplifier OP10 when stimulation signals N1 and N2 appear simultaneously and the circuit is in a learning state. The voltage-controlled switch S4 then acts on the voltage control module II to suppress the second synapse.

[0075] Feedback circuit II includes resistors R21 and R20, operational amplifier OP11, voltage-controlled switch S5, and NAND gate D25. One end of resistor R21 is connected to the output of the mathematical operation unit ABM2, and the other end is connected to the negative input of operational amplifier OP11. The positive input of operational amplifier OP11 is grounded. The output of operational amplifier OP11 is connected to the first contact of voltage-controlled switch S5 and one end of resistor R20. The other end of resistor R20 is connected to the negative input of operational amplifier OP11. Voltage-controlled switch D25... One end of the voltage-controlled switch S5 is connected to the output of operational amplifier OP11. The other end of the voltage-controlled switch S5 is connected to one input of adder SUM3 in voltage control module I and one end of resistor R19. The other end of resistor R19 is grounded. The negative input of voltage-controlled switch S5 is grounded. The positive input of voltage-controlled switch S5 is connected to the output of AND gate D24. The output of NAND gate D25 is connected to the positive input of voltage-controlled switch S3 in empirical module II. One input of NAND gate D25 is connected to the output of AND gate D24. The other input of NAND gate D25 is connected to the output of operational amplifier OP9 in decision module.

[0076] When both synapses are in the learning state, both inputs and output of AND gate D24 are high, indicating mutual inhibition. The weights output from the mathematical operation unit ABM1 are amplified by a proportional amplifier and then connected to the adder SUM7 of the voltage control module II via voltage-controlled switch S4. This causes the learning voltage of the second synapse to increase, reducing its learning efficiency and thus achieving a masking effect. After synapse one completes learning first, the operational amplifier OP7 of the comparator outputs a signal, which is input to NAND gate D25, causing it to output a low level and opening voltage-controlled switch S8. Synapse two cannot continue learning and becomes blocked.

[0077] like Figure 1 As shown, the decision-making module includes delay circuit I, delay circuit II, comparator circuit I, comparator circuit II, trigger circuit, and logic output circuit. The input of delay module I is connected to signal source N1, and the output of delay module I is connected to the input of logic output circuit. The input of comparator circuit I is connected to the output of empirical module I, and the output of comparator circuit I is connected to the input of logic output circuit. The input of delay module II is connected to signal source N2, and the output of delay module II is connected to the input of logic output circuit. The input of comparator circuit II is connected to the output of empirical module II, and the output of comparator circuit II is connected to the input of logic output circuit. The output of trigger circuit is connected to the input of logic output circuit. The input of logic output circuit is connected to the outputs of delay circuit I, delay circuit II, comparator circuit I, comparator circuit II, and trigger circuit. Logic output circuit outputs action signal N5.

[0078] Delay circuit I includes capacitor C6, resistors R29 and R30, diode D9, NOT gate D31, and AND gate D32. One end of capacitor C6 is grounded, and the other end of capacitor C6 is connected to the cathode of diode D9, one end of resistor R29, and one input of AND gate D32. The anode of diode D9 is connected to signal source N1, and the other end of resistor R29 is grounded. The other input of AND gate D32 is connected to the output of NOT gate D31, and the input of NOT gate D31 is connected to signal source N1. The output of AND gate D32 is connected to one input of AND gate D33 in the logic output circuit and one end of resistor R30, the other end of which is grounded. When the input is a high-level signal, the signal charges capacitor C6 through diode D9. When the signal source becomes low-level, diode D9 prevents capacitor C6 from discharging from the input, and capacitor C6 discharges through resistor R29. The values ​​of resistor R29 and capacitor C6 determine the discharge time, i.e., the delay duration. Resistor R30 is a pull-down resistor. The function of delay circuit I is to synchronize the stimulus signal N1 with the output of experience module I. The stimulus signal N1 charges capacitor C6 through diode D9. Simultaneously, when the stimulus signal N1 is present, the second input of AND gate D32 is at a high level. The first input of AND gate D32 is connected to the stimulus signal N1 through NOT gate D31. When the stimulus signal N1 is present, AND gate D32 does not produce an output. When the stimulus signal N1 disappears, capacitor C6 begins to discharge, and AND gate D32 begins to produce an output. The duration of the output is determined by capacitor C6 and resistor R29.

[0079] Delay circuit II includes capacitor C8, resistors R35 and R36, diode D15, NOT gate D40, and AND gate D41. One end of capacitor C8 is grounded, and the other end of capacitor C8 is connected to the cathode of diode D15, one end of resistor R35, and one input of AND gate D41. The anode of diode D15 is connected to signal source N2, and the other end of resistor R35 is grounded. The other input of AND gate D41 is connected to the output of NOT gate D40, and the input of NOT gate D40 is connected to signal source N2. The output of AND gate D41 is connected to one input of AND gate D39 in the logic output circuit and one end of resistor R36, and the other end of resistor R36 is grounded.

[0080] Comparator circuit I includes diodes D7 and D8, voltage sources V7 and V15, and operational amplifier OP7. The anode of diode D8 is grounded, and its cathode is connected to the negative input of operational amplifier OP7 and the anode of diode D7. The cathode of diode D7 is connected to the anode of voltage source V7, and the cathode of voltage source V7 is grounded. The anode of voltage source V15 is connected to the positive input of operational amplifier OP7, and its cathode is grounded. The output of operational amplifier OP7 is connected to one input of AND gate D33 in the logic output circuit and one input of NAND gate D23 in the feedback module. The function of this circuit is to determine whether the synaptic circuit has reached the learning stage. Voltage sources V7, diodes D7 and D8, and the output signal of ABM1 form a clamping circuit to prevent the ABM module from generating an output exceeding the limit due to excessively low input voltage during the learning stop period. The subsequent operational amplifier OP7 and voltage source V15 form a voltage comparator circuit to determine whether the memristor weight has been learned.

[0081] Comparator circuit II includes diodes D14 and D13, voltage sources V14 and V16, and operational amplifier OP9. The anode of diode D13 is grounded, and the cathode of diode D13 is connected to the negative input of operational amplifier OP9 and the anode of diode D14. The cathode of diode D14 is connected to the anode of voltage source V14, and the cathode of voltage source V14 is grounded. The anode of voltage source V16 is connected to the positive input of operational amplifier OP9, and the cathode of voltage source V16 is grounded. The output of operational amplifier OP9 is connected to one input of AND gate D39 in the logic output circuit and one input of NAND gate D25 in the feedback module.

[0082] The trigger circuit includes capacitor C7, resistors R31, R32, R33, and R34, diodes D10 and D11, and operational amplifier OP8. One end of capacitor C7 is grounded, and the other end of capacitor C7 is connected to the negative input of operational amplifier OP8, one end of resistor R31, one end of resistor R32, the other end of resistor R31 is connected to the negative terminal of diode D10, and the positive terminal of diode D10 is connected to the output of operational amplifier OP8. The other end of resistor R32 is connected to the positive terminal of diode D11, and the negative terminal of diode D11 is connected to the output of operational amplifier OP8. One end of resistor R33 is grounded, and the other end of resistor R33 is connected to the positive input of operational amplifier OP8 and one end of resistor R34, the other end of resistor R34 is connected to the output of operational amplifier OP8. The output of operational amplifier OP8 is connected to one input of AND gate D36 and one input of AND gate D37 in the logic output circuit. Operational amplifier OP8 is used as a voltage comparator to compare the voltages of capacitor C7 and resistor R33. Capacitor C7 is used to adjust the output period, and diodes D10 and D11 control the charging and discharging of capacitor C7, respectively, to achieve adjustable duty cycle. The trigger circuit generates a periodic square wave signal, with resistor R31 controlling the high output time and resistor R32 controlling the low output time. The external stimulus signal is generally a random signal. This signal and the generated periodic signal are ANDed through AND gate D36 to produce a probabilistic output, the probability of which is determined by the duty cycle of the output square wave.

[0083] The operational amplifier OP8 operates at a voltage of +15V to -15V. When it first starts working, it is in a stable output state, with a voltage of +15V or -15V. At this time, the output of the operational amplifier OP8 will charge the capacitor C7 through resistor R31 or discharge the capacitor C7 through resistor R32. When the voltage of capacitor C7 is higher or lower than the positive input of the operational amplifier OP8, the output of the operational amplifier OP8 will flip once, thus forming a stable periodic signal.

[0084] The logic output circuit includes AND gates D33, D36, D37, D39, OR gates D34, D35, and D38. One input of AND gate D33 is connected to the output of AND gate D32 in delay circuit I, and the other input of AND gate D33 is connected to the output of operational amplifier OP7 in comparator circuit I. The output of AND gate D33 is connected to one input of OR gate D34. One input of AND gate D36 is connected to the output of AND gate D32 in delay circuit I, and the other input of AND gate D36 is connected to the output of operational amplifier OP8 in trigger circuit I. The output of AND gate D36 is connected to one input of OR gate D34. One input of AND gate D37 is connected to the output of AND gate D41 in delay circuit II. The other input of AND gate D37 is connected to the output of operational amplifier OP8 in the trigger circuit. The output of AND gate D37 is connected to one input of OR gate D38. One input of AND gate D39 is connected to the output of AND gate D41 in delay circuit II. The other input of AND gate D39 is connected to the output of operational amplifier OP9 in comparator circuit II. The output of AND gate D39 is connected to one input of OR gate D38. The output of OR gate D34 is connected to one input of OR gate D35. The output of OR gate D38 is connected to one input of OR gate D35. The output of OR gate D35 is the action signal. The function of this circuit is to output an action signal. When there is an external stimulus input, its corresponding synaptic circuit is in the learned state, and the circuit will generate the output action signal N5.

[0085] Specifically, this invention implements adaptive learning under low-reward conditions for operant conditioning. During normal learning, three signals, N1, N3, and N4, exist. When the food stimulus signal N3 disappears, the circuit enters an active forgetting state. The AND gate D19 in voltage control module I outputs a signal, and the recovery voltage V1 of memristor M1 is applied to adder SUM4 via voltage-controlled switch S2. If the circuit has not completely forgotten (i.e., it can autonomously generate an action signal upon receiving external stimulus N1), the operational amplifier OP2 of the voltage comparator will output, correspondingly controlling the learning voltage V5 of memristor M1 to open. The resistance of memristor M1 decreases, and the amplitude of the feedback signal generated by the operational amplifier OP1 of the inverting amplifier increases. After subtraction by subtractor SUB1 and the active forgetting voltage V4, the active forgetting voltage decreases, meaning the active forgetting slows down. When the circuit is repeatedly put into a learning-forgetting state, the resistance of memristor M1 is trained to a relatively low value. At this time, the active forgetting voltage becomes very low, which means that even after the food stimulus signal N3 disappears, the circuit can still respond to the stimulus signal N1 for a long time. When the circuit is completely forgotten, the learning voltage V5 disappears, and the recovery voltage V1 continues to act on the memristor M1, causing the resistance of the memristor M1 to continue to rise.

[0086] Specifically, this invention implements the blocking effect in operant conditioning. Once synapse I learns, the hallmark output is a high-level output from operational amplifier OP7 of the voltage comparator. When both synapses begin learning, AND gates D20 and D27 output high levels, causing AND gate D24 in the feedback module to output a hallmark high level. The competing signal output from AND gate D24 and the connecting signal output from operational amplifier OP7 together cause NAND gate D23 to output a low level, opening voltage-controlled switch S8. This prevents memristor M3 of synapse II from receiving the learning voltage, thus stopping learning and completing the blocking effect of stimulus signal N1 on stimulus signal N2.

[0087] Specifically, this invention implements the masking effect in operant conditioning. When both synapses begin learning, AND gates D20 and D27 output a high level, causing AND gate D24 in the competing module to output a significant high level. Voltage-controlled switches S4 and S5 close. The weights of the two synapses output by mathematical operation units ABM1 and ABM2, after passing through a proportional amplifier circuit, and then through voltage-controlled switches S4 and S5, are finally connected to adders SUM3 and SUM7. This causes the learning voltage of these two synapses to increase, thereby reducing the learning rate and creating mutual inhibition, thus achieving the masking effect.

[0088] This invention trains dogs to associate sounds with actions by receiving external sound and motion signals, as well as food rewards. When a dog hears a sound or a person makes a specific action, the dog will sit down on its own. Figure 1 The two voltage signal sources N1 and N2 connected to the hippocampus module represent external sound and motion signal output sources when forming a blocking and competing operant conditioned reflex. Since dogs are naturally more sensitive to sound signals than motion signals, sound signals dominate in the competition between these two signals. If a dog sits after hearing a sound signal and is trained by a human and then receives a food reward, after a period of training, the dog will automatically sit upon hearing the sound even without human assistance. Figure 2 As shown in N5, positive feedback learning of operant conditioning is achieved. If a dog learns to sit voluntarily after hearing a sound, but does not receive a food reward after performing the correct action, the dog will eventually decouple the sound from sitting after several trials. Figure 2 As shown by signals N1, N3, and N5, active forgetting of operant conditioning is achieved. Once the dog has learned the sound and training is stopped, the association between the sound and "sit" will gradually disappear over time, and the resistance of memristor M2, representing the weight of this association, will gradually increase again. Figure 2 As shown in M2, natural forgetting is achieved. If, after learning, the dog is only given a sound signal without a food reward, it will stop responding after three responses to the sound, as shown in the example. Figure 3As shown in N5, if the dog is rewarded with food for a period of time after every three times, it can gradually adapt to the reduction in food. When the food stops appearing, it will develop greater tolerance to the sound and maintain a longer response time, achieving adaptive learning under low-reward conditions. Figure 3 As shown. If a dog receives both auditory and visual signals while being trained to sit, the auditory signal will have a competitive advantage in the test, and will therefore associate with the action output first. Figure 4 The first test showed that, however, compared to Figure 2 The separate learning of the sound signal took longer, demonstrating the mutual interference between the two signals. When the sound signal disappeared, the visual signal quickly associated with the action, achieving the masking effect of operant conditioning. Once the dog had learned the association between the sound signal and the action, attempting to learn sound and visual signals simultaneously would result in the visual signal learning being blocked by the sound signal, preventing it from proceeding. Figure 5 As shown in M3, when the sound signal disappears, the learning of M3 can continue, realizing the blocking effect of operant conditioning.

[0089] Figure 2 The simulation results demonstrate the learning and forgetting process of operant conditioning considering time intervals in this invention. (a) shows the voltage waveform of input signal N1, (b) shows the voltage waveform of input signal N3, (c) shows the voltage waveform of input signal N4, (d) shows the voltage waveform of output signal N5, and (e) shows the resistance change waveform of memristor M2. The first 32 seconds are the assisted learning stage, in which each 4-second period is a learning cycle. First, a sound signal appears, then an action signal appears, then a food signal appears, and finally there is a 1-second time interval, completing one cycle of learning. The resistance of memristor M2 represents the degree of connection between the sound signal and the output action. The smaller the resistance of memristor M2, the stronger the connection. When the resistance of memristor M2 is small enough, it can actively output the action signal of sitting down without external assistance, realizing the learning process of operant conditioning. Once the dog has learned the association between sound and action, even after the food signal is removed, the dog will still sit down upon hearing the sound. However, the resistance of memristor M2 will increase. After five cycles, the dog will no longer respond to the sound signal, thus achieving the forgetting process of operant conditioning.

[0090] Figure 3The simulation results demonstrate the reinforcement learning process of the operant conditioning reflex under low-reward conditions, which is an adaptive learning method of the present invention. (a) shows the voltage waveform of the input signal N1, (b) shows the resistance change waveform of the memristor M2, (c) shows the voltage waveform of the input signal N3, (d) shows the voltage waveform of the output signal N5, and (e) shows the voltage waveform of the input signal N4. The first 40 seconds constitute the first learning process, during which the sound signal N1 just establishes a connection with the output signal N5. The first test is then performed, and after three outputs, no further output is generated to the sound signal N1. Then, four more learning cycles are performed, followed by an active forgetting test. N5 can output four waveforms. After another four learning cycles, N5 can output six waveforms, indicating that the active forgetting speed has slowed down. This is also evident from the changes in the memristor M2 storing the weights; the resistance recovery speed in the third test is slower than in the first two, reflecting the adaptive learning under low-reward conditions. The sound signal N1 and the output action signal N5 form a closer connection.

[0091] Figure 4 The simulation results demonstrate the learning process of the operant conditioning reflex that produces a masking effect when two stimuli are learned simultaneously, as presented in this invention. (a) shows the voltage waveform of input signal N3, (b) shows the voltage waveform of input signal N1, (c) shows the voltage waveform of input signal N2, (d) shows the voltage waveform of output signal N5, (e) shows the voltage waveform of input signal N4, and (f) shows the resistance change waveforms of memristors M2 and M3. The first 44 seconds represent the process of simultaneous learning of sound signal N1 and visual signal N2. The resistance changes of memristors M2 and M3 show that their decreasing speed is slower than when learning alone, with memristor M3 decreasing even slower than memristor M2. Therefore, in the tests at 44s and 48s, the sound signal N1 was associated with the output action N5, while the visual signal N2 was not associated with the action signal N5. The learning speed was still very slow during the simultaneous learning after the test. However, after the sound signal N1 was removed at 68s, the learning speed of the visual signal became very fast, realizing the occlusion effect and de-occlusion.

[0092] Figure 5The simulation results demonstrate the learning process of the operant conditioning that produces a blocking effect when the two stimuli are not learned simultaneously. (a) is the voltage waveform of the input signal N3, (b) is the voltage waveform of the input signal N1, (c) is the voltage waveform of the input signal N2, (d) is the voltage waveform of the output signal N5, (e) is the resistance change waveform of memristors M2 and M3, and (f) is the voltage waveform of the input signal N4. The first 32 seconds are the process of learning the sound signal alone. The output signal generated at the 29th second indicates that the sound signal N1 has established a connection with the output action signal N5. From the 32nd to the 64th second, the sound signal N1 and the visual signal N2 learn simultaneously. Judging from the resistance values ​​of memristors M2 and M3, it can be seen that the learning of memristor M3 has not been carried out. At the 64th second, a separate test of the visual signal N2 did not generate the output action signal N5, indicating that the learning of the visual signal N2 was blocked. After the 64th second, the sound signal N1 was removed, and the learning and forgetting of memristor M3 was able to proceed normally. Overall, the blocking and deblocking process of signal N1 on signal N2 was realized.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-functional operant conditioning neural network circuit with blocking and competition effects, characterized by, The hippocampus module, the cerebellum module, the voltage control module, the experience module and the decision module are connected with the input end of the hippocampus module, one excitation signal and the output signal are connected with the input end of the cerebellum module, another excitation signal, the output end of the hippocampus module and the output end of the cerebellum module are connected with the input end of the voltage control module, the output end of the voltage control module is connected with the input end of the experience module and one input end of the feedback module, the output end of the experience module is connected with the input end of the decision module and another input end of the feedback module, the output end of the feedback module is connected with the input end of the voltage control module and the input end of the experience module, the input end of the decision module is connected with the excitation signal, one output end of the decision module is connected with one input end of the feedback module, and another output end of the decision module obtains the output signal; The hippocampus module realizes receiving and time-delay transmitting of the external excitation signal; The cerebellum module realizes sensing and time-delay transmitting of the action signal; the voltage control module realizes voltage control of the experience module, including receiving various feedback voltages to control learning and forgetting of the experience module; the experience module is responsible for storing and updating the learned weight value to realize learning, forgetting and reinforcement learning; The feedback module is responsible for realizing feedback among multiple experience modules to realize masking and blocking effect; the decision module realizes weighting the weight values of multiple experience modules and judging whether to output the action signal, and realizes probability response to the input signal.

2. The multi-functional operant conditioning neural network circuit blocking and competing effects of claim 1, wherein, The excitation signal includes pulse voltage signal N1 and pulse voltage signal N2, one excitation signal is action signal N4, another excitation signal is food signal N3, the hippocampus module includes hippocampus module I and hippocampus module II, the voltage control module includes voltage control module I and voltage control module II, the experience module includes experience module I and experience module II, the pulse voltage signal N1 is connected with the input end of the decision module and the input end of the hippocampus module I, the output end of the hippocampus module I, the food signal N3 and the output end of the cerebellum module are connected with the input end of the voltage control module I, the output end of the voltage control module I is connected with the input end of the experience module I, the output end of the experience module I is connected with the input end of the decision module, the input end of the voltage control module I and another input end of the feedback module, the pulse voltage signal N2 is connected with the input end of the decision module and the input end of the hippocampus module I, the output end of the hippocampus module I, the food signal N3 and the output end of the cerebellum module are connected with the input end of the voltage control module II, the output end of the voltage control module II is connected with the input end of the experience module II, the output end of the experience module II is connected with the input end of the decision module, the input end of the voltage control module II and another input end of the feedback module, the output end of the feedback module is connected with the input end of the voltage control module I, the input end of the voltage control module II, the input end of the experience module I and the input end of the experience module II, the output signal is connected with the input end of the cerebellum module through diode D46, and the action signal N4 is connected with the input end of the cerebellum module through diode D47.

3. The multi-functional operant conditioning neural network circuit blocking and competing effects of claim 2, wherein, The hippocampus module I and the hippocampus module II of the hippocampus module each comprise a first-stage charging circuit, a second-stage charging circuit and a logic judgment circuit, the pulse voltage signal N1 is connected with the input end of the first-stage charging circuit, the second-stage charging circuit and the logic judgment circuit respectively, the output end of the first-stage charging circuit is connected with the input end of the second-stage charging circuit, the output end of the second-stage charging circuit is connected with the logic judgment circuit, and the output end of the logic judgment circuit is connected with the input end of the voltage control module I or the voltage control module II.

4. The multi-functional operant conditioning neural network circuit blocking and competing effects according to claim 2 or 3, characterized in that, The cerebellum module comprises a resistor R4, a resistor R5, a capacitor C3, a diode D3, a NOT gate D44 and an AND gate D45, the output signal is connected with the anode of the diode D3 through the diode D46, and the action signal N4 is connected with the anode of the diode D3 through the diode D47, the output signal and the action signal N4 are connected with the input end of the NOT gate D44 through the diode D46 and the diode D47 respectively, the cathode of the diode D3 is connected with one end of the resistor R4, one end of the capacitor C3 and one input end of the AND gate D45 respectively, the output end of the NOT gate D44 is connected with the other input end of the AND gate D45, the output end of the AND gate D45 is connected with one end of the resistor R5 and the input end of the voltage control module I or the voltage control module II respectively, and the other end of the resistor R4, the other end of the resistor R5 and the other end of the capacitor C3 are grounded.

5. The multi-functional operant conditioning neural network circuit blocking and competing effects of claim 4, wherein, The voltage control module I and the voltage control module II each comprise a logic input circuit, a feedback receiving circuit, a learning control circuit and an output circuit I, the food signal N3, the output end of the cerebellum module and the output end of the hippocampus module I or the hippocampus module II are connected with the input end of the logic judgment circuit, the output end of the logic input circuit is connected with one input end of the feedback module, one input end of the feedback receiving circuit and the input end of the learning control circuit respectively, the other input end of the feedback receiving circuit is connected with one output end of the experience module I or the experience module II, the output end of the learning control circuit, the output end of the feedback receiving circuit and the output end of the feedback module are connected with the input end of the output circuit I, and the output end of the output circuit I is connected with the input end of the experience module I or the experience module II.

6. The multi-functional operant conditioning neural network circuit blocking and competing effects of claim 5, wherein, The experience module I and the experience module II each comprise an experience generating circuit, an experience control circuit and an experience feedback circuit, the output circuit of the voltage control module I or the voltage control module II is connected with the input end of the experience generating circuit, one output end of the experience generating circuit is connected with one input end of the experience control circuit, and the other output end of the experience generating circuit is connected with the feedback module and the decision module respectively, the other two input ends of the experience control circuit are connected with the output end of the logic input circuit and the output end of the learning control circuit respectively, the output end of the experience control circuit is connected with the input end of the experience feedback circuit, and the output end of the experience feedback circuit is connected with the other input end of the feedback receiving circuit.

7. The multi-functional operant conditioning neural network circuit blocking and competing effects of claim 6, wherein, The feedback module comprises feedback circuit I, feedback circuit II and experience feedback circuit, one input end of the feedback circuit I is connected with another output end of the experience generating circuit of the experience module I, and the output end of the feedback circuit I is connected with the input end of the output circuit I of the voltage control module I; One input end of the feedback circuit II is connected with another output end of the experience generating circuit of the experience module II, and the output end of the feedback circuit II is connected with the input end of the output circuit I of the voltage control module II; the output end of the decision module, the output end of the logic input circuit of the voltage control module I and the voltage control module II are all connected with the input end of the experience feedback circuit, and the output end of the experience feedback circuit is respectively connected with another input end of the feedback circuit I, the feedback circuit II, and the input end of the experience generating circuit of the experience module I and the experience module II.

8. The multi-functional operant conditioning neural network circuit blocking and competing effects of any of claims 5-7, wherein, The decision module comprises first delay circuit, second delay circuit, first comparison circuit, second comparison circuit, trigger circuit and logic output circuit, one input end of the first comparison circuit is connected with another output end of the experience generating circuit of the experience module I, one input end of the second comparison circuit is connected with another output end of the experience generating circuit of the experience module II, and the output end of the first comparison circuit and the output end of the second comparison circuit are all connected with the input end of the experience feedback circuit of the feedback module; pulse voltage signal N1 is connected with the input end of the first delay circuit, pulse voltage signal N2 is connected with the input end of the second delay circuit, the output end of the first delay circuit, the output end of the second delay circuit, the output end of the first comparison circuit, the output end of the second comparison circuit are all connected with the input end of the logic output circuit, the output end of the trigger circuit is connected with the input end of the logic circuit, and the output end of the logic output circuit outputs output signal.

9. The multi-functional operant conditioning neural network circuit blocking and competing effects of claim 8, wherein, The trigger circuit comprises capacitor C7, resistor R31, resistor R32, resistor R33, resistor R34, diode D10, diode D11 and operational amplifier OP8, one end of the capacitor C7 and one end of the resistor R33 are grounded, the other end of one end of the capacitor C7 is connected with the inverting input end of the operational amplifier OP8, one end of the resistor R31 and one end of the resistor R32 respectively, the other end of the resistor R31 is connected with the anode of the diode D10, the other end of the resistor R32 is connected with the anode of the diode D11, the other end of the resistor R33 is connected with one end of the resistor R34 and the non-inverting input end of the operational amplifier OP8 respectively, the cathode of the diode D10, the cathode of the diode D11 and the other end of the resistor R34 are all connected with the output end of the operational amplifier OP8, and the output end of the operational amplifier OP8 is connected with the logic output.

10. The multi-functional operant conditioning neural network circuit blocking and competing effects of claim 9, wherein, The first-stage charging circuit comprises a first resistor, a first capacitor, a first diode and a first PMOS transistor, the pulse voltage signal N1 is connected to the anode of the first diode and the gate of the first PMOS transistor respectively, the cathode of the first diode is connected to the source of the first PMOS transistor and one end of the first capacitor respectively, the drain of the first PMOS transistor is connected to the input end of the second-stage charging circuit and one end of the first resistor respectively, and the other end of the first capacitor and the other end of the first resistor are grounded; The second-stage charging circuit comprises a second resistor, a second capacitor, a second diode and a second PMOS transistor, the drain of the first PMOS transistor is connected to the gate of the second PMOS transistor, the pulse voltage signal N1 is connected to the anode of the second diode, the cathode of the second diode is connected to the source of the second PMOS transistor and one end of the second capacitor respectively, the drain of the second PMOS transistor is connected to the input end of the logic judgment circuit and one end of the second resistor respectively, and the other end of the second capacitor and the other end of the second resistor are grounded; The logic judgment circuit comprises a first NOT gate, a first AND gate and a third resistor, the pulse voltage signal N1 is connected to the input end of the first NOT gate, the output end of the first NOT gate and the drain of the second PMOS transistor are connected to the input end of the first AND gate respectively, the output end of the first AND gate is connected to one end of the third resistor and the input end of the voltage control module I or the voltage control module II respectively, and the other end of the third resistor is grounded; The logic input circuit comprises a second NOT gate, a second AND gate, a third AND gate and a fourth AND gate, the output end of the first AND gate of the logic judgment circuit of the hippocampus module I or the hippocampus module II and the output end of the AND gate D45 of the cerebellum module are connected to the input end of the fourth AND gate respectively, the output end of the fourth AND gate is connected to the input end of the feedback receiving circuit I and one input end of the third AND gate and one input end of the second AND gate respectively, the food signal N3 is connected to the input end of the second NOT gate and the other input end of the third AND gate respectively, the output end of the third AND gate is connected to one input end of the feedback module and the input end of the learning control circuit I respectively, the output end of the second NOT gate is connected to the other input end of the second AND gate, and the output end of the second AND gate is connected to the input end of the learning control circuit I and the input end of the experience module I or the experience module II respectively; The feedback receiving circuit comprises a first voltage-controlled switch, a fourth resistor, a first voltage source and a first subtractor, the positive input end of the first voltage-controlled switch is connected to the output end of the fourth AND gate, one output end of the experience module I or the experience module II and the positive electrode of the first voltage source are connected to the first subtractor, the output end of the first subtractor is connected to the first contact of the first voltage-controlled switch, the second contact of the first voltage-controlled switch is connected to the input end of the output circuit I and one end of the fourth resistor respectively, and the negative input end of the first voltage-controlled switch, the other end of the fourth resistor and the negative electrode of the first voltage source are grounded. The learning control circuit comprises a second voltage-controlled switch, a second voltage source, a third voltage source, a first NMOS transistor, a fifth resistor and a sixth resistor, an output end of a second AND gate is connected with a positive phase input end of the second voltage-controlled switch, a first contact of the second voltage-controlled switch is connected with a positive pole of the second voltage source, a second contact of the second voltage-controlled switch is connected with one end of the sixth resistor and an input end of the experience module I or the experience module II respectively, a positive pole of the third voltage source is connected with a drain of the first NMOS transistor, a gate of the first NMOS transistor is connected with an output end of a third AND gate, a source of the first NMOS transistor is connected with one end of the fifth resistor and an input end of an output circuit I respectively, a negative phase input end of the second voltage-controlled switch, the other end of the fifth resistor, the other end of the sixth resistor, a negative pole of the third voltage source and a negative pole of the second voltage source are grounded; The output circuit I comprises a first adder, a second adder and a third adder, input ends of the first adder are connected with the second contact of the first voltage-controlled switch and the source of the first NMOS transistor respectively, output ends of the first adder and a positive pole of a fourth voltage source are connected with two input ends of the second adder respectively, a negative pole of the fourth voltage source is grounded, an output end of the second adder and one output end of the experience module I or the experience module II are connected with two input ends of the third adder respectively, an output end of the third adder is connected with the input end of the experience module I or the experience module II; The experience generation circuit comprises a third voltage-controlled switch, a first memristor, a third operational amplifier, a seventh resistor and a mathematical operation unit, a positive phase input end of the third voltage-controlled switch is connected with an output end of a feedback module, a first contact of the third voltage-controlled switch is connected with the output end of the third adder of the output circuit I, an inverting input end of the third voltage-controlled switch is grounded, a second contact of the third voltage-controlled switch is connected with a positive pole of the first memristor and one input end of the mathematical operation unit respectively, a negative pole of the first memristor is connected with the inverting input end of the third operational amplifier and one end of the seventh resistor respectively, the other end of the seventh resistor is connected with an output end of the third operational amplifier, the positive phase input end of the third operational amplifier is grounded, the output end of the third operational amplifier is connected with the other input end of the mathematical operation unit and one input end of an experience control circuit respectively, an output end of the mathematical operation unit is connected with the other input end of the feedback module and an input end of a decision module respectively. The experience control circuit comprises a second operational amplifier, a fifth voltage source, a sixth voltage source, an eighth resistor, a fifth AND gate, a second NMOS transistor and a fourth adder, an output terminal of the third operational amplifier is connected with a non-inverting input terminal of the second operational amplifier, an inverting input terminal of the second operational amplifier is grounded, an output terminal of the second operational amplifier and an output terminal of the second AND gate of the logic input circuit are connected with input terminals of the fifth AND gate, an output terminal of the fifth AND gate is connected with a gate of the second NMOS transistor, a drain of the second NMOS transistor is connected with a positive electrode of the fifth voltage source, a source of the second NMOS transistor is connected with one end of the eighth resistor and one input terminal of the fourth adder, the other input terminal of the fourth adder is connected with a second contact of the second voltage-controlled switch of the learning control circuit, a negative electrode of the fifth voltage source and the other end of the eighth resistor are grounded, and an output terminal of the fourth adder is connected with an input terminal of the experience feedback circuit; The experience feedback circuit comprises a second memristor, a first operational amplifier, a ninth resistor, a tenth resistor and a diode I, the output terminal of the fourth adder is connected with a positive electrode of the second memristor, a negative electrode of the second memristor is connected with one end of the ninth resistor and an inverting input terminal of the first operational amplifier, a non-inverting input terminal of the first operational amplifier is grounded, the other end of the ninth resistor is connected with an output terminal of the first operational amplifier, the output terminal of the first operational amplifier is connected with a negative electrode of the diode I, a positive electrode of the diode is connected with one input terminal of the third adder of the feedback receiving circuit and one end of the tenth resistor, and the other end of the tenth resistor is grounded; The feedback circuit I and the feedback circuit II each comprise a fourth operational amplifier and a fourth voltage-controlled switch, a non-inverting input terminal of the fourth operational amplifier is grounded, an inverting input terminal of the fourth operational amplifier is connected with one end of an eleventh resistor and one end of a twelfth resistor, the other end of the eleventh resistor is connected with an output terminal of the mathematical operation unit of the experience generation circuit of the experience module I or the experience module II, the other end of the twelfth resistor is connected with an output terminal of the fourth operational amplifier, the output terminal of the fourth operational amplifier is connected with a first contact of the fourth voltage-controlled switch, a non-inverting input terminal of the fourth voltage-controlled switch is connected with one output terminal of the experience feedback circuit, an inverting input terminal of the fourth voltage-controlled switch is grounded, a second contact of the fourth voltage-controlled switch is connected with one end of a thirteenth resistor and one input terminal of the third adder of the output circuit I of the voltage control module I or the voltage control module II, and the other end of the thirteenth resistor is grounded; The experience feedback circuit comprises a sixth AND gate, a seventh AND gate and an eighth AND gate, the input terminals of the sixth AND gate are connected with the output terminals of the second AND gates of the logic input circuits of the voltage control module I or the voltage control module II, the output terminal of the sixth AND gate is connected with one input terminal of the seventh AND gate, one input terminal of the eighth AND gate, the non-inverting input terminals of the fourth voltage-controlled switches of the feedback circuit I and the feedback circuit II, the other input terminal of the seventh AND gate and the other input terminal of the eighth AND gate are connected with the output terminal of the decision module, The first delay circuit and the second delay circuit each comprise a third capacitor, a fourteenth resistor, a fifteenth resistor, a diode II, a third NOT gate and a ninth AND gate, the pulse voltage signal N1 or the pulse voltage signal N2 is connected to the anode of the diode II and the input end of the third NOT gate respectively, the output end of the third NOT gate is connected to one input end of the ninth AND gate, the cathode of the diode II is connected to the other input end of the ninth AND gate, one end of the third capacitor and one end of the fourteenth resistor respectively, the output end of the ninth AND gate is connected to one end of the fifteenth resistor and the input end of the logic output circuit respectively, and the other end of the third capacitor, the other end of the fourteenth resistor and the other end of the fifteenth resistor are grounded; The first comparison circuit and the second comparison circuit each comprise a diode III, a diode IV, a seventh voltage source, an eighth voltage source and a fifth operational amplifier, the output end of the mathematical operation unit of the experience generation circuit of the experience module I or the experience module II is connected to the inverting input end of the fifth operational amplifier, the anode of the diode III and the anode of the diode IV respectively, the non-inverting input end of the fifth operational amplifier is connected to the anode of the eighth voltage source, the cathode of the diode III, the cathode of the diode IV and the cathode of the eighth voltage source are grounded, and the output end of the fifth operational amplifier is connected to the input end of the logic output circuit and the other input end of the seventh AND gate or the other input end of the eighth AND gate of the experience feedback circuit of the feedback module respectively; The logic output circuit comprises an AND gate D33, an AND gate D36, an AND gate D37, an AND gate D39, an OR gate D34, an OR gate D35 and an OR gate D38, the output end of the ninth AND gate of the first delay circuit is connected to one input end of the AND gate D33 and one input end of the AND gate D36 respectively, the other input end of the AND gate D33 is connected to the output end of the fifth operational amplifier of the first comparison circuit, the other input end of the AND gate D36 is connected to the output end of the operational amplifier OP8 of the trigger circuit, the output end of the AND gate D33 and the output end of the AND gate D36 are connected to the input end of the OR gate D34; the output end of the ninth AND gate of the second delay circuit is connected to one input end of the AND gate D37 and one input end of the AND gate D39 respectively, the other input end of the AND gate D39 is connected to the output end of the fifth operational amplifier of the first comparison circuit, the other input end of the AND gate D37 is connected to the output end of the operational amplifier OP8 of the trigger circuit, the output end of the AND gate D37 and the output end of the AND gate D39 are connected to the input end of the OR gate D38; the output end of the OR gate D34 and the output end of the OR gate D38 are connected to the input end of the OR gate D35; and the output end of the OR gate D35 outputs the output signal N5.

Images