Multi-sensory mutual association memory network circuit based on cross-species bionics

Abstract

The invention belongs to the cross technical field of combination of neuromorphic engineering and bionic intelligence, and more specifically relates to a multi-sensory mutual association memory network circuit based on cross-species bionics. Comprising n bionic stimulation enhancement circuits and a multi-sensory interconnection memory circuit. Each enhancement circuit is used for judging whether an external sensing signal is an effective signal or not according to the strength of the signal, and cooperatively selecting one enhancement unit to enhance the effective signal; each of the two enhancement units comprises an operational amplifier unit which is used for enhancing and amplifying an external sense signal and comprises a memristor, the memristor determines an enhancement mode for the external sense signal, and parameters of the memristor respectively and correspondingly determine an enhancement mode for an effective signal; the two enhancement modes are determined by a processing mechanism of an animal bionic by the corresponding bionic stimulation enhancement circuit on an effective signal of the corresponding mode; the multi-sense interconnection idea memory circuit realizes interconnection idea between modes and retrieval of each mode through the n synaptic circuits and the n retrieval neuron circuits.

Description

TECHNICAL FIELD

[0001] The present application belongs to the cross technical field of neuromorphic engineering and bionic intelligence combined, more specifically, relates to a multi-sensory mutual associative memory network circuit based on cross-species bionics. BACKGROUND

[0002] With the evolution of artificial intelligence towards brain inspiration, neuromorphic engineering, with the core advantages of low power consumption and high interpretability, has become a key field to break through the bottleneck of traditional von Neumann architecture "storage-computation separation", and the integration of bionic intelligence technology further promotes it to simulate the complex perception and learning ability of living beings. As the core device of neuromorphic circuit, the resistance of memristor can be adjusted by the amount of charge flowing through it, and it has been widely used to build associative memory networks due to its dynamic adjustment characteristics of synaptic weight (such as simulating the long-term potentiation / inhibition of biological synapses). However, the existing technology still has significant shortcomings in the integration of cross-species bionic mechanisms and the realization of multi-sensory associative memory functions in two dimensions, making it difficult to adapt to dynamic perception and decision-making needs in complex scenarios.

[0003] In the aspect of cross-species bionics, the bionic design of existing neuromorphic circuits is mostly limited to a single species (such as only simulating the perception mechanism of humans or specific mammals), and the sensory processing advantages of different species are not fully tapped. In the biological world, different species have evolved efficient processing mechanisms for specific sensory modalities: for example, the horizontal cells and bipolar cells of pigeon retinas can construct peripheral visual enhancement circuits that can accurately capture dynamic visual signals in complex environments; the dendritic branching density of brown bear olfactory bulbs is about 5 times that of humans, supporting parallel and efficient processing of olfactory signals; dolphins transmit sound waves to the middle ear through the lower jaw, achieving directional enhancement and noise suppression of auditory information. However, existing circuits only use the perception logic of a single species, and cannot adapt to corresponding cross-species bionic enhancement strategies according to the characteristics of different sensory modalities such as vision, hearing, and olfaction. This leads to problems such as signal coding distortion, weak anti-interference ability, and poor adaptability of enhancement effect when facing non-standardized multi-sensory inputs (such as blurred flame vision in a fire scene, faint distress calls in a noisy environment, and low-concentration toxic gas olfactory signals), making it difficult to meet the perception needs in complex environments.

[0004] At the level of multisensory inter-associative memory network, there are several core limitations in the prior art: first, most multisensory associative circuits only realize one-way association (such as "visual signal -> auditory signal" single mapping), lack of bidirectional inter-associative ability between multi-modal signals, and cannot simulate the cross-associative memory of "visual-auditory-olfactory" in the biological brain (such as seeing the flame at the same time associated with the smoke smell and the cry for help); second, the integration of multisensory signals is insufficient, and the existing circuits mostly treat different sensory modalities as independent channels for processing, without building a unified inter-associative memory framework. When a certain sensory channel is disturbed (such as smoke shielding vision), the perception information cannot be supplemented through other channel signals (such as olfactory toxic gas signal, auditory cry for help signal), which further affects subsequent decision-making.

[0005] In actual application scenarios (such as fire rescue robots), the above defects are more prominent: rescue robots need to process three types of non-standardized signals: vision (flame location), hearing (survivor cry for help), and olfaction (toxic gas concentration). The existing circuits lack cross-species biomimetic enhancement and cannot effectively amplify weak cries for help (such as not borrowing the dolphin hearing enhancement mechanism) or suppress the interference of smoke on visual signals (such as not introducing the pigeon vision enhancement logic). At the same time, due to the lack of multisensory inter-associative function, the cross-memory association of "flame-cry for help-toxic gas" cannot be established, and when the visual signal is blocked by thick smoke, the survivor cannot be located through olfactory or auditory signals.

[0006] Therefore, it has become a key technical problem to be solved in the cross field of neuromorphic engineering and biomimetic intelligence to develop a multisensory inter-associative memory network circuit integrating cross-species biomimetic mechanisms SUMMARY In view of the above defects or improvement needs of the prior art, the present application provides a multisensory inter-associative memory network circuit based on cross-species biomimetics, which aims to solve the problems of signal coding distortion and weak anti-interference ability when non-standardized multisensory inputs are processed in complex scenarios, and to improve the processing ability of weak and disturbed signals, better meeting the complex environmental perception needs.

[0007] To achieve the above-mentioned purpose, according to one aspect of the present application, a multisensory inter-associative memory network circuit based on cross-species biomimetics is provided, comprising n one biomimetic stimulus enhancement circuit and one multisensory inter-associative memory circuit; n is an integer not less than 2; Each bionic stimulus enhancement circuit includes two neuron circuit control units and two external sensory signal enhancement units; the two control units are used to judge whether the received external sensory signal is a non-noise effective signal according to the signal strength, and to select one of the two enhancement units to enhance the external sensory signal judged as an effective signal; the circuit structures of the two enhancement units are the same, and each includes an operational amplifier unit containing a memristor for enhancing and amplifying the external sensory signal, wherein the memristor determines the enhancement mode of the external sensory signal, including the enhancement rate and the enhancement multiple, and the parameters of the memristors in the two enhancement units are different, corresponding to the enhancement modes of strong and weak effective signals, respectively, and the two enhancement modes are determined by the processing mechanism of the corresponding bionic animal to the corresponding modal effective signal. The multi-sensory interconnected memory circuit includes n n synaptic circuits and n n retrieval neuron circuits; one mode corresponds to one synaptic circuit and one retrieval neuron circuit, each synaptic circuit includes one forward branch and n-1 retrieval feedback branches, the n-1 retrieval feedback branches are used to receive the outputs of the retrieval neuron circuits of the remaining n-1 modes except the mode corresponding to the synaptic circuit one by one, and are connected to the corresponding outputs through a positive terminal and the memristors of the forward branch through a negative terminal, respectively, each output is multiplied by the weight of the corresponding memristor, the forward branch is used to add the products and the enhanced signal from the bionic stimulus enhancement circuit of the corresponding mode, and then transmit them to the corresponding retrieval neuron circuit, which simulates the retrieval of the human brain to the enhanced signal, and outputs the retrieval signal of the corresponding mode; each two synaptic circuits also share an associative learning branch, one end is used to receive the enhanced signals from the bionic stimulus enhancement circuits of the two corresponding modes, and the other end is connected to the positive terminals of the two memristors in the two synaptic circuits, which is used to learn the bidirectional association between the two modes through the memristors when the two corresponding modes have enhanced signal inputs and store it in the form of memristor weight.

[0008] Further, the two neuron circuit control units are respectively denoted as a first neuron circuit control unit and a second neuron circuit control unit, and the two external sensory signal enhancement units are respectively denoted as a first external sensory signal enhancement unit and a second external sensory signal enhancement unit; each external sensory signal enhancement unit uses a transmission gate circuit as an input port to transmit the non-noise effective signal to the operational amplifier unit; The first neuron circuit control unit includes a first comparator switch and an AND gate The second neuron circuit control unit includes a second comparator switch and an inverter The control end opening threshold voltages of the first comparator switch and the second comparator switch are different; the output of the second comparator switch is connected to the inverter the input of the first comparator switch and the output of the inverter the output of the second comparator switch and the input of the transmission gate circuit in the second external sensory signal enhancement unit the output of the second comparator switch and the input of the transmission gate circuit in the second external sensory signal enhancement unit the output of the second comparator switch and the input of the transmission gate circuit in the second external sensory signal enhancement unit The control end opening threshold voltage of the first comparator switch and the second comparator switch is set by bionics, which cooperatively determines whether the first external sensory signal enhancement unit and the second external sensory signal enhancement unit are opened under the current external sensory signal input.

[0009] Further, the first comparator switch and the second comparator switch are the same in structure, and each includes one NMOS tube and one PMOS tube. The gate of the NMOS tube is the input end of the comparator switch, the drain of the NMOS tube is connected with the gate of the PMOS tube, and the drain of the PMOS tube is the output end of the comparator switch. The drain of the NMOS tube and the drain of the PMOS tube are connected with the ground through a grounding resistor. The source of the NMOS tube and the source of the PMOS tube are respectively connected with one preset DC voltage source. The first comparator switch and the second comparator switch totally include four preset DC voltage sources, and the voltages of the four preset DC voltage sources are different, which meet the requirement that the corresponding bionic stimulation enhancement circuit can imitate the corresponding bionic animal to distinguish the strength of the corresponding modal external sensory signal, filter out noise signals, and perform subsequent processing on non-noise effective signals.

[0010] Further, the input end of the memristor in the first external sensory signal enhancement unit and the input end of the memristor in the second external sensory signal enhancement unit are respectively connected with the source of one PMOS tube. The drain of the PMOS tube is connected with a reset voltage, and the gate of the PMOS tube is connected with the input end of the transmission gate circuit in the corresponding enhancement unit. When the signal received by the transmission gate circuit from the comparator switch is at a low level, the PMOS tube is opened, and the resistance state of the memristor in the corresponding enhancement unit is reset through the reset voltage.

[0011] Further, each associated learning branch includes two inverters, two memristors, and two PMOS tubes in cascade. The gates of the two PMOS tubes are respectively connected with the outputs of the two inverters. The source of one PMOS tube is connected with an associative learning voltage The drain of the other PMOS tube is connected with two memristors for learning and storing the association weight between two inter-associated modalities. When the two synaptic circuits corresponding to the two inter-associated modalities have the corresponding modal enhancement signal input, the two PMOS tubes in cascade are turned on through the reverse of the inverter, and the associative learning voltage The positive ends of the two memristors in the feedback branches of the two corresponding synapse circuits are applied by two PMOS transistors, the resistance states of the two memristors are both decreased, and the corresponding amplification weights are both increased, thereby realizing associative learning.

[0012] Further, the parameters of all the memristors in the multi-sensory inter-associative memory circuit are the same.

[0013] Further, a proportional operational amplifier is integrated in each retrieval neuron circuit to adjust the amplitude of the output signal of the synapse circuit, and then the corresponding retrieval signal is generated after the comparison circuit as the retrieval neuron channel, wherein if the enhanced signal after amplitude adjustment reaches the threshold voltage of the retrieval neuron channel in the retrieval neuron circuit, the retrieval signal is activated, otherwise the retrieval signal cannot be activated.

[0014] The application also provides a robot comprising the multi-sensory inter-associative memory network circuit as described above.

[0015] Overall, compared with the prior art, the technical scheme provided by the application has the following beneficial effects: 1. The application provides a multi-sensory inter-associative memory network circuit based on cross-species bionics, comprising n a bionic stimulus enhancement circuit and a multi-sensory inter-associative memory circuit, the design of the bionic stimulus enhancement circuit breaks through the limitation of single-species bionics, and can integrate the advantages of cross-species sensory processing such as pigeon vision, brown bear olfaction, and dolphin hearing, adapt personalized cross-species bionic enhancement strategies for different modal sensory signals such as vision, hearing, and olfaction, effectively solve the problems of signal coding distortion and weak anti-interference ability when non-standardized multi-sensory input in complex scenes, and improve the processing ability of weak and disturbed signals, better meet the complex environment perception demand. The design of the multi-sensory inter-associative memory circuit realizes bidirectional inter-associative memory between multi-sensory modalities, builds a unified multi-sensory inter-associative memory framework, is no longer limited to one-way mapping, can simulate the cross-association memory of the biological brain "vision-hearing-olfaction", and when a certain sensory channel is disturbed, the perception information can be supplemented through other channel signals to ensure the accuracy of subsequent decision-making. Therefore, the application overcomes the defects of the prior art in cross-species bionic mechanism integration, multi-sensory inter-associative memory function implementation, and dynamic scene adaptability, realizes personalized enhancement mode of multi-sensory signal channel, and simultaneously realizes bidirectional association and enhancement between multiple sensory modalities. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A multi-sensory inter-associative memory network circuit structure based on cross-species bionics is provided for the embodiments of the application. Figure 2A specific circuit implementation diagram of cross-species biomimetic stimulation enhancement provided for the embodiment of the present application is shown in the figure; Figure 3 A circuit implementation diagram of an associative memory network under three-modal input provided for the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0018] Embodiment one A multisensory inter-associative memory network circuit based on cross-species biomimetics, comprising n one biomimetic stimulation enhancement circuit and one multisensory inter-associative memory circuit; n is an integer not less than 2; Each biomimetic stimulation enhancement circuit includes two neuron circuit control units and two external sensory signal enhancement units; the two control units are used to judge whether the received external sensory signal is a non-noise effective signal according to the signal strength, and to select one of the enhancement units to enhance the external sensory signal judged as an effective signal; the circuit structures of the two enhancement units are the same, and each includes an operational amplifier unit containing a memristor for enhancing and amplifying the external sensory signal, wherein the memristor determines the enhancement mode of the external sensory signal containing the enhancement rate and the enhancement multiple, the memristor parameters in the two enhancement units are different, and correspond to the enhancement mode of strong and weak effective signals respectively, and the two enhancement modes are determined by the processing mechanism of the corresponding modal effective signal of the corresponding biomimetic stimulation enhancement circuit of the animal being mimicked; The multisensory inter-associative memory circuit includes n one synapse circuit and none search neuron circuit; one synapse circuit and one search neuron circuit for each modality, each synapse circuit including one forward branch and n-1 search feedback branches, the n-1 search feedback branches being used to receive the outputs of the search neuron circuits of the remaining n-1 modalities one by one, and connecting the corresponding outputs to the positive terminals of the memristors and the negative terminals of the forward branch respectively, each output being multiplied by the weight of the corresponding memristor, the forward branch being used to add the enhancement signals received from the corresponding modality of the biomimetic stimulus enhancement circuit and the products, and then transmitting them to the corresponding search neuron circuit, and outputting the search signals of the corresponding modality through the search neuron circuit to simulate the search of the human brain on the enhancement signals; each two synapse circuits also share an associative learning branch, one end of which is used to receive the enhancement signals from the biomimetic stimulus enhancement circuits of the two corresponding modalities, and the other end is connected to the positive terminals of the two memristors in the two synapse circuits, which is used to learn the bidirectional association between the two modalities through the memristors when the two corresponding modalities have enhancement signals input, and store the use in the form of the weight of the memristor.

[0019] Figure 1 is the structural diagram of the multi-sensory inter-association memory network circuit based on cross-species biomimetics designed in the embodiment. As shown in Figure 1 , the circuit includes three parts of biomimetic stimulus enhancement circuit, synapse circuit and search neuron circuit.

[0020] In Figure 1In this embodiment, the n bionic stimulation enhancement circuits first process the external sensory signals of each modality (such as auditory sound waves and visual images) and convert them into pulse enhancement signals through cross-species bionic mechanisms (such as simulating the noise reduction of dolphin hearing and the visual enhancement of pigeons). The signals are transmitted in two ways: one is sent to the logic control part (i.e., the associative learning branch), which provides the basis for regulation; the other is transmitted to the corresponding synaptic circuit as input for weight calculation. The logic control part generates a control signal based on the pulse enhancement signal through built-in logic operations. This signal directly acts on the learning voltage switch of the memristor in the synaptic circuit. When the switch is on, the learning voltage is applied to the memristor, which adjusts the synaptic weight by changing its resistance. When the switch is off, the weight remains stable. The mth synaptic circuit integrates two types of signals: stored signals of the same modality and retrieval signals from the other j-1 retrieval neuron circuits. After multiplying these two types of signals by the corresponding synaptic weights, the sum signal is output to the retrieval neuron circuit. The retrieval neuron circuit calculates the signal product sum based on Kirchhoff's law. If the value exceeds the activation threshold, a high-level retrieval signal is output and fed back to the other j synaptic circuits. When the kth and jth modality sensory information is input simultaneously, the resistance of the memristor in the kth row and jth column and the jth row and kth column of the synaptic array decreases, and the corresponding synaptic weight increases, establishing a strong association between the two modalities. After continuous learning, only the kth modality information is input, and the kth and jth modality information can be retrieved simultaneously, realizing cross-modality associative thinking.

[0021] In this embodiment, the single-species bionic limitation is broken, and based on the enhancement circuit design, multiple species sensory processing advantages can be integrated, such as pigeon vision, brown bear olfaction, dolphin hearing, and other cross-species sensory processing advantages. For different modality sensory signals such as vision, hearing, and olfaction, personalized cross-species bionic enhancement strategies are adapted to effectively solve the problems of signal coding distortion and weak anti-interference ability in non-standardized multi-sensory input in complex scenarios, improve the processing ability of weak and disturbed signals, and better meet the complex environmental perception needs.

[0022] In addition, in the circuit design of the embodiment, each two synapse circuits also share an associative learning branch, and the on-off management of the voltage control switch of the internal memristor is performed by the signal generated by the logical operation of any two enhanced external sensory signals, so that the resistance state of the memristor is dynamically changed, and then the precise adjustment of the synaptic weight is realized; the storage signal of the present mode and the other modal retrieval signal obtained by the feedback input are respectively multiplied by the adjusted corresponding synaptic weight, and finally all the multiplication results are accumulated and processed, and the summed electrical signal is output to the retrieval neuron circuit, realizing the bidirectional interconnection of the associative memory among multiple sensory modalities, constructing a unified multi-sensory interconnection memory framework, and no longer being limited to one-way mapping, which can simulate the cross-association memory of "visual - auditory - olfactory" of the biological brain, and when a certain sensory pathway is disturbed, the perception information can be supplemented through other pathway signals to ensure the accuracy of subsequent decision-making.

[0023] The embodiment learns from the cross-species perception advantages of pigeon visual enhancement, brown bear olfactory parallel processing, dolphin auditory directional noise reduction, and combines the dynamic adjustment characteristics of the synapse-like weight of the memristor, to simulate the individualized enhancement and coding of non-standardized multi-sensory signals (such as fuzzy vision, weak hearing, and low-concentration olfactory signals) of different species, and to convert the disturbed original sensory signal into a stable pulse signal suitable for neural morphological processing. In the synapse circuit, the resistance of multiple types of memristors (such as sulfide-based memristors suitable for olfaction and hafnium oxide-based memristors suitable for hearing) will be adjusted in real time online according to the correlation strength difference of cross-species multi-sensory signals, that is, the weight parameters of different sensory modalities are dynamically adapted during the operation of the circuit. The enhanced signal of the same sensory pathway can directly activate the retrieval signal of the present mode, and the enhanced signal of different sensory pathways can activate the associated retrieval signal of other pathways through the cross-modal feedback link of the retrieval signal, to realize the bidirectional interconnection of multiple modalities. Since the circuit adopts modular design, each bionic stimulus enhancement and association module can be independently expanded, so a large-scale associative memory network of multiple species and multiple senses can be constructed, realizing the complex functions of associative memory acquisition, extinction, cross-modal transmission and long-term consolidation, and applied to multi-signal association of fire rescue robots, cross-species bionic AI scenario memory and other scenes. At the same time, the synapse-like unit is constructed with the memristor as the core in the circuit design, avoiding the use of large-area and high-power traditional operation devices, which has a significant advantage in chip area occupation and energy consumption.

[0024] As a preferred embodiment, the two neuron circuit control units are respectively denoted as a first neuron circuit control unit and a second neuron circuit control unit, and the two external sensory signal enhancement units are respectively denoted as a first external sensory signal enhancement unit and a second external sensory signal enhancement unit; each external sensory signal enhancement unit uses a transmission gate circuit as an input port to transmit a non-noise effective signal to an operational amplifier unit; The first neuron circuit control unit includes a first comparator switch and an AND gate. The second neuron circuit control unit includes a second comparator switch and an inverter. The control terminals of the first and second comparator switches have different threshold voltages; the output of the second comparator switch is connected to an inverter. The input of the second external sensory signal enhancement unit and the input of the transmission gate circuit; AND gate The two inputs are the output of the first comparator switch and the inverter. The output of the gate The output is connected to the transmission gate circuit in the first external sensory signal enhancement unit; By setting the control terminals of the first and second comparator switches with biomimetic settings to open the threshold voltage, the system collaboratively determines whether the first and second external sensor signal enhancement units are turned on or off under the current external sensor signal input.

[0025] As a further preferred embodiment, the first comparator switch and the second comparator switch have the same structure, both including: one NMOS transistor and one PMOS transistor. The gate of the NMOS transistor is the input terminal of the comparator switch, the drain of the NMOS transistor is connected to the gate of the PMOS transistor, and the drain of the PMOS transistor is the output terminal of the comparator switch. The drains of the NMOS transistor and the PMOS transistor are grounded through a grounding resistor. The sources of the NMOS transistor and the PMOS transistor are respectively connected to a preset DC voltage source. The first comparator switch and the second comparator switch together contain four preset DC voltage sources. The four preset DC voltage sources have different voltage magnitudes, which satisfies the following: the corresponding bionic stimulus enhancement circuit can imitate the corresponding bionic animal's distinction between strong and weak external sensory signals of the corresponding modality, filter out noise signals, and perform subsequent processing on non-noise effective signals.

[0026] As a further preferred embodiment, the memristor input terminals of the first external sensory signal enhancement unit and the second external sensory signal enhancement unit are respectively connected to the source of a PMOS transistor. The drain of the PMOS transistor is connected to the reset voltage, and the gate of the PMOS transistor is connected to the input terminal of the transmission gate circuit in the corresponding enhancement unit. When the signal received by the transmission gate circuit from the comparator switch is low, the PMOS transistor is turned on, and the resistance state of the memristor in the corresponding enhancement unit is reset by the reset voltage.

[0027] In specific implementations, regarding biomimetic stimulation enhancement circuits, for example, such as... Figure 2 As shown, it can mainly include: a neuron circuit control module and a memristor. and (Resistive state), transmission gate circuit and and the operational amplifier, etc. The first neuron circuit control unit comprises: transistor , transistor , (stable signal), etc. The input voltage is connected to the control end of transistor , transistor , and the related resistance, etc. form a specific circuit structure, and the output end is connected with the subsequent AND gate , etc. The second neuron circuit control unit comprises: transistor , transistor , etc. The input voltage is also connected to the control end of transistor , transistor , and the related resistance, etc. form a circuit structure, and the output end is connected with the subsequent inverter , etc. The transmission gate is controlled by the AND gate , and the inverter controls the transmission gate, which is used to realize the transmission control of the signal, and the internal also involves a sub-circuit composed of a memristor, etc. The operational amplifier part comprises: operational amplifier , operational amplifier , operational amplifier , operational amplifier and multiple resistors, etc. The operational amplifier , etc. cooperates with the related resistance to process the signal passing through the transmission gate circuit, and the related output ends of the operational amplifier circuit finally constitute the output of the whole system. In addition, there are transistors , etc. cooperating with , etc. to realize the reset function of the circuit, etc.

[0028] Figure 2 is the core of the bionic stimulation enhancement circuit for multi-sensory signal preprocessing. In this example, it is divided into dolphin hearing enhancement sub-circuit (mode 1) and pigeon vision enhancement sub-circuit (mode 2) according to the mode. Taking the "fire rescue scene" as an example, the input signal is: hearing signal (survivor's distress sound converted to =0.3V, containing 1kHz noise), visual signal (flame image converted to =0.4V, containing fuzzy interference), and the two signals are synchronously input into the circuit. is the reset voltage, which is used to reset the memristor resistance value during circuit initialization. NMOS tube (threshold voltage 1.6V), PMOS tube (threshold voltage -0.8V).

[0029] The working process of the circuit is explained in detail below. In the dolphin hearing enhancer circuit, the input signal is the voltage converted from environmental sound waves (0.1-0.6V, containing 1kHz low-frequency noise), the circuit integrates the dolphin hearing biomimetic mechanism-a weak signal amplifier with a gain of 10 times (amplifying low-frequency weak distress signals); retains the 50kHz target sound wave (distress signal main frequency), triggers the NMOS tube to turn on, the tube turns off, the neuron circuit 2 outputs a low level, the neuron circuit 1 outputs a high level, and the NOT gate outputs a high level, and the AND gate outputs a high level, and the control transmission gate turns on, and the memristor has a voltage applied across its two terminals, and its resistance decreases from 20kΩ to 0.5kΩ as the charge accumulates; at the same time, the enhanced signal is finally output through the operational amplifier circuit . The reset voltage connects the PMOS tube , which is used to reset the memristor resistance value during circuit initialization. In the pigeon vision enhancer circuit: the input signal is the voltage converted from image pixels (more than 0.6V, containing fuzzy interference), the circuit integrates the pigeon vision biomimetic mechanism-contains a 640x480 pixel sampling unit, an edge enhancement amplifier with a gain of 20dB (highlighting the flame, target contour), extracts the contour voltage through the pixel sampling unit, and the edge enhancement amplifier amplifies the flame contour voltage by 20dB, and the frame synchronizer generates a pulse signal with a frequency of 20kHz; this signal triggers the NMOS tube and turns on, and the inverter outputs a low level, and the control transmission gate turns on, and the memristor resistance value decreases from 15kΩ to 0.5kΩ, and the enhanced signal is finally output through the operational amplifier circuit . The operational amplifier passes through a same-phase / anti-phase amplifier and a summing circuit (such as and to form a differential summing structure) to "amplify and superimpose" the "weighted signals"; the final is the result of "weighted integration" of the multi-channel input signal, simulating the "integration output of neurons to multi-synaptic input". The voltage of

[0030] After that, the circuit repeats the above process again and continuously outputs a pulse voltage of a certain frequency, i.e. the enhanced multi-modal signal.

[0031] In a specific implementation, regarding the synaptic circuit, as shown in Figure 3 , n synaptic circuits (numbered 1~n) can include: inverters , resistors and a plurality of (n*n-1) memristors, etc. The input voltage is respectively connected to the input end of the corresponding inverter , and is also respectively connected to the subsequent circuit composed of memristors, resistors, etc. through resistors , etc. The output end of the circuit part related to is used to realize the function of learning, etc. Finally, the signals of multiple branches are output from the output end of , etc. after being processed by the related circuit.

[0032] As a preferred implementation, each associative learning branch includes two inverters, two memristors, and two PMOS tubes in cascade, the gates of the two PMOS tubes are respectively connected to the outputs of the two inverters, the source of one PMOS tube is connected to the associative learning voltage , and the drain of the other PMOS tube is connected to the two memristors used to store the associative weight between the two inter-associated modalities. When the two synaptic circuits corresponding to the two inter-associated modalities have corresponding modality enhancement signals input, through the reverse of the inverter, the two PMOS tubes in cascade are turned on, and the associative learning voltage is applied to the positive end of the memristor in the feedback branch of the corresponding synaptic circuit through the two PMOS tubes, respectively, and the resistance state of the two memristors is decreased, and the corresponding amplification weight is increased, realizing associative learning.

[0033] As a preferred implementation, the parameters of all the memristors in the multi-sensory inter-associative memory circuit are the same.

[0034] As a preferred implementation, a proportional operational amplifier is integrated inside each retrieval neuron circuit to adjust the amplitude of the synaptic circuit output signal, and then the corresponding retrieval signal is generated after the comparison circuit as the retrieval neuron channel, wherein if the enhanced signal after amplitude adjustment reaches the threshold voltage of the retrieval neuron channel in the retrieval neuron circuit, the retrieval neuron is activated to obtain the retrieval signal, otherwise it cannot be activated and no retrieval signal is generated.

[0035] As described above, regarding the synaptic circuit, as shown in Figure 3 , n retrieval neuron circuits (numbered 1~n) include: operational amplifiers , resistors , transistors , etc. Taking one of them as an example, the operational amplifier The inverting input terminal is connected to a resistor The operational amplifier is connected to the output of the synaptic circuit, etc. The non-inverting input terminal is grounded, and the resistor... One end is connected to the operational amplifier The inverting input terminal is connected to the other end of the operational amplifier. Non-inverting input terminal; operational amplifier After the signal is processed in conjunction with relevant resistors, it is connected to the transistor. The control terminal, transistor With resistance The circuit is constructed from these components, ultimately from... The output terminal outputs a signal, and other channels (corresponding to) The structure is similar to the one mentioned above.

[0036] In one embodiment, the retrieval neuron circuit multiplies the output enhanced modal signal with n-1 retrieval signals fed back from the retrieval neuron circuit, and sums them with the corresponding synaptic weights. The summed value is then compared with the threshold reference voltage of the NMOS transistor in the retrieval neuron circuit to achieve the corresponding function.

[0037] In one embodiment, n The output signal of the biomimetic stimulation enhancement circuit is simultaneously input into the synaptic circuit, and the inverter output control signal controls the switching of the PMOS transistor in the synaptic circuit; when the PMOS transistor is turned on, the source voltage (Vlearn) of the PMOS transistor is applied to the memristor, thereby changing the synaptic weight.

[0038] The following details Figure 3 This section uses an example to illustrate the working mechanism of a multisensory associative memory network circuit. Figure 3 It is a synaptic circuit and a retrieval neuron circuit of a multisensory associative memory network circuit, which shows the details of three synaptic circuits and three retrieval neuron circuits. It is the voltage mapped from the enhanced signal output by the biomimetic stimulation enhancement circuit, representing the signal characterization channels of n modes respectively. This represents the output channel of each channel in a multisensory associative memory network. Taking the input channel as an example, One side is connected to the inverter , The output terminal is connected to two PMOS transistors. and The gate. On the other hand... Connected to resistor , The other end is connected to a synaptic memristor and The "-" terminal. Synaptic memristor. and The "+" terminal is connected to the resistor on one side. On the other hand, the output signal and A subtractor and a resistor are connected in series. Connection. Two PMOS transistors. and The drain output is connected to the resistor. and synaptic memristor and Between the "+" terminals. On the feedforward channel of the synaptic circuit. Output and resistance One end, operational amplifier The inverting input terminal is connected. Operational amplifier. The output terminal is connected to a resistor. and operational amplifier The inverting input terminal is connected. Operational amplifier. The output terminal and the NMOS transistor Gates connected, The source is connected to a -1.2V DC voltage source, and one end of the drain is connected to a resistor. One end is connected to a PMOS transistor. The gate of the PMOS transistor. The source of the PMOS transistor is connected to a 1V DC voltage source. One end of the drain is connected to a resistor. A section connected to the output signal .

[0039] To better illustrate the operation of multisensory associative memory networks, a trimodal associative memory network circuit is introduced here, using n=3 as an example. The following sections will describe the cross-species biomimetic associative memory network circuit in modules. , Through resistors respectively , and The input is sent to the synaptic circuit module. The first two pathways are used as examples for explanation. When the enhanced signals output by bionic stimulation enhancement circuit 1 and bionic stimulation enhancement circuit 2... and When both are high, the inverter and The output is low. This low-level signal causes the PMOS transistor to... and On, learn positive voltage Applied to memristor and The "+" terminal makes the memristor and As the resistance decreases, the synaptic connection weight increases. Retrieve three identical resistors in the neuron. and operational amplifier This forms an inverting amplifier circuit, which stabilizes the signal gain and enables effective transmission. It is a 0.5kΩ resistor. It is a 1kΩ resistor. Based on Kirchhoff's laws and the virtual short / virtual open principle of operational amplifiers, taking three modes as examples... The output voltage can be calculated as follows:

[0040] The output voltage can be calculated as follows:

[0041] The output voltage can be calculated as follows:

[0042] in The representation of is as follows:

[0043] because It is a memristor with a resistance between 1kΩ and 50kΩ. The resistance value is usually negligible, so , , , , , These represent the pairwise connection weights between modes. and When the voltage decreases under the learning voltage The output voltage at the point gradually increases, and when it exceeds the threshold voltage of 0.5V, the PMOS transistor... A high-level output indicates that the retrieval neuron is activated. The output signal is then fed back to the synaptic circuit as a retrieval signal, enabling the associative memory function of multiple sensory signals.

[0044] The cross-species biomimetic characteristics of the circuit and the inter-associative memory function are synergized to rely on the sensory processing advantages of different species to adapt multi-modal signal enhancement, and to realize cross-modal bidirectional association through dynamic adjustment of the weight of the memristor. Taking the "visual-auditory" dual modal in the fire rescue scene as an example, at the cross-species biomimetic level, the visual signal (flame image) adopts the pigeon visual enhancement mechanism, extracts the contour features through the pixel sampling unit, simulates the suppression of smoke blur interference, and converts the original 0.6V blurred visual signal into a 3V clear contour enhanced signal; the auditory signal (survivor's cry for help) adopts the dolphin auditory enhancement mechanism, amplifies the 0.3V low-frequency cry for help with the help of the weak signal amplifier, filters the 1kHz environmental noise, and outputs a 3V pure auditory enhanced signal.

[0045] The inter-associative memory function is realized through the feedback linkage of the synapse circuit and the retrieval neuron circuit: when the visual and auditory enhanced signals are input synchronously, the two signals generate a control signal through logical operation, triggering the conduction of the two PMOS tubes in the synapse circuit at the same time, and the learning voltage is applied to the memristor (vision-auditory association) and (auditory-visual association), so that the resistance values of the two memristors gradually decrease from the initial 50kΩ to 1kΩ, forming a strong association synapse weight. If the visual pathway is blocked by thick smoke (no visual signal input) subsequently, only a 3V auditory enhanced signal is input, which can directly activate the retrieval neuron circuit 2 through the resistance corresponding to the auditory modality in the synapse circuit, so that it outputs a high level (1V); As a feedback signal, it is transmitted to the retrieval neuron circuit 1. At this time, although there is no visual input, because the resistance value has been reduced to a low resistance state, the current applied to can be efficiently loaded to the inverse proportional operational amplifier circuit, finally making the retrieval neuron circuit 1 output a high level (1V). That is, a single auditory storage signal can retrieve auditory and visual associated signals at the same time, relying on cross-species biomimetics to ensure the accuracy of signal enhancement, and realizing multi-modal inter-associative memory through memristor weight adjustment, effectively solving the problem of perception completion after single channel failure in complex scenarios.

[0046] In general, the embodiment is based on core devices such as memristors, combined with multi-module collaborative work, and provides technical support closer to the biological complex perception and learning ability for the evolution of artificial intelligence towards brain inspiration on the basis of the advantages of neuromorphic engineering low power consumption and high interpretability. In practical application scenarios such as fire rescue robots, it can effectively amplify weak distress signals, suppress the interference of smoke on visual signals, and establish cross-memory association of "flame-distress sound-toxic gas", thereby improving rescue efficiency and accuracy. Therefore, the embodiment overcomes the defects of existing neuromorphic circuits in the integration of cross-species bionic mechanisms, the implementation of multi-sensory associative memory function, and the adaptability of dynamic scenes, realizes personalized enhancement mode of multi-sensory signal pathways, and simultaneously realizes bidirectional association and enhancement between multiple sensory modalities.

[0047] Embodiment two A robot comprising a multi-sensory associative memory network circuit as described in the above embodiments.

[0048] The related technical solutions are the same as above and will not be repeated here.

[0049] Those skilled in the art will readily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A multisensory inter-associative memory network circuit based on cross-species biomimicry, characterized by, comprising n one biomimetic stimulus enhancement circuit and one multisensory interlinked memory circuit; n is an integer not less than 2; Each bionic stimulation enhancement circuit comprises two neuron circuit control units and two external sensory signal enhancement units; the two control units are used to judge whether the received external sensory signal is a non-noise effective signal according to the signal strength, and to select one of the two enhancement units to enhance the external sensory signal judged as an effective signal; the two enhancement units have the same circuit structure, and each comprises an operational amplifier unit comprising a memristor for enhancing and amplifying the external sensory signal, wherein the memristor determines the enhancement mode of the external sensory signal, including the enhancement rate and the enhancement multiple; the parameters of the memristors in the two enhancement units are different, and correspond to the enhancement modes of strong and weak effective signals respectively, and the two enhancement modes are determined by the processing mechanism of the corresponding modal effective signal of the bionic animal corresponding to the bionic stimulation enhancement circuit. The multisensory interconnected thought memory circuit comprises n a synapse circuit and n a retrieval neuron circuit; one mode corresponds to one synapse circuit and one retrieval neuron circuit, each synapse circuit comprises one forward branch and n-1 retrieval feedback branches, n-1 retrieval feedback branches are used to receive the outputs of the retrieval neuron circuits of the remaining n-1 modes corresponding to the synapse circuit one by one, and are connected to the corresponding outputs through a positive terminal and the negative terminal of the memristor connected to the forward branch respectively, each output is multiplied by the weight of the corresponding memristor, the forward branch is used to add the enhancement signal received from the corresponding mode of the biomimetic stimulus enhancement circuit and each product, and then transmit to the corresponding retrieval neuron circuit, simulate the retrieval of the human brain to the enhancement signal through the retrieval neuron circuit, and output the retrieval signal of the corresponding mode; each two synapse circuits also share an associative learning branch, one end is used to receive the enhancement signal from the biomimetic stimulus enhancement circuit of the two corresponding modes, and the other end is connected to the positive terminals of the two memristors in the two synapse circuits, used to learn the bidirectional association between the two modes through the memristor when the two corresponding modes have enhancement signal input and store it in the form of memristor weight.

2. The multisensory inter-associative memory network circuit of claim 1, wherein, The two neuron circuit control units are respectively referred to as a first neuron circuit control unit and a second neuron circuit control unit, and the two external sensory signal enhancement units are respectively referred to as a first external sensory signal enhancement unit and a second external sensory signal enhancement unit; each external sensory signal enhancement unit uses a transmission gate circuit as an input port to transmit the non-noise effective signal to the operational amplifier unit. The first neuron circuit control unit includes a first comparator switch and an AND gate. The second neuron circuit control unit includes a second comparator switch and an inverter. The control terminals of the first and second comparator switches have different threshold voltages; the output of the second comparator switch is connected to an inverter. The input of the second external sensory signal enhancement unit and the input of the transmission gate circuit; AND gate The two inputs are the output of the first comparator switch and the inverter. The output of the gate The output is connected to the transmission gate circuit in the first external sensory signal enhancement unit; The control ends of the first comparator switch and the second comparator switch are opened by setting a threshold voltage to determine whether the first external sensory signal enhancement unit and the second external sensory signal enhancement unit are opened under the current external sensory signal input.

3. The multisensory inter-associative memory network circuit of claim 2, wherein, The first comparator switch and the second comparator switch have the same structure, and each comprises an NMOS tube and a PMOS tube, the gate of the NMOS tube is the input end of the comparator switch, the drain of the NMOS tube is connected with the gate of the PMOS tube, and the drain of the PMOS tube is the output end of the comparator switch; the drain of the NMOS tube and the drain of the PMOS tube are connected to the ground through a grounding resistor; the source of the NMOS tube and the source of the PMOS tube are respectively connected to a preset DC voltage source; The first comparator switch and the second comparator switch comprise four preset DC voltage sources in total, and the voltages of the four preset DC voltage sources are different, satisfying that the bionic stimulation enhancement circuit can imitate the strong and weak distinction of the corresponding bionic animal to the corresponding modal external sensory signal, filter out noise signals, and perform subsequent processing on non-noise effective signals.

4. The multisensory inter-associative memory network circuit of claim 2, wherein, The source of a PMOS tube is further connected to the memristor input end of the first external sensory signal enhancement unit and the second external sensory signal enhancement unit, the drain of the PMOS tube is connected to a reset voltage, and the gate of the PMOS tube is connected to the input end of the transmission gate circuit in the corresponding enhancement unit; when the signal received by the transmission gate circuit from the comparator switch is at a low level, the PMOS tube is opened, and the resistance state of the memristor in the corresponding enhancement unit is reset through the reset voltage.

5. The multisensory inter-associative memory network circuit of claim 1, wherein, Each associative learning branch includes two inverters, two memristors, and two PMOS tubes in cascade, the gates of the two PMOS tubes are respectively connected to the outputs of the two inverters, the source of one PMOS tube is connected to the associative learning voltage , and the drain of the other PMOS tube is connected to the two memristors for learning and storing the associative weight between the two inter-associated modalities; When the two synapse circuits corresponding to the two interrelated modes have corresponding mode enhancement signal inputs, the two PMOS transistors in cascade are turned on through the reverse of the inverter, and the associative learning voltage The resistance states of the two memristors are both decreased, and the corresponding amplification weights are both increased, so that the associative learning is realized.

6. The multisensory inter-associative memory network circuit of claim 5, wherein, The parameters of all the memristors in the multi-sensory interconnected memory circuit are the same.

7. The multisensory inter-associative memory network circuit of claim 1, wherein, The proportional operational amplifier is integrated in each retrieval neuron circuit to adjust the amplitude of the synaptic circuit output signal, and then the adjusted signal is input into the comparison circuit as the retrieval neuron channel to generate the corresponding retrieval signal. If the adjusted signal reaches the threshold voltage of the retrieval neuron channel in the retrieval neuron circuit, the retrieval signal is activated; otherwise, the retrieval signal cannot be activated.

8. A robot, characterized in that The multisensory inter-associative memory network circuit as claimed in any one of claims 1 to 7.

Images