Closed-loop vision-motion man-machine interaction system based on NbOIphotosynapse

Abstract

The invention discloses a closed-loop vision-to-motion man-machine interaction system based on NbOI photoelectric synapse, and belongs to the technical field of neuromorphic electronics, the closed-loop vision-to-motion man-machine interaction system is characterized in that a NbOI photoelectric synapse module takes a two-dimensional ferroelectric semiconductor NbOI as a unique core function layer and has optical pulse and electric pulse dual-mode regulation and control capability; an inhibitory / excitatory synaptic current can be selectively output, and the energy consumption of a single synaptic event is less than or equal to 100aJ; a visual optical signal output by the visual perception module can be directly input into the synapse module for processing, and an additional optical-electric conversion sub-module is not needed; the motion execution module executes a target action according to the regulation and control signal and feeds back a state to realize closed-loop self-adaptive regulation and control; all the modules form a closed loop feedback link through the signal interaction module. The problems of high energy consumption, single function and poor stability of traditional synapses are effectively solved, the system architecture is simplified, scenes such as edge computing and intelligent robots are adapted, and the method has important practical value.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of neuromorphic electronics, and particularly relates to a closed-loop visual-to-motor human-computer interaction system based on an NbOI2 optoelectronic synapse. BACKGROUND

[0002] With the development of intelligent robots, brain-computer interfaces, edge computing and the like, the traditional separated architecture of "visual processing (high-power chip)-decision making (CPU / GPU)-motor control (special driver)" has obvious limitations. The edge scene has very high requirements for "low power consumption", and the energy consumption of the traditional GPU / CPU is difficult to meet the requirements. Human-computer cooperation needs "real-time response", and the signal transmission delay of the separated architecture will reduce the interaction efficiency. Biological systems realize efficient regulation and control through parallel and low-energy synapses, which provides a simulation paradigm for artificial systems.

[0003] The traditional CMOS device is difficult to efficiently simulate the functions of biological synapses such as "continuous weight update" and "optoelectronic cooperative regulation" due to physical limitations. The emergence of two-dimensional ferroelectric semiconductors (such as NbOI2) solves this problem. Two-dimensional materials have an atomic thickness and strong interface regulation capability, and the synapse current can be precisely adjusted through a gate. The ferroelectric characteristic endows the device with a "non-volatile memory" function, such as long-term storage of LTP / LTD, without the need for continuous power supply. The optoelectronic cooperative response (both optical pulses and electrical pulses can regulate the synapse function) can directly interface with visual signals (optical input) and electrical regulation (motor instructions), simplifying the "perception-regulation" link.

[0004] Traditional human-computer interaction systems are mostly "open-loop", such as mechanical arms moving according to pre-set programs, which do not respond to environmental changes in real time. However, biological systems realize self-adaptation through a closed loop of "perception-feedback-adjustment", such as humans adjusting hand movements through neural feedback after seeing a target. The emergence of neuromorphic devices (such as NbOI2 synapses) makes it possible to integrate a closed loop of "perception (vision)-memory / regulation (synapse)-execution (mechanical arm)", which can simulate the efficiency and adaptability of biological pathways.

[0005] Therefore, there is an urgent need for a closed-loop visual-to-motor human-computer interaction system based on an NbOI2 optoelectronic synapse to address the bottlenecks of high energy consumption, single function and poor stability of synapse devices, including excessively high energy consumption, the energy consumption of traditional synapses is usually pJ level (10 - 12 J), or even μJ level (10 -6 J), which is much higher than that of biological synapses (about 100 fJ, 10 -13J), it is difficult to meet the low-power demand of edge devices; the function is limited, and most synapses only support electrical pulse regulation and cannot directly interface with visual signals (light input), which requires additional "light-to-electricity conversion modules", resulting in increased system complexity; some synapses cannot achieve "inhibitory current regulation" and can only simulate excitatory synapse functions, limiting the computational flexibility of neural networks; and the stability is insufficient, and synapses based on non-ferroelectric materials have short memory retention time (volatile) and low weight update accuracy, affecting the long-term reliability of the system. SUMMARY

[0006] To solve the above technical problems, a closed-loop visual-to-motor human-computer interaction system based on an NbOI2 optoelectronic synapse is provided, which includes a visual perception module, an NbOI2 optoelectronic synapse module, a motor execution module, and a signal interaction module. The visual perception module, the NbOI2 optoelectronic synapse module, and the motor execution module are sequentially connected by the signal interaction module and form a closed-loop feedback link. The NbOI2 optoelectronic synapse module has a two-dimensional ferroelectric semiconductor NbOI2 as the only core functional layer and has dual-mode regulation capability of light pulses and electrical pulses. The positive gate pulse regulation can output stable inhibitory current, the light pulse regulation can output stable excitatory current, and the energy consumption of a single synapse event is ≤100 aJ. The environmental visual light signal output by the visual perception module can be directly input to the NbOI2 optoelectronic synapse module for signal processing without the need for additional configuration of a light-to-electricity conversion submodule. The motor execution module executes target actions according to the regulation signal output by the NbOI2 optoelectronic synapse module and feeds back the action execution state to the visual perception module through the signal interaction module, realizing closed-loop adaptive regulation.

[0007] As a preferred scheme of the closed-loop visual-to-motor human-computer interaction system based on the NbOI2 optoelectronic synapse, the visual perception module includes a binocular image acquisition unit and a target recognition and positioning submodule. The binocular image acquisition unit is used to acquire environmental three-dimensional visual information, and the target recognition and positioning submodule is constructed based on a YOLO-v8 neural network and is used to perform real-time recognition and spatial positioning on target objects in the three-dimensional visual information. The positioning result is output as an environmental visual light signal to the signal interaction module.

[0008] As a preferred scheme of the closed-loop visual-to-motor human-computer interaction system based on the NbOI2 optoelectronic synapse, the NbOI2 optoelectronic synapse module is a three-terminal device structure, sequentially comprising a gate electrode, a gate dielectric layer, an NbOI2 functional layer, a source electrode and a drain electrode from top to bottom, and the source electrode and the drain electrode are symmetrically distributed at both ends of the NbOI2 functional layer; the thickness of the NbOI2 functional layer is 35 nm, and the amplitude of the inhibitory synaptic current and the excitatory synaptic current can be continuously adjusted by adjusting the gate voltage.

[0009] As a preferred scheme of the closed-loop visual-to-motor human-computer interaction system based on the NbOI2 optoelectronic synapse, the NbOI2 optoelectronic synapse module can simulate the paired pulse depression (PPD), excitatory postsynaptic current (EPSC), paired pulse facilitation (PPF), long-term potentiation (LTP) and long-term depression (LTD) core neural behaviors of biological synapses; and the non-volatile memory storage is realized based on the ferroelectric characteristics of the NbOI2 material.

[0010] As a preferred scheme of the closed-loop visual-to-motor human-computer interaction system based on the NbOI2 optoelectronic synapse, the motion execution module comprises a multi-degree-of-freedom mechanical arm and a five-channel “synapse-TIA” driving sub-module; the five-channel “synapse-TIA” driving sub-module is used for converting the synaptic current signal output by the NbOI2 optoelectronic synapse module into a mechanical arm driving voltage signal, and controlling the multi-degree-of-freedom mechanical arm to complete target approach, accurate grabbing and multi-joint collaborative parallel motion.

[0011] As a preferred scheme of the closed-loop visual-to-motor human-computer interaction system based on the NbOI2 optoelectronic synapse, the signal interaction module comprises a signal preprocessing sub-module and a feedback adjustment sub-module; the signal preprocessing sub-module is used for carrying out noise reduction and filtering processing on the light signal output by the visual perception module, and converting the current signal of the NbOI2 optoelectronic synapse module into a digital signal compatible with the motion execution module; the feedback adjustment sub-module collects the action position, speed and attitude signals of the motion execution module in real time, and feeds back to the visual perception module to trigger the update of the environmental visual information.

[0012] As a preferred scheme of the closed-loop visual-to-motor human-computer interaction system based on the NbOI2 optoelectronic synapse, the neural computing module is further included, which is bidirectionally connected with the NbOI2 optoelectronic synapse module, and is used for extracting synaptic weight update parameters of the module and constructing a convolutional neural network.

[0013] As a preferred embodiment of the closed-loop vision-to-motion human-computer interaction system based on NbOI2 photosynapses described in this invention, the NbOI2 photosynapse module has a single synaptic event energy consumption ≤89aJ, an inhibitory synaptic current modulation range of 10pA-1μA, and an excitatory synaptic current modulation range of 10pA-2μA.

[0014] As a preferred embodiment of the closed-loop vision-to-motion human-computer interaction system based on NbOI2 photosynapses described in this invention, a computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the closed-loop vision-to-motion human-computer interaction system based on NbOI2 photosynapses as described above.

[0015] As a preferred embodiment of the closed-loop vision-to-motion human-computer interaction system based on NbOI2 photosynapses described in this invention, it includes a computer device, comprising a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the closed-loop vision-to-motion human-computer interaction system based on NbOI2 photosynapses.

[0016] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the described closed-loop vision-to-motion human-computer interaction system based on NbOI2 photosynapses.

[0017] The beneficial effects of this invention are: The NbOI2 opto-synapse module uses the two-dimensional ferroelectric semiconductor NbOI2 as its sole core functional layer. The energy consumption of a single synaptic event is ≤100aJ, which is 3-6 orders of magnitude lower than the pJ or even μJ energy consumption of traditional CMOS and metal oxide synapses. It is far superior to the energy consumption level of biological synapses of about 100fJ, completely solving the problem of low power supply for edge devices. It can operate stably for a long time in scenarios without continuous power supply, such as portable robots and wearable brain-computer interfaces.

[0018] The module supports dual-mode control of optical pulses and electrical pulses. The ambient visual light signal output by the visual perception module can be directly input for processing without the need for an additional optical-to-electrical conversion submodule, which greatly reduces the system complexity and size. At the same time, it can selectively output gate-adjustable inhibitory and excitatory synaptic currents, breaking through the limitation of traditional synapses that only support single excitatory control, improving the computational flexibility of neural networks, and adapting to more complex visual-motor decision-making logic.

[0019] Leveraging the ferroelectric properties of NbOI2 material, the module achieves non-volatile memory storage, with long memory retention time and high weight update accuracy. This avoids problems such as synaptic volatility and parameter drift in traditional non-ferroelectric materials, ensuring that the system can maintain accurate signal control and action execution capabilities even during long-term, high-frequency interactions.

[0020] The closed-loop link of visual perception-synaptic control-motion execution-state feedback can respond to environmental changes and action execution status in real time, achieve adaptive control, solve the problems of response delay and rigid interaction in traditional open-loop systems, and make robotic arm grasping and multi-degree-of-freedom collaboration more precise and flexible, providing stable and reliable technical support for intelligent human-machine collaboration. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is an internal framework diagram of a closed-loop vision-to-motion human-computer interaction system based on NbOI2 photoelectric synapses; Figure 2 A conceptual design diagram for a device based on NbOI2 photoelectric synapses; Figure 3 A diagram illustrating inhibitory synaptic plasticity modulated by electrical impulses; Figure 4 This is a diagram illustrating the excitatory synaptic plasticity regulated by light pulses. Figure 5 This is a graph showing the image recognition performance based on artificial neural networks; Figure 6 This is a schematic diagram of closed-loop visual motion control and multi-degree-of-freedom human-computer interaction based on NbOI2 photosynapses. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0024] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a closed-loop vision-to-motion human-computer interaction system based on NbOI2 photosynapses, comprising: The system includes a visual perception module, an NbOI2 photosynaptic module, a motion execution module, and a signal interaction module. The visual perception module, the NbOI2 photosynaptic module, and the motion execution module are sequentially connected through the signal interaction module to form a closed-loop feedback link. The NbOI2 photoelectric synapse module uses the two-dimensional ferroelectric semiconductor NbOI2 as its only core functional layer. It has dual-mode control capability of optical pulse and electrical pulse, and can selectively output inhibitory synaptic current or excitatory synaptic current with adjustable gate voltage. The energy consumption of a single synaptic event is ≤100aJ. The ambient visual light signal output by the visual perception module can be directly input to the NbOI2 photoelectric synapse module for signal processing, without the need for an additional photo-to-electric conversion submodule. The motion execution module executes the target action according to the control signal output by the NbOI2 photosynaptic module, and feeds back the action execution status to the visual perception module through the signal interaction module, thereby realizing closed-loop adaptive control.

[0025] It can selectively output inhibitory or excitatory synaptic currents: By adjusting the optical pulse parameters, electrical pulse parameters, or a combination of both input to the NbOI2 photoelectric synapse module, differential changes in the polarization state or carrier transport characteristics of the two-dimensional ferroelectric semiconductor NbOI2 functional layer are triggered, thereby selecting to output inhibitory or excitatory synaptic currents. The optical pulse parameters include wavelength, intensity, pulse width, or repetition frequency, and the electrical pulse parameters include the polarity (positive or negative), amplitude, pulse width, or pulse interval of the gate voltage. Specifically, when a positive gate voltage pulse is applied, the module outputs an inhibitory synaptic current; when an optical pulse is applied, the module outputs an excitatory synaptic current, and the amplitude of both types of currents is precisely controlled by continuously adjusting the pulse intensity and width.

[0026] Visual perception module: retinal morphology photosensitive array, using quantum dot-perovskite stacked structure (QE>90%), with spectral response range extended to 300-800nm ​​event-driven imaging, outputting pulse signals only when light intensity changes >5% (bandwidth compression ratio 100:1). The optical signal direct drive capability allows the optical pulse sequence (wavelength / frequency encoding) output by the vision module to be directly injected into the NbOI2 synapse, avoiding the delay caused by photoelectric conversion; The neuromuscular interface of the motion execution module, pulse-force mapping algorithm: ; In the formula, F represents force, which may refer to the force generated by the muscle; k is the electromechanical conversion coefficient, used to convert nerve signals into muscle force; λ is the muscle viscoelastic attenuation factor, used to describe the attenuation characteristics of muscle force after stimulation. This represents the synaptic current or nerve impulse input at time t, i.e., the input signal at the neuromuscular interface. This represents the exponential decay function, used to simulate the decay of muscle force over time; Closed-loop feedback mechanism: The electromyography sensor monitors joint angle error in real time → generates error pulse code → feeds back to the vision module to correct target recognition.

[0027] Signal interaction module: Spatiotemporal compression of pulsed neural coding, differential pulse interval modulation (DPIM): Table 1 Information type Encoding rule Bandwidth requirement Visual feature Wavelength + pulse interval (5-100 μs) < 200 kbps Motion feedback Pulse cluster phase modulation < 50 kbps Anti-interference transmission enhancement technology, ferroelectric domain wall noise suppression, utilizing the nonlinear dielectric response of NbOI2 (ε_r>250), maintains a signal-to-noise ratio of >40dB under 10GHz interference; adaptive retransmission mechanism, automatically triggers pulse retransmission when the postsynaptic current peak does not reach the threshold (success rate>99.99%).

[0028] Example 2, refer to Figure 2 This is the second embodiment of the present invention, which differs from the first embodiment in that: in the conceptual design diagram of the NbOI2-based photosynapse device, Figure 2 a is a schematic diagram of signal transmission in human neural pathways; Figure 2 b represents the crystal structure of NbOI2; Figure 2 c represents a device structure based on a three-terminal top-gate NbOI2 photosynapse, employing graphene electrodes and h-BN packaging; Figure 2 d represents the nonlinear weight update parameter extracted from the LTP / LTD curve, which is mapped to the weight update rule and embedded into a fully connected ANN to simulate visual perception and classification. Figure 2 e represents the cumulative photocurrent memory effect under light pulse stimulation, which enables closed-loop control of the robotic arm, achieving progressive extension and grasping actions.

[0029] like Figure 2 As shown in figure a, in the grasping control of a biological hand based on a vision-guided system, scattered light signals from external objects enter the eye through refraction and pupillary adjustment, focusing onto the retina, and are then converted into electrical signals by photoreceptor cells. These electrical signals are transmitted to the visual cortex of the brain via neurons and synapses, where recognition and decision-making take place. The feedback signal is then transmitted along the motor pathway to the effector muscles, causing the limb to gradually approach and eventually grasp the target object. Inspired by this vision-motor pathway, NbOI2 has been used to achieve similar information processing and execution functions.

[0030] like Figure 2 As shown in b, NbOI2 is a transition metal dihalooxide with a moderate band gap (1.6-1.9 eV). Its basic structural unit is a twisted [NbO2I4] octahedron. In the lattice, NbOI2 layers are stacked along the a-axis by van der Waals forces, while O-Nb-O and I-Nb-I chains extend along the b-axis and c-axis, respectively. Displacement of Nb atoms from the center of the octahedron induces spontaneous polarization along the Nb-O bond direction, thereby generating a built-in electric field. This electric field suppresses photogenerated carrier recombination and prolongs carrier lifetime, thus providing a physical basis for brain-like synaptic plasticity in artificial photoelectric synapses. Furthermore, compared to the stronger Nb-O bonds along the b-axis, the Nb-I bonds along the c-axis are weaker, leading to preferred cleavage orientation along the b-axis during mechanical exfoliation. Therefore, the exfoliated NbOI2 flakes typically exhibit rectangular or elongated morphologies with nearly perpendicular edges, facilitating rapid identification of crystal orientation under an optical microscope. EDS analysis of the exfoliated NbOI2 sheets showed that the spatial distribution of Nb and I elements in the samples was uniform and the composition was consistent.

[0031] like Figure 2 As shown in Figure c, to fully utilize the inherent in-plane polarization characteristics of NbOI2, a three-terminal top-gate photosynaptic device was fabricated on a flexible PET substrate. This device is arranged along the b-axis of the NbOI2 substrate. The device uses an NbOI2 sheet as the channel and graphene as the source, drain, and top-gate electrodes, respectively. Figure 2 As shown in d, H-BN is used as an inert encapsulation layer to stabilize the interface and suppress environmental oxidation, exhibiting a van der Waals heterogeneous integrated top-gate structure; nonlinear parameters extracted from the LTP and LTD plasticity of artificial synapses can be incorporated into the weight update rules of the ANN, thereby achieving reliable simulation of visual perception and image classification. Furthermore, as... Figure 2 As shown in e, the cumulative photocurrent memory effect can simulate the neural processes of motor neuron excitation and muscle contraction under continuous visual stimulation, thereby realizing the controlled progressive extension and grasping movements of the robotic arm.

[0032] Example 3, referring to Figure 3 This is the third embodiment of the present invention, which differs from the first two embodiments in that: Figure 3 Inhibitory synaptic plasticity regulated by electrical impulses, in which Figure 3 a is a schematic diagram of biological synaptic information transmission; Figure 3 b represents positive gate voltage pulses of different amplitudes (2, 4, 6, 8, and 10V, pulse width = 0.5s, V). DS IPSC response of the device at 0.1V, with an inset showing the dependence of the IPSC peak value on the pulse amplitude. Figure 3c represents the positive gate voltage pulses with different pulse widths (0.3, 0.5, 1.5, 2.5, 5, 7.5, and 10 s, pulse amplitude = 5V, V DS IPSC response at 0.1V, illustration: IPSC peak value dependence on pulse width; Figure 3 d represents the PPD exponent as a function of the pulse interval (0.2-10s, pulse amplitude = 8V, pulse width = 0.5s, V). DS =0.1V); Illustration: Composed of two continuous electrical pulses (pulse amplitude = 8V, pulse width = 0.5s, pulse interval = 8s, V DS A representative PPD response triggered by (=0.1V); Figure 3 e represents the transition from STD to LTD as the number of electrical pulses increases (pulse amplitude = 8V, pulse width = 0.5s, pulse interval = 0.5s, V). DS =0.1V). Figure 3 f represents the transition from STD to LTD as the pulse amplitude increases (pulse number = 16, pulse width = 0.5s, pulse interval = 0.5s, V). DS =0.1V).

[0033] Synapses are the basic units of information transmission between neurons in living organisms, consisting of the presynaptic membrane, the synaptic cleft, and the postsynaptic membrane. For example... Figure 3 As shown in Figure a, when an action potential reaches the presynaptic terminal, neurotransmitters are released from the presynaptic membrane, diffuse across the synaptic cleft, and subsequently bind to receptors on the postsynaptic membrane. This process induces excitatory postsynaptic currents (EPSCs) or excitatory postsynaptic currents (IPSCs), thereby completing the transmission of neural signals. The plasticity of these impulse-modulated postsynaptic currents plays a crucial role in regulating synaptic strength and forms the basis for important brain functions such as learning and memory.

[0034] like Figure 3 As shown in b, in order to systematically evaluate the synaptic behavior of the NbOI2-based three-terminal photosynaptic device using electrical simulation, its current response and related plasticity characteristics under positive gate voltage pulse stimulation were studied (all measurements were performed at V). DS (Performed in dark mode at 0.1V). When a single electrical pulse is applied through the top gate, the device exhibits a clear IPSC, a first observed in NbOI2 synapses. The instantaneous current drops below the initial baseline after pulse termination and then gradually recovers. Figure 3The illustration in Figure b shows that as the pulse amplitude increases, the IPSC peak (defined as the instantaneous minimum current after pulse withdrawal) decreases from 21 pA to 12 pA, indicating a gradual increase in suppression. The dependence of IPSC amplitude on gate voltage intensity demonstrates the voltage-dependent suppression plasticity of this device. The underlying mechanism can be attributed to the combined effect of the gate-induced electric field on carrier distribution and defect trapping within the NbOI2 sheet. When a positive gate voltage is applied, electrons are driven into the channel, resulting in an initial current increase, while some electrons are trapped in defects. After the gate pulse is removed, the free carrier density in the channel recovers rapidly, while the trapped electrons cannot be instantaneously untrapped. This transient imbalance causes the current to drop below the initial value, manifesting as IPSC. Increasing the pulse amplitude leads to a higher electron concentration and a greater number of electrons trapped in defects, thus gradually enhancing IPSC suppression. Besides pulse amplitude, the duration of the electrical pulse also significantly affects synaptic behavior. Figure 3 As shown in Figure c, a single positive gate pulse with widths of 0.3, 0.5, 1.5, 2.5, 5, 7.5, and 10 s was applied, with a fixed pulse amplitude of 5 V. As the pulse width increased, the IPSC peak value gradually decreased from 19 pA to 16 pA, indicating enhanced suppression. The inset further illustrates the dependence of the IPSC amplitude on the pulse width, confirming the time-tunable nature of the suppression plasticity.

[0035] To verify the synaptic properties of the device under time-dependent stimuli, we investigated paired-pulse plasticity behavior. In biological synapses, paired-pulse plasticity is a typical manifestation of short-term plasticity (STP), reflecting the ability to process consecutive stimuli. It describes two postsynaptic pulses generated by two consecutive inputs. If the amplitude of the current triggered by the second pulse exceeds that of the first pulse, the behavior is defined as paired-pulse facilitation (PPF). Otherwise, it is called paired-pulse depletion (PPD). The exponent of paired-pulse plasticity is defined as: ; Where A1 and A2 represent the IPSC triggered by the first and second pulses, respectively. The dependence of the PPD exponent on the pulse interval Δt can be fitted using a double exponential function: ; Where C1 and C2 are the initial amplitudes of rapid and slow decay, respectively, and τ1 and τ2 are the relaxation time constants. Figure 3 As shown in the inset in d, the device exhibits PPD behavior under two consecutive positive gate voltage pulses (amplitude: 8V, width: 0.5s). The PPD exponent monotonically decreases from 117% to 107% as the interval Δt increases from 0.2s to 10s. Fitting results yield time constants τ1 = 0.3s and τ2 = 29.98s.

[0036] After confirming the existence of short-term plasticity, the long-term plasticity characteristics of the device under multi-pulse stimulation were investigated. For example... Figure 3 As shown in Figure e, electrical pulse sequences of 4, 8, 16, 32, and 50 pulses were applied at a constant gate voltage amplitude of 8V. For a small number of pulses, limited electron trapping in the NbOI2 sheet resulted in short-term inhibition (STD). As the number of pulses increased, more electrons were trapped, leading to a significant decrease in current and the emergence of long-term inhibition (LTD), demonstrating that STD can be converted to LTD by adjusting the number of pulses. Furthermore, the effect of stimulation intensity on LTD modulation was investigated. Figure 3 As shown in f, with a fixed number of pulses of 16, electrical pulses with amplitudes of 2V, 4V, 6V and 8V are applied; increasing the gate voltage amplitude can also achieve the transition from STD to LTD, which highlights that inhibitory synaptic plasticity can be dually regulated by the number of pulses and the pulse intensity.

[0037] The results of the electrical stimulation experiments revealed the unique ability of the NbOI2-based device to modulate inhibitory synaptic plasticity. However, effective information processing in biological neural networks depends not only on inhibitory regulation but also on dynamic synergy with excitatory signals. Therefore, to fully assess the brain-like functional potential of this device, it is crucial to systematically study the excitatory synaptic behavior of the device under light pulse stimulation.

[0038] Example 4, refer to Figure 3 This is the fourth embodiment of the present invention, which differs from the previous three embodiments in that: Figure 4 This refers to excitatory synaptic plasticity regulated by light pulses, in which... Figure 4 a represents a single light pulse (V) DS =0.001V, λ=405nm, optical power density=2.8mW / cm² 2 EPSC response at pulse width = 0.1s; Figure 4 b represents the change in optical power density with increasing values ​​(2.8, 6.5, 9.1, 14.3, and 19.2 mW / cm²). 2 V DS EPSC response at 0.01V, λ=405nm, pulse width=0.3s, illustration: dependence of EPSC peak value on optical power density; Figure 4 c represents different optical pulse widths (0.3, 1, 2, 3, and 4.5 s; V DS =0.01V, λ=405nm, optical power density=9.1mW / cm² 2 EPSC response under ( ); Illustration: Dependence of EPSC peak value on pulse width; Figure 4 d represents the PPF exponent as a function of the optical pulse interval Δt (0.1-50s; optical power density = 9.1mW / cm²). 2(Pulse width = 0.3s), Inset: Representative PPF response (Vp) triggered by two consecutive light pulses. DS =0.01V, λ=405nm, optical power density=9.1mW / cm² 2 (Pulse width = 0.3s, pulse interval = 2s); Figure 4 e represents the transition from STP to LTP as the number of optical pulses increases (VT). DS =0.01V, λ=405nm, optical power density=2.8mW / cm² 2 (Pulse width = 0.3s, pulse interval = 0.3s). Figure 4 f represents the change in optical power density (V) as a function of optical power density. DS =0.01V, λ=405nm, pulse number=16, pulse width=0.3s, pulse interval=0.3s), transition from STP to LTP; Figure 4 g represents the change in light pulse width as V increases. DS =0.01V, λ=405nm, optical power density=6.5mW / cm² 2 (Pulse number = 16, pulse interval = 0.3s), transition from STP to LTP; Figure 4 h represents a simulation of the learning-forgetting-relearning process.

[0039] First, the single-pulse energy consumption was characterized to evaluate the excitatory synaptic plasticity and energy efficiency of the device under optical stimulation. Then, the dependence of the EPSC on optical parameters and plastic behavior was systematically investigated. Figure 4 As shown in Figure a, with a source-drain voltage of 1mV, a single 405nm optical pulse (optical power density: 2.8mW / cm²) 2 (Pulse width: 0.1s) induces a photocurrent of 1.14 pA in the device. The energy consumption of each synaptic event is calculated according to the following formula: ; Among them, V DS For source-drain voltage, I DS Let t be the postsynaptic current and t be the pulse duration. The energy consumption of each synaptic event was calculated to be 89 aJ, which is about two orders of magnitude lower than that of natural biological synapses (~10 fJ), thus demonstrating the ultra-low power consumption characteristics of NbOI2 photosynapses.

[0040] After confirming the ultra-low energy consumption of a single light pulse, the dependence of EPSC on external light stimulation was systematically studied. For example... Figure 4 As shown in b, when different intensities (2.8, 6.5, 9.1, 14.3 and 19.2 mW / cm) are applied to the device... 2During a single light pulse (pulse width = 0.3s), EPSC increases rapidly during illumination and gradually decays back to its initial state after the light is removed. As light intensity increases, the peak EPSC gradually rises, such as... Figure 4 The intensity dependence is shown in illustration b. This behavior can be attributed to the synergistic effect between photogenerated carriers and defect trapping. During illumination, a large number of photogenerated electrons and holes are generated in the channel, leading to a sharp increase in current. Simultaneously, intrinsic defect states in NbOI2 trap some carriers. Once illumination ceases, electrons and holes rapidly recombine, causing a sharp drop in current. Furthermore, as... Figure 4 As shown in c, the optical power density is fixed at 9.1 mW / cm². 2 Under these conditions, by varying the pulse width (0.3, 1, 2, 3, and 4.5 s), the EPSC gradually increases with increasing pulse duration, indicating that the device exhibits typical pulse duration-dependent plasticity. Importantly, the device also exhibits intensity-dependent and pulse duration-dependent excitatory behavior.

[0041] Then, short-term plasticity was further examined by applying two consecutive light pulses to simulate PPF. The peak current A2 induced by the second pulse was significantly greater than the peak current A1 generated by the first pulse. Figure 4 As shown in Figure d, as the pulse interval gradually increases from 0.1 s to 50 s, the PPF exponent decreases from 145% to 111%, and the decay behavior conforms to a double exponential decay function. The relaxation time constants τ1 are 0.58 s and 25.35 s, respectively, which are in good agreement with the typical relaxation dynamics observed in biological synapses. The observed PPF effect can be attributed to the accumulation of photogenerated carriers in the channel within the shorter pulse interval, thereby enhancing the device's response to subsequent stimuli.

[0042] Human memory is generally divided into short-term memory (STM) and long-term memory (LTM). At the synaptic level, this process manifests as a transition from STP to LTP, and LTP typically relies on repetitive or reinforcing stimulation to achieve synaptic consolidation. To simulate this process, 405nm light pulses with a fixed width (0.3s), interval (0.3s), and optical power density (2.8mW / cm²) were applied to the device. When the number of pulses was small (e.g., 4), only STP was observed, and the EPSC rapidly decayed to its original state within a short time. However, as... Figure 4 As the number of pulses increased to 8, 16, 32, and 50, the EPSC amplitude gradually increased within the same decay period, indicating a clear transition from STP to LTP. This result demonstrates that increasing the number of optical pulses can effectively modulate the STP-to-LTP conversion. Similarly, as... Figure 4As shown in f, with fixed pulse number (16), pulse width (0.3s), and pulse interval (0.3s), increasing the optical power density (2.8-19.2mW / cm²) results in... 2 This can also achieve the transition from STP to LTP. Furthermore, such as... Figure 4 g at ​​a constant optical power density (6.5 mW / cm²) 2 With the pulse count (N=16) and pulse interval (0.3s), increasing the pulse width (0.1, 0.3, 0.7, and 1.5s) also achieved the same transition from STP to LTP. To further simulate more advanced cognitive processes, the device response was examined during forgetting and relearning cycles. Figure 4 As shown in h, with an optical pulse width of 0.3s, an optical interval of 0.3s, and a power density of 6.5mW / cm², 2 Under stimulation, NbOI2 synapses exhibited a strong learning effect. After 50 pulse sequences, EPSCs decayed over time, indicating a forgetting process. After applying 22 more pulses, the NbOI2 synaptic response not only recovered to its initial level but even exceeded the initial learning current, demonstrating enhanced synaptic strength. This behavior aligns with the Ebbinghaus forgetting curve, which shows that human memory, after partial forgetting, can be rapidly recovered and strengthened through review. These results further highlight the ability of NbOI2 photosynapses to simulate the "learning-forgetting-relearning" dynamic, thus demonstrating their potential to mimic advanced brain-like learning behaviors.

[0043] Example 5, refer to Figure 5 This is the fifth embodiment of the present invention, which differs from the previous four embodiments in that: Figure 5 To assess the image recognition performance based on artificial neural networks, where Figure 5 'a' is a schematic diagram of the ANN architecture, consisting of 784 input neurons, 512 hidden neurons, and 10 output neurons, used for classification tasks on the MNIST and Fashion-MNIST datasets. Figure 5 b represents photoelectric dual-mode synaptic plasticity, where LTP consists of 16 optical pulses (λ=405nm, optical power density=6.5mW / cm²). 2 The pulse width is 0.3s and the pulse interval is 0.3s. The LTD is induced by 16 electrical pulses (pulse amplitude = 8V, pulse width = 0.5s, pulse interval = 0.5s). Figure 5 c is the confusion matrix of the MNIST dataset after 100 training epochs, where the weight updates are extracted from device characteristics; Figure 5 d is the confusion matrix of the Fashion-MNIST dataset after 100 training epochs, where the weight updates are for specific devices; Figure 5e represents a comparison of the recognition accuracy of a three-terminal opto-synaptic device based on NbOI2 and an ideal device over 100 training cycles. Figure 5 f represents representative image samples under different noise levels; Figure 5 g represents the recognition accuracy of the MNIST and Fashion-MNIST datasets after 100 training cycles under different noise levels.

[0044] To demonstrate neuromorphic computing inspired by biological vision, we constructed a three-layer fully connected neural network (ANN) and used it to classify images from the 28×28 pixel MNIST and Fashion-MNIST datasets. Figure 5 As shown in Figure a, this ANN consists of an input layer, a hidden layer, and an output layer. The input layer contains 784 neurons (corresponding to image pixels), the hidden layer contains 512 neurons, and the output layer contains 10 neurons. All neurons are interconnected through synaptic weights. In this framework, X0-X... 783 The input pixel values ​​represent the image values, and Y0-Y9 represent the output category. For the MNIST dataset, Y0-Y9 correspond to the numbers "0" to "9"; for the Fashion-MNIST dataset, they correspond to ten clothing categories, such as T-shirts, pants, and shoes. Each dataset contains 60,000 training images and 10,000 test images to evaluate recognition performance.

[0045] Given that LTP and LTD are crucial for analog neuromorphic computing, their linearity and dynamic conductance range are directly related to the recognition accuracy of ANNs. These weight update parameters are extracted by utilizing the plasticity behavior of the device under dual-mode photoelectric pulse stimulation. Figure 5 The device conductivity shown in b increases gradually with the number of pulses, corresponding to LTP; conversely, when an electrical pulse is applied, the conductivity gradually decreases with the number of pulses, corresponding to LTD. By fitting the LTP and LTD curves, the nonlinearity (NL) of the weight update process can be quantified using the following formula: ; ; Among them, G n and G n+1 G represents the conductance values ​​of the nth and (n+1)th pulses, respectively. min and G max These are the minimum and maximum conductance values, respectively. α p and α d These represent the step sizes on the enhancement and inhibition curves, respectively. β p and β d These represent the neurotransmitter (NL) coefficients for the enhancing and inhibiting processes, respectively. The calculated β...p and β d The values ​​are 3.06 and 2.15, respectively. During ANN training, when the gradient is positive, the synaptic weights follow the LTP update rule; when the gradient is negative, the synaptic weights follow the LTD update rule.

[0046] like Figure 5 c and Figure 5 As shown in Figure d, based on the extracted weight update parameters, synapse-based weight update rules are embedded into the ANN to simulate an image recognition task. Using these synapse-based update rules, confusion matrices are presented using the MNIST and Fashion-MNIST datasets after 100 training epochs. The high consistency between the predicted categories and the true labels confirms the network's effective classification ability on both datasets. Figure 5 The classification performance was further compared between the ideal weight update (implemented using an Adam optimizer with backpropagation and cross-entropy loss) and the NbOI2 synapse-derived weight update. In both cases, the classification accuracy progressively improved with increasing training iterations. After 100 training iterations, the accuracy based on NbOI2 synapses reached 93.61% on MNIST and 84.57% on Fashion-MNIST. These results are only slightly lower than the corresponding ideal accuracies of 98.37% and 89.61%, thus fully demonstrating the feasibility of using NbOI2-based photosynapses for ANN classification tasks.

[0047] like Figure 5 As shown in f, to evaluate the noise tolerance of the ANN, additive random noise of different intensities was introduced into the training and testing datasets, with the noise level gradually increasing from 0% to 40%. Figure 6 The results show that as the noise intensity increases from 0% to 40%, the classification accuracy of the MNIST dataset decreases from 93.61% to 89.08%, a drop of 4.53%; while the classification accuracy of the Fashion-MNIST dataset decreases from 84.57% to 80.16%, a drop of 4.41%. These results demonstrate that even under significant noise interference, ANNs exhibit robust recognition performance, highlighting the reliability of NbOI2-based photosynapses in practical neuromorphic computing applications.

[0048] The results demonstrate that the ANN built upon NbOI2 three-terminal photonic synaptic devices not only achieves efficient classification on standard benchmark datasets but also maintains stable performance in noisy environments. These findings fully validate the practical potential of such devices as important building blocks for next-generation neuromorphic computing systems.

[0049] Example 6, refer to Figure 6 This is the sixth embodiment of the present invention, which differs from the previous five embodiments in that:Figure 6 This is for closed-loop vision-motion control and multi-degree-of-freedom human-computer interaction based on NbOI2 photosynapses, where Figure 6 a is a schematic diagram of a closed-loop interactive system architecture inspired by the biological visual-neural-motor pathway; Figure 6 b is a dynamic demonstration of the grasping process. The left image shows the change of the voltage signal after TIA conversion over time, and the right image shows real-time target recognition performed on a computer using a stereo camera combined with a YOLO-v8 neural network; Figure 6 c demonstrates multi-channel scalability: five independent “device-TIA” channels drive the five fingers of the robotic arm, enabling transitions from a fist gesture to multiple predefined hand gestures.

[0050] To further validate the application of NbOI2 synapses in real-world scenarios, inspired by the biological "visual-neural-motor" pathway, a closed-loop human-computer interaction system was designed and implemented. This system uses a front-end binocular camera paired with a YOLO-v8 neural network as the "eye-brain" to achieve real-time detection and localization of target objects and the robotic arm's palm. Two NbOI2 opto-synaptic devices act as "neural synapses," responsible for converting and transmitting sensory signals. A transimpedance amplifier (TIA) (gain = 1010V / A, bandwidth = 10kHz) and a stepper motor-driven robotic arm serve as "motion effectors." The system employs an STM32 microcontroller as the central control unit to acquire and process the current signal amplified by the TIA and the position data transmitted from the front end, generating control commands to drive the robotic arm's movement. Through this integrated process, the system establishes a closed-loop control chain of perception-decision-execution-re-perception, effectively simulating the biological sensory-motor pathway.

[0051] Real-time recognition and distance estimation are performed using a binocular camera and a YOLO-v8 neural network. For example... Figure 6 As shown in Figure a, a binocular camera captures scene images, and a YOLO-v8 neural network performs real-time recognition and bounding box detection on a computer for the "robotic arm" and the "target object (mannequin)," outputting their relative positions and distance estimates on the image plane. When the system determines that the target object is within the reach of the robotic arm, the microcontroller triggers an illumination source to emit light pulses, stimulating NbOI2 synapses 1 and 2 respectively. The photocurrent generated by synapse 1 is converted into a voltage signal via a TIA. When this voltage exceeds a preset threshold, the forward step size of the robotic arm is calculated based on a pre-calibrated transfer function to adjust the extension distance of the robotic arm. Similarly, the photocurrent generated by synapse 2 is converted into a voltage signal via a TIA. When this signal exceeds a threshold, it triggers the robotic arm's grasping action. These two synaptic signals act sequentially to achieve the sequential execution of "approaching the target" and "grasping the target."

[0052] Dynamic details of the crawling process, such as Figure 5As shown in Figure b. The left image shows the voltage signal amplified by TIA, and the right image shows the real-time image generated by the binocular camera system combined with the YOLO-v8 network and calibrated by the computer. When the target is not present, the outputs of NbOI2 synapse 1 and NbOI2 synapse 2 show only slight fluctuations, and the image recognition module only calibrates the "hand" and "doll" objects. When the target object (doll) enters, the system applies a light pulse to NbOI2 synapse 1. The EPSC generated by NbOI2 synapse 1 is converted into a voltage signal by TIA, and its peak amplitude is mapped to the rotation angle of the stepper motor, driving the robotic arm to approach the target. If the grasping conditions are not met, the system continues to apply light pulses to synapse 1, using the synaptic memory effect and the acceleration of accumulated current to gradually amplify the voltage peak, making the robotic arm approach the target. When the "hand-doll" unit recognizes the appropriate distance that meets the grasping conditions, the system immediately stimulates NbOI2 synapse 2 with a light pulse. The voltage output of synapse 2 triggers the robotic arm to close and complete the grasp. Subsequently, the system instructs the robotic arm to lift the target. This closed-loop iteration of "visual feedback → synaptic accumulation → step control → re-recognition" enables the system to autonomously perform approach and grasping operations.

[0053] To evaluate the scalability of NbOI2 synapses in multi-channel motion control, we constructed a five-channel device-TIA configuration corresponding to the five fingers of a robotic hand. Without stimulation, the robotic hand maintains a clenched fist posture. When a light pulse stimulates a given synapse, an EPSC is generated and converted into a voltage output via the TIA, corresponding to an extended finger. By selectively stimulating different combinations of synaptic channels, the robotic hand can execute predefined gesture transition sequences, such as changing from a clenched fist to an "OK" gesture. Experimental evidence highlights the programmability and scalability of NbOI2-based synaptic devices in multi-degree-of-freedom parallel motion control, thus confirming their potential in complex human-computer interaction scenarios.

[0054] This section demonstrates the application of NbOI2-based opto-synapses in a system-level "observation-movement-grasping" closed-loop architecture. Environmental perception and state assessment are achieved through binocular vision and target detection, and synaptic outputs are converted into effective control signals via TIA (Transient Induction Algorithm). The inherent memory characteristics of synapses enable adaptive acceleration of the robot during approach, thereby achieving arrival determination and executing the grasping action. By constructing a five-channel "synapse-TIA" pathway, independent and coordinated finger actuation can be achieved without relying on a visual recognition module. These results demonstrate that the proposed NbOI2 synapse and system architecture not only support algorithm-level pattern recognition tasks (… ​ Furthermore, it provides stable and reusable perception-control integration in real-world human-computer interaction scenarios. This result offers a feasible approach for hardware platforms geared towards neuromorphic vision and intelligent human-computer collaboration.

[0055] Example 7, the seventh embodiment of the present invention, differs from the previous six embodiments in that it is a closed-loop vision-to-motion human-computer interaction system based on NbOI2 photosynapses, comprising: If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0056] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0057] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0058] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented in combination with any of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0059] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A closed-loop vision-to-motor human-machine interaction system based on NbO12 optoelectronic synapse, characterized in that: The visual perception module, the NbOI2 optoelectronic synapse module, the motion execution module, and the signal interaction module are sequentially connected by the signal interaction module and form a closed-loop feedback link. The NbOI2 optoelectronic synapse module has a two-dimensional ferroelectric semiconductor NbOI2 as a unique core functional layer and has a double-mode regulation and control capability of optical pulses and electrical pulses. The positive gate pulse regulation can output a stable inhibitory current, and the optical pulse regulation can output a stable excitatory current. The energy consumption of a single synapse event is less than or equal to 100 aJ. The environmental visual light signal output by the visual perception module can be directly input to the NbOI2 optoelectronic synapse module for signal processing without additional configuration of an optical-electric conversion submodule. The motion execution module executes a target action according to the regulation and control signal output by the NbOI2 optoelectronic synapse module and feeds back the action execution state to the visual perception module through the signal interaction module to realize closed-loop adaptive regulation and control.

2. The closed-loop visual-to-motor human-machine interaction system based on NbO I2optical synapse of claim 1, wherein: The visual perception module includes a binocular image acquisition unit and a target recognition and positioning submodule. The binocular image acquisition unit is used to acquire environmental three-dimensional visual information. The target recognition and positioning submodule is constructed based on a YOLO-v8 neural network and is used to perform real-time recognition and spatial positioning on target objects in the three-dimensional visual information. The positioning result is output as an environmental visual light signal to the signal interaction module.

3. The closed-loop visual-to-motor human-machine interaction system based on NbO I2optical synapse of claim 1, wherein: The NbOI2 optoelectronic synapse module is a three-terminal device structure. From top to bottom, it includes a gate electrode, a gate dielectric layer, an NbOI2 functional layer, a source electrode, and a drain electrode. The source electrode and the drain electrode are symmetrically distributed at both ends of the NbOI2 functional layer. The thickness of the NbOI2 functional layer is 35 nm. By adjusting the gate voltage, the amplitude of the inhibitory synaptic current and the excitatory synaptic current can be continuously adjusted.

4. The closed-loop visual-to-motor human-machine interaction system based on NbO I2optical synapse of claim 1, wherein: The NbOI2 optoelectronic synapse module can simulate the paired pulse depression (PPD), excitatory postsynaptic current (EPSC), paired pulse facilitation (PPF), long-term potentiation (LTP), and long-term depression (LTD) core neural behaviors of biological synapses. Non-volatile memory storage is realized based on the ferroelectric properties of the NbOI2 material.

5. The closed-loop visual-to-motor human-machine interaction system based on NbO I2optoelectronic synapse of claim 1, wherein: The motion execution module includes a multi-degree-of-freedom mechanical arm and a five-channel "synapse-TIA" driving submodule. The five-channel "synapse-TIA" driving submodule is used to convert the synaptic current signal output by the NbOI2 optoelectronic synapse module into a mechanical arm driving voltage signal to control the multi-degree-of-freedom mechanical arm to complete target approaching, precise grabbing, and multi-joint collaborative parallel motion.

6. The closed-loop visual-to-motor human-machine interaction system based on NbO I2optoelectronic synapse of claim 1, wherein: The signal interaction module includes a signal preprocessing submodule and a feedback adjustment submodule. The signal preprocessing submodule is used to perform noise reduction and filtering processing on the light signal output by the visual perception module and convert the current signal of the NbOI2 optoelectronic synapse module into a digital signal compatible with the motion execution module. The feedback adjustment submodule collects the action position, speed, and attitude signals of the motion execution module in real time and feeds them back to the visual perception module to trigger environmental visual information updating.

7. The closed-loop visual-to-motor human-machine interaction system based on NbO I2optical synapse of claim 1, wherein: Also comprising a neural computing module, which is in bidirectional signal connection with the NbOI2 optoelectronic synapse module, for extracting synaptic weight update parameters of the module and constructing a convolutional neural network.

8. A closed-loop visual-to-motor human-machine interaction system based on NbO12 photoelectric synapse, characterized in that, Comprising: The energy consumption of a single synaptic event of the NbOI2 optoelectronic synapse module is ≤89 aJ, the regulation range of inhibitory synaptic current is 10 pA-1 μA, and the regulation range of excitatory synaptic current is 10 pA-2 μA. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to realize the steps of the closed-loop visual-to-motor human-computer interaction system based on the NbOI2 optoelectronic synapse in any one of claims 1 to 8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the closed-loop visual-to-motor human-computer interaction system based on the NbOI2 optoelectronic synapse in any one of claims 1 to 8.

Images