Multimode floating gate type photoelectric synapse transistor-based endangered animal monitoring method

Abstract

The invention provides an endangered animal monitoring method based on a multi-mode floating gate type photoelectric synapse transistor, realizes multi-mode fusion of images (optical signals) and sounds (electric signals) by utilizing the light absorption capability of quantum dots and the high carrier mobility of a two-dimensional channel material, and belongs to the technical field of sensing, storage and calculation integration and endangered animal identification and monitoring. According to the method, a photoelectric synapse device is used as a reservoir, and input time sequence signals are respectively applied to a multi-mode fused photoelectric synapse transistor in the form of light and electric pulse. And along with the co-stimulation of light and electric pulse, the change response of source-drain current is generated, and the fusion processing of the multi-mode signal is realized. By adopting the method, a human multi-sensory fusion signal processing mode is successfully simulated, and accurate recognition under complex and changeable conditions can be realized by utilizing a mode of complementation of visual signals and auditory signals. Under an extreme environment or a high noise condition, edge processing and identification can be carried out on pictures and sounds of endangered animals, and a high-accuracy monitoring result can be obtained without a large amount of data return, so that the energy consumption is reduced and the monitoring efficiency is enhanced.

Description

Technical Field

[0001] The present invention belongs to the cross - technical field of neuromorphic computing and intelligent wildlife monitoring, and particularly relates to a hybrid - dimensional floating - gate optoelectronic synaptic transistor constructed based on zero - dimensional quantum dots and two - dimensional materials, which is used for identifying endangered animals with multi - modal audio - visual fusion under extreme weather or noisy environments. Background Art

[0002] The number of globally endangered wild animals continues to increase, and their living environments are often threatened by climate change, habitat destruction, illegal hunting, etc. To timely discover and protect endangered animal populations, it is necessary to monitor their living areas for a long time. However, traditional monitoring methods (such as simply using camera traps or acoustic fingerprint samplers) often have difficulty in stably identifying or need to transmit massive amounts of data back to the background for complex analysis when facing environments such as extreme lighting conditions, strong noise, or limited network bandwidth, resulting in high latency and high power consumption.

[0003] If multi - modal signals such as audio and video can be directly fused and initially identified in the front - end sensor, it can significantly reduce data transmission and power consumption and ensure relatively accurate identification results can still be obtained in harsh environments. For this purpose, new devices need to simultaneously possess efficient light response, sensitivity to electrical signals, and multi - level storage characteristics to support multi - modal fusion. The hybrid - dimensional floating - gate optoelectronic synaptic transistor constructed from zero - dimensional quantum dots (CsPbBr3 QDs) and two - dimensional materials (MoS2) is a potential implementation of such devices: zero - dimensional quantum dots can significantly enhance light absorption and storage characteristics, while two - dimensional material MoS2 can provide high carrier mobility and tunable conductance channels to achieve the fusion processing of multi - modal signals of images and sounds. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi - modal (image + sound) fusion monitoring method and its hardware device based on a zero - dimensional - two - dimensional hybrid - dimensional floating - gate optoelectronic synaptic transistor for the real - time identification and protection of endangered animals. Compared with the traditional separate acquisition - transmission - calculation mode, the present invention can complete the initial fusion and identification of multi - modal information at the sensor end, reduce the dependence on communication bandwidth and energy consumption, and still maintain a high identification accuracy through the complementary advantages of different modal information under extreme weather interference or high - noise environments.

[0005] The technical solution provided by the present invention is as follows:

[0006] The zero - dimensional - two - dimensional hybrid - dimensional floating - gate optoelectronic synaptic transistor includes a support substrate, a two - dimensional insulating substrate, source - drain electrodes, a channel layer, a tunneling layer, a floating - gate layer, a gate dielectric layer, and a top - gate electrode. The specific device structure includes:

[0007] (1) The supporting substrate is SiO2 / Si of any thickness, quartz glass, or hafnium oxide.

[0008] (2) The two-dimensional insulating substrate is hexagonal boron nitride h-BN.

[0009] (3) The channel layer is molybdenum disulfide MoS2, with a thickness of about 2.45 nm.

[0010] (4) The source and drain electrodes are graphene, with a thickness of 2 - 3 layers.

[0011] (5) The tunneling layer is hexagonal boron nitride h-BN, with a thickness of about 9.93 nm.

[0012] (6) The floating gate layer is perovskite quantum dot CsPbBr3, with a thickness of about 20.45 nm.

[0013] (7) The top gate dielectric layer is hexagonal boron nitride h-BN, with a thickness of about 23.38 nm.

[0014] (8) The top gate electrode is graphene, with a thickness of 2 - 3 layers.

[0015] (9) The source-drain electrodes and the top gate electrode are both led out by depositing a double-layer metal electrode of chromium and gold. In the double-layer metal electrode, the upper layer material is gold (with a thickness of 50 nm), and the lower layer material is chromium (with a thickness of 8 nm).

[0016] A method for monitoring endangered animals based on a multi-modal floating gate optoelectronic synaptic transistor, the specific steps include:

[0017] (1) Using the optoelectronic synaptic device as a reservoir, the input timing signals are respectively applied to the multi-modal fusion optoelectronic synaptic transistor in the form of optical and electrical pulses. With the collaborative stimulation of the optical and electrical pulses, a change response of the source-drain current is generated to achieve the fusion processing of multi-modal signals.

[0018] (2) Under extreme weather conditions (such as heavy fog, heavy rain, etc.), the complementarity of visual and auditory signals is used to improve the recognition accuracy of endangered animals.

[0019] (3) Edge processing and recognition are performed on the pictures and sounds of endangered animals, and high-accuracy monitoring results can be obtained without a large amount of data being transmitted back, thereby reducing energy consumption and enhancing monitoring efficiency.

[0020] The beneficial effects of the present invention include:

[0021] (1) The present invention fuses optical and electrical signals on the same device, and uses the light absorption ability of quantum dots and the high carrier mobility of two-dimensional channel materials to achieve collaborative programming of optical and electrical stimuli.

[0022] (2) The present invention uses optical stimulation to simulate visual information and electrical stimulation to simulate auditory information, achieving the synchronous fusion of audiovisual signals at the physical level. Compared with traditional endangered animal monitoring methods, this solution can maintain a high recognition accuracy in adverse weather conditions (such as heavy fog and heavy rain), and complete feature extraction at the hardware level, significantly reducing data redundancy and energy consumption, making it suitable for unattended and low-power field animal monitoring. This result highlights the practical value of multi-sensory fusion for the monitoring of endangered species in complex environments, providing a new idea for subsequent ecological monitoring under extreme conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Shows a schematic diagram of the device structure of the hybrid-dimensional floating-gate optoelectronic synaptic transistor of the present invention.

[0024] Figure 2 Shows the electrical performance of the floating-gate optoelectronic synaptic transistor of the present invention after optical pulse programming. Figure (a) is the transfer characteristic curve after optical writing and electrical erasing, Figure (b) is the optical pulse time-dependent plasticity (wavelength: 405 nm, power density: 0.375 mW cm -2 ), Figure (c) is the optical pulse intensity-dependent plasticity (pulse width: 150 ms), and Figure (d) is the multi-level storage state under optical pulse stimulation.

[0025] Figure 3 Shows the electrical performance of the floating-gate optoelectronic synaptic transistor of the present invention after electrical pulse programming. Figure (a) is the transfer characteristic curve of the gate voltage backscanning, and Figure (b) is the electrical pulse intensity-dependent plasticity.

[0026] Figure 4 Shows the multi-modal test based on the floating-gate optoelectronic synaptic transistor of the present invention. Figure (a) is the reservoir computing network architecture of the hybrid input mode, Figure (b) is the firing demonstration diagram of optical pulses and electrical pulses shown in combinations of 1111, 1110, 1100, and 1000 in the LLEE mode, Figure (c) is the source-drain current response characteristics of the device in combinations of 1111, 1110, 1100, and 1000 (L: 405 nm, 0.375 mW cm -2 , 0.15 s; E: -3.75 V, 0.15 s), and Figure (d) is the conductance state distribution corresponding to 16 different input combinations in the LLEE mode;

[0027] Figure 5 Shows a schematic diagram of the identification and monitoring of endangered animals using the optoelectronic co-programming ability of the floating-gate synaptic transistor and combining multi-modal audiovisual fusion of the present invention.

[0028] Figure 6This is a simulation diagram of endangered animal recognition based on audiovisual fusion in the present invention. Figure (a) shows the comparison of the recognition accuracy of the endangered animal recognition task under single-modal and multi-modal information processing (the sound signal-to-noise ratio is -20 dB; the picture signal-to-noise ratio is 0 dB), and Figure (b) shows the recognition accuracy of the endangered animal recognition task under different signal-to-noise ratio conditions. Detailed implementation method

[0029] Example

[0030] In order to fabricate the required hybrid-dimensional floating-gate optoelectronic synaptic transistor device, the following multi-step preparation scheme was adopted:

[0031] The first step: Obtain the required two-dimensional few-layer materials from high-quality bulk crystals (MoS2, h-BN, and graphite) by mechanical exfoliation. To ensure the integrity and controllability of the few-layer materials during the transfer process, in this paper, stamps of polydimethylsiloxane (PDMS) and polycarbonate (PC) films were used in combination to precisely stack MoS2, h-BN, and Graphite on a clean SiO2 / Si substrate in sequence in a one-step transfer process.

[0032] The second step: Use ultraviolet DMD maskless lithography technology to define the micro-nano electrode pattern on the SiO2 / Si substrate. Subsequently, deposit a Cr / Au (8 nm / 50 nm) metal layer by electron beam evaporation to form functional source / drain electrodes, ensuring good contact between the device and the semiconductor channel.

[0033] The third step: Spin-coat the pre-prepared and purified CsPbBr3 QDs dispersion at a speed of 3000 rpm for 40 seconds to uniformly disperse the quantum dots on the surface of the bottom structure. Then, anneal at 100 °C for 40 minutes to remove the residual n-hexane solvent and improve the density and stability of the quantum dot film.

[0034] The fourth step: Mechanically exfoliate h-BN and graphite using blue tape respectively, directly adhere the blue tape to PDMS and peel it off, and use an optical microscope to find and mark the materials with appropriate size and thickness on PDMS. Then, with the help of PDMS dry transfer technology, stack them step by step on the prepared heterojunction bottom structure of MoS2 / h-BN / CsPbBr3 QDs to construct the gate dielectric layer and control gate layer of the hybrid-dimensional floating-gate optoelectronic synaptic transistor. The precise stacking of this multi-layer structure not only improves the response characteristics of the device under light and electrical stimulation but also lays a good foundation for subsequent device integration and performance optimization.

[0035] Test example

[0036] Under optical or electrical stimulation, the device can achieve multi-level non-volatile storage and synaptic plasticity regulation. The hybrid-dimensional floating-gate optoelectronic synaptic transistor can be programmed by optical pulses and negative gate voltage pulses, and erased by positive gate voltage pulses. The device exhibits a large storage window and a high on-off ratio. At the same time, in terms of neuromorphic characteristics, the device can simulate biological neural synaptic behaviors such as paired-pulse facilitation (PPF), short-term synaptic plasticity (STP), long-term synaptic plasticity (LTP), synaptic potentiation and inhibition, and can simulate various synaptic plasticities by adjusting conditions such as the width, intensity, and frequency of the stimulation pulses.

[0037] In view of the fact that the hybrid-dimensional floating-gate optoelectronic transistor responds to both optical and electrical stimuli simultaneously and exhibits rich nonlinear relaxation characteristics and dynamic behaviors with adjustable scales, the present invention designs a reservoir computing system for multimodal signal processing based on this device. In order to evaluate the performance of the optoelectronic reservoir in terms of multimodal sensing, storage, and computing integration, the source-drain current data of the device under mixed input of 4-bit binary streams in different modes were tested. Taking the "LLEE" mode as an example, it means that the first two pulses are optical pulses (L), and the last two pulses are electrical pulses (E). Each square wave represents an input signal, where the "off" and "on" of the optical or electrical pulse correspond to binary "0" and "1", respectively. When the optical pulse represents "1", its intensity is 0.375 mW cm -2 ², and the pulse width is 0.15 s; when the electrical pulse represents "1", the applied gate voltage is -3.75 V, and the pulse width is 0.15 s. Figure 4 (c) shows the variation of the source-drain current with time caused by different input sequences (combinations of 1111, 1110, 1100, 1000) in the "LLEE" mode. It can be seen that the final source-drain current state depends on both the pulse type, sequence, and the number of inputs.

[0038] Figure 4 (d) shows the conductance state distribution corresponding to 16 different input combinations in the "LLEE" mode under the same initial conditions. It can be observed that the conductance states in different modes show different degrees of separation characteristics inside the reservoir, which mainly benefits from the nonlinear relaxation effect of the reservoir and the memory ability of the input historical information. These characteristics enable different input combinations to form separable conductance state distributions in the high-dimensional feature space, thereby enhancing the robustness of information encoding and improving the accuracy of signal classification. This result indicates that the multimodal reservoir computing based on this optoelectronic synaptic transistor can effectively utilize the dynamic response characteristics of the physical reservoir to achieve efficient processing of multimodal information.

[0039] The present invention realizes the identification of endangered animals with audio-visual integration at the hardware level using optoelectronic synaptic transistors, which has the following advantages compared with the multimodal endangered animal identification implemented by pure software:

[0040] 1. Rich non - linear transformation: Since the optoelectronic synaptic transistor of the present invention can exhibit short - term plasticity under weak light / weak electrical stimulation and enter multiple non - volatile storage states under strong light / strong electrical stimulation, its diverse dynamic responses provide a richer feature mapping ability for the hardware - implemented multi - modal fusion system, improving the recognition accuracy.

[0041] 2. Diversity of hardware network nodes: In the present invention, the response patterns of each optoelectronic synaptic transistor node under light illumination, voltage pulses, and noise conditions are different, forming a natural high - dimensional dynamic reservoir with a stronger ability to characterize time - series data. Under the same scale, compared with the mode that relies on a large number of convolutional or recurrent layers in traditional neural networks, the network complexity required by the present invention is lower.

[0042] 3. Lower power consumption and real - time monitoring: Traditional multi - modal fusion algorithms usually require huge computing resources and data backhaul, resulting in high power consumption and high latency. The present invention directly completes audio - visual feature extraction and recognition at the hardware level, reducing or eliminating a large amount of data transmission and redundant calculations, significantly reducing the overall energy consumption, and is suitable for unattended endangered animal monitoring scenarios.

[0043] 4. High - efficiency recognition and interference tolerance: In extreme weather or strong noise interference environments, the recognition of images and sounds alone is vulnerable to influence; using the multi - modal hardware fusion of the present invention can effectively compensate for the deficiencies of a single modality and maintain the accurate recognition of the images and acoustic features of endangered animals. The actual test results show that the hardware system constructed by the present invention can be comparable to or even superior to a multi - layer perceptron and a recurrent neural network of the same scale in terms of recognition efficiency, and has better robustness to noise and environmental interference.

[0044] Therefore, compared with traditional multi - modal algorithms, the present invention has comprehensive advantages such as high recognition accuracy, low power consumption, and good real - time performance in the recognition of endangered animals, providing a more practical solution for field monitoring and ecological protection.

[0045] Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make many possible changes and modifications to the technical solution of the present invention, or modify it into an equivalent embodiment with equivalent changes, without departing from the scope of the technical solution of the present invention. Therefore, any simple modification, equivalent change, and modification made to the above - mentioned embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the protection of the technical solution of the present invention.

Claims

1. An endangered animal monitoring method based on a multi-modal floating-gate optoelectronic synaptic transistor, and its specific steps include: (1) The floating-gate optoelectronic synaptic transistor is composed of zero-dimensional quantum dots (CsPbBr3 QDs) as the floating-gate layer, two-dimensional material (MoS2) as the channel layer and tunneling dielectric layer, and external metal electrodes, and can realize the response to optical and electrical stimuli; (2) The picture signal is input into the floating-gate optoelectronic synaptic transistor after being encoded by optical pulses, and the sound signal is input into the optoelectronic transistor after being encoded by electrical pulses; (3) Using the source-drain current of the device as the reservoir for multi-modal input, when an optical pulse or an electrical pulse is applied to the device, the source-drain current changes in real time with the input of the two modes, realizing the fusion of multi-modal signals; (4) Read the conductance change of the floating-gate optoelectronic synaptic transistor under different modal inputs in real time, and perform the recognition and judgment of the target animal according to the conductance change.

2. The audiovisual fusion device based on the zero-dimensional-two-dimensional hybrid dimensional floating gate optoelectronic synaptic transistor according to claim 1, wherein The zero-dimensional quantum dot (CsPbBr3 QDs) floating-gate layer generates photo-generated carriers under illumination and couples with the two-dimensional material (MoS2) channel layer to realize non-volatile storage of visual information or behaviors such as synaptic plasticity.

3. The audiovisual fusion device based on the zero-dimensional-two-dimensional hybrid dimensional floating gate optoelectronic synaptic transistor according to claim 1, characterized in that, The two-dimensional material (MoS2) channel layer combines with external sound electrical signals to realize the dynamic modulation of electrical pulses and change the stored charges in the floating-gate layer, thereby completing the processing of sound information.

4. The audiovisual fusion device based on the zero-dimensional-two-dimensional hybrid-dimension floating-gate optoelectronic synaptic transistor according to claim 1, wherein, The readout and processing module includes hardware or software algorithms for neuromorphic computing or reservoir computing, linearly reads the output current or conductance from the floating-gate optoelectronic synaptic transistor, and extracts multi-modal features, so as to realize the fusion recognition of the target animal image and sound features.

5. The audiovisual fusion device based on the zero-dimensional-two-dimensional hybrid-dimension floating-gate optoelectronic synaptic transistor according to claim 1, characterized in that The device is used to identify the species and behaviors of wild animals, especially suitable for the monitoring of endangered animals in extreme weather or noisy environments, and can operate in an embedded or unattended state.

6. The audiovisual fusion device based on the zero-dimensional-two-dimensional hybrid-dimension floating-gate optoelectronic synaptic transistor according to claim 1 or 5, characterized in that, Under the mixed stimulation of optical and electrical signals, the floating-gate optoelectronic synaptic transistor has a highly sensitive multi-level adjustable conductance state, and can still maintain the recognition accuracy of the fusion of image and sound information through the complementarity of visual signals and sound signals under noise interference.

7. The audiovisual fusion device based on the zero-dimensional-two-dimensional hybrid-dimension floating-gate optoelectronic synaptic transistor according to claim 1 or 6, characterized in that, Through the function of sensing, storing and computing integration of the floating-gate optoelectronic synaptic transistor, the image and sound data can be preprocessed and fused locally, greatly reducing the amount of data transmission back, and realizing low-power monitoring and protection of endangered animals.

Images