Optical memristor and intelligent monitoring method based on optical memristor

Abstract

The application provides an optical memristor and an intelligent monitoring method based on the optical memristor. The method comprises the following steps: building a plurality of edge nodes in different monitoring areas, and deploying a multi-modal sensor array, an optical memristor array, a GCN model and a classifier in each edge node; collecting multi-modal data by the multi-modal sensor array, and establishing a graph structure; inputting the graph structure into the GCN model for propagation, and in the propagation process, accelerating the calculation by using the optical memristor array, inputting the output of the GCN model into the classifier, detecting the abnormal behavior of the monitoring area of the current edge node, and performing local physical alarm; and finally, cloud synchronization, whether to send warning information to the mobile terminal through the cloud. The intelligent monitoring method based on the optical memristor can directly embed the GCN model into the edge device based on the neuromorphic computing architecture based on the optical memristor, and utilize the analog computing characteristics of the optical memristor to realize real-time inference while reducing energy consumption.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent monitoring, and particularly relates to an optical memristor and an intelligent monitoring method based on the optical memristor. BACKGROUND

[0002] In the evolution process of intelligent monitoring technology, the "memory wall" problem of the traditional von Neumann architecture leads to a data transmission energy consumption ratio as high as 85%, and the video analysis system based on CNN faces a processing delay of more than 300 ms when processing complex scenes (such as multi-target tracking, cross-modal data fusion). The graph convolutional network (GCN) models the spatio-temporal association in the monitoring scene (such as the crowd flow network and the device sensor interaction) through the graph convolution operation, and its message passing mechanism can effectively capture the high-order dependency relationship between nodes. However, the GCN inference implemented by the traditional digital circuit faces the challenge of low computing energy efficiency ratio when deployed on edge devices, and a single graph convolution operation consumes about 10^9 floating point operations, which makes the device endurance the key factor restricting its large-scale application. SUMMARY

[0003] In view of the above situation, the main purpose of the present application is to provide an optical memristor and an intelligent monitoring method based on the optical memristor to solve the above technical problems.

[0004] The present application provides an optical memristor, which comprises a substrate, a waveguide layer, an isolation layer and an active layer, the substrate, the waveguide layer and the isolation layer are sequentially stacked, and a plasmonic nano-antenna layer is arranged between the waveguide layer and the isolation layer, the active layer is made of a phase change material, the active layer is located on the isolation layer, and electrodes are arranged on both sides of the isolation layer.

[0005] The present application also provides an intelligent monitoring method based on the optical memristor, which adopts the above-mentioned optical memristor, and the method comprises the following steps:

[0006] Step 1, several edge nodes are built in different monitoring areas, and a multi-modal sensor array, an optical memristor array, a GCN model and a classifier are deployed in each edge node;

[0007] Step 2, multi-modal data is collected by the multi-modal sensor array, and a graph structure is established according to the function information of the module components involved in the edge node, the sensor type information in the multi-modal sensor array and the data type in the preprocessed multi-modal data;

[0008] Step 3, input the graph structure into the GCN model for propagation, and use the optical memristor array to accelerate the calculation in the propagation process to obtain the output of the GCN model;

[0009] Step 4, input the output of the GCN model into a classifier to detect abnormal behavior of the current edge node monitoring area;

[0010] Step 5, synchronize the multi-modal data to the cloud through an incremental backup mechanism, and according to the abnormal behavior detection result, determine whether to issue a physical alarm for the current edge node monitoring area, and whether to send a warning message to the mobile terminal through the cloud.

[0011] Compared with the prior art, the beneficial effects of the present application are as follows:

[0012] (1) Dynamic real-time analysis capability: Traditional monitoring systems rely on predefined rules or offline models, making it difficult to cope with real-time changes in complex scenarios. The intelligent monitoring method based on photorememberable resistors in the present application combines GCN and photorememberable resistor monitoring devices, enabling in-situ storage and parallel computing of data through photorememberable resistor arrays, while utilizing the graph structure learning capability of GCN to dynamically capture spatiotemporal correlations in videos (such as crowd flow patterns and abnormal behavior trajectories).

[0013] (2) Multi-modal data fusion and precise positioning: Traditional methods often result in high false positive rates due to single feature extraction. The intelligent monitoring method based on photorememberable resistors in the present application supports parallel processing of multi-modal data (such as images, spectra, and environmental parameters) through the synergy of GCN and photorememberable resistors: photorememberable resistor arrays can simulate synaptic plasticity and efficiently encode multi-dimensional information such as light intensity and color; GCN integrates deep correlations of different modal data through message passing between nodes.

[0014] (3) Low-power edge intelligence deployment: Traditional monitoring devices rely on cloud computing, resulting in high energy consumption and dependence on networks. The intelligent monitoring method based on photorememberable resistors in the present application can directly embed GCN models into edge devices through neuromorphic computing architectures based on photorememberable resistors, utilizing the analog computing characteristics of photorememberable resistors to reduce energy consumption while enabling real-time inference. For example, a vision system combining all-optical photorememberable resistors with GCN can operate continuously for several weeks without external power supply, making it suitable for remote areas or emergency monitoring scenarios, and through dynamic weight adjustment, it can adapt to different lighting conditions, avoiding the frequent calibration problems of traditional cameras due to environmental changes.

[0015] (4) The optical memristor structure of the present application can realize a nanoscale contraction structure, and the nanoscale contraction structure can perform a thermal confinement effect, so that Joule heat is concentrated in a local area of the phase change material, heat diffusion is reduced, the energy required for phase change is reduced, the electrical switching energy can be as low as a picojoule level, the optical switching energy is also low, and power consumption is greatly reduced. The rapid phase change characteristics of the phase change material itself, combined with the precise thermal management structure, enable the amorphization speed of the device to be tens of nanoseconds, and the crystallization speed to be within a few hundred nanoseconds, which can meet the demand of high-speed signal processing and calculation. The structure has both memristor and optical modulation functions, and can be used to build an optical memristor to realize photoelectric fusion information storage and processing.

[0016] Additional aspects and advantages of the present application will be made apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 It is a schematic diagram of the optical memristor structure of the present application;

[0018] Figure 2 It is a flowchart of the intelligent monitoring method based on the optical memristor of the present application;

[0019] Figure 3 It is a framework diagram of each edge node;

[0020] Figure 4 It is a schematic diagram of a graph structure;

[0021] Figure 5 It is a schematic diagram of a GCN model structure;

[0022] Figure 6 It is a workflow diagram of a GCN model;

[0023] Figure 7 It is a schematic diagram of an optical memristor array structure;

[0024] Figure 8 It is a schematic diagram of two mapping schemes of a GCN model on a Crossbar cross array architecture;

[0025] Figure 9 It is a schematic diagram of a CMOS peripheral circuit and an optical memristor array structure;

[0026] In the figure, 1. substrate, 2. waveguide layer, 3. isolation layer, 41. metal electrode layer, 42. auxiliary electrode layer, 5. active layer, 6. cover layer, 7. plasmonic nanoantenna layer, 8. van der Waals hetero layer, 9. heterojunction photovoltaic layer. DETAILED DESCRIPTION

[0027] Embodiments of the present application are described below in detail with reference to the drawings, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary only, and are used to explain the present application, and should not be understood as limiting the present application.

[0028] These and other aspects of embodiments of the present application will be more apparent from the following description and accompanying drawings. In these descriptions and drawings, some specific implementations of embodiments of the present application are specifically disclosed to represent the principles of implementing embodiments of the present application, but it should be understood that the scope of embodiments of the present application is not limited thereto.

[0029] Embodiment 1

[0030] Referring to Figure 1 The present embodiment provides an optical memristor, which comprises a substrate 1, a waveguide layer 2, an isolation layer 3, and an active layer 5. The active layer is made of a phase change material. The substrate, the waveguide layer, and the isolation layer are sequentially stacked. The active layer is located on the isolation layer, and electrodes 4 are provided on both sides of the isolation layer.

[0031] In the present embodiment, the active layer is made of Ge2Sb2Te5. Through photoelectric pulse induction, reversible phase change between "crystalline state (low resistance) and amorphous state (high resistance)" is achieved. The resistance change is used to store optical signals in intelligent monitoring scenes, and in-situ signal processing is supported.

[0032] The substrate is used for structural support, providing physical support for the entire optical memristor and ensuring the stability of the interlayer structure. Thermal management foundation: the silicon substrate has a certain thermal conductivity, which can assist in dissipating heat during device operation, preventing heat accumulation in the phase change material nano antenna layer, and causing thermal runaway.

[0033] The waveguide layer is used for signal directional transmission, accurately transmitting optical signals in intelligent monitoring scenes to the ion plasma nano antenna layer and the phase change material, avoiding light scattering loss, and ensuring high fidelity of optical signal processing.

[0034] The isolation layer is used for electrical isolation and leakage current suppression: blocking interlayer leakage current, stabilizing the resistance state of the phase change material (avoiding false triggering caused by leakage current), and ensuring the accuracy of optical signal processing in intelligent monitoring. Thermal management cooperation; quickly export the excess heat of the upper functional layer, prevent heat accumulation caused by phase change disorder, and ensure the stability of the device during continuous optical signal processing.

[0035] In the embodiment, considering the cost, the structure can be adapted to low-performance materials, the waveguide layer can be made of titanium dioxide, indium phosphide, gallium nitride, aluminum nitride or silicon, etc., and the embodiment is made of single crystal silicon; the electrode can be made of a general conductive metal material, and the embodiment is made of copper; the substrate can be made of aluminum oxide, silicon carbide or silicon dioxide, etc., and the embodiment is made of silicon dioxide; and the isolation layer can be made of silicon nitride, boron nitride, silicon dioxide or aluminum nitride, etc., and the embodiment is made of boron nitride.

[0036] As a preferred embodiment of the application, the active layer is externally covered by a cover layer 6, and the cover layer is made of ZnS-SiO2.

[0037] The structure is based on the characteristics of the phase change material. The phase change material has two stable states: crystalline state and amorphous state, and the resistance and optical characteristics of the two states are significantly different. By applying a pulse voltage between the electrodes, the phase change material is heated by the Joule heating effect. When the temperature exceeds the melting point, the material is quickly cooled and changes to an amorphous state; when the temperature is near the crystallization temperature and is maintained for a certain period of time, the material will change to a crystalline state. The crystalline phase change material has the characteristics of low resistance and high optical transmittance, and the amorphous phase change material is the opposite, which realizes the function of memristor and optical signal modulation. The working mode is to apply pulse voltages of different amplitudes, widths and quantities on the electrodes. The voltage pulse makes the current pass through the phase change material, generating Joule heat. When the pulse energy is high enough, the local temperature of the phase change material rapidly rises above the melting point (about 890K), and then rapidly cools down, the material changes to an amorphous state, the resistance increases, and the optical transmittance decreases; if a lower energy pulse is applied, the temperature is maintained near the crystallization temperature, the material changes from an amorphous state to a crystalline state, the resistance decreases, and the optical transmittance increases. The silicon (Si) waveguide is used to transmit the optical signal, and when the light passes through the phase change material region, the transmission, reflection and other characteristics of the light will change due to the phase change of the phase change material, thereby realizing the modulation of the optical signal.

[0038] Design advantage: nanoscale shrinkage structure realizes thermal confinement effect, which makes joule heat concentrated in the local area of the phase change material, reduces heat diffusion, and reduces the energy required for phase change. The electrical switching energy can be as low as picojoule level, and the optical switching energy is also low. The power consumption is greatly reduced. The fast phase change characteristics of the phase change material itself, combined with precise heat management structure, make the amorphization speed of the device reach tens of nanoseconds, and the crystallization speed is within hundreds of nanoseconds, which can meet the demand of high-speed signal processing and calculation. The structure has the functions of memristor and optical modulation at the same time, and can be used to build an optical memristor to realize the information storage and processing of photoelectric fusion. The silicon-based material and preparation process adopted are compatible with the existing complementary metal oxide semiconductor (CMOS) process, which is convenient for large-scale integrated manufacturing, reduces production cost, and is conducive to industrialization and popularization. In some power-sensitive monitoring scenarios, the optical memristor can be in a low-power standby state for a long time, and only when there is a change in the optical signal or an electrical pulse is received, the state switching and signal processing are performed, prolonging the device endurance time.

[0039] As a preferred embodiment of the present application, the bottom of the electrode is connected with the isolation layer, and one end of the electrode is close to the active layer. Although the above-mentioned effect can also be achieved under this non-embedded structure, under this structure, the electrode acts as an external heater, and the heat is transmitted to the phase change material through heat conduction, which will cause a large range of heat diffusion.

[0040] In order to ensure better effect, on the basis of the bottom of the electrode being connected with the isolation layer, one end of the electrode is embedded in the active layer. This embedded structure can form a nanoscale constraint structure, which concentrates joule heat in a narrow area of the phase change material through geometric constraints, and the heat distribution overlaps the optical field accurately, so that the size of the phase change area can be controlled to realize multi-level switching. Moreover, the heat constraint effect is remarkable, the electrical switching energy is ultra-low, and the energy consumption per modulation depth is low. The design of embedding the electrode in the phase change material can be integrated with the optical waveguide to reduce optical loss through optimization of the coupling interface and support photoelectric dual-mode reading. The external electrode may introduce additional optical loss, or due to insufficient heat-optical field overlap, leading to low photoelectric coupling efficiency.

[0041] Please refer to Figure 1 The embodiment provides an optical memristor, which comprises a substrate 1, a waveguide layer 2, an isolation layer 3 and an active layer 5, the substrate, the waveguide layer and the isolation layer are sequentially stacked, and a plasmonic nanoantenna layer 7 is arranged between the waveguide layer and the isolation layer, the active layer is made of a phase change material, the active layer is located on the isolation layer, and electrodes are arranged on both sides of the isolation layer.

[0042] In the embodiment, the active layer adopts Ge2Sb2Te5, realizes reversible phase change of "crystalline state (low resistance) and amorphous state (high resistance)" through photoelectric pulse induction, stores the optical signal in the intelligent monitoring scene by utilizing resistance change, and supports in-situ signal processing.

[0043] The substrate is used for structural support, provides physical support for the entire optical memristor, and ensures the stability of the interlayer structure. Thermal management foundation: the silicon substrate has a certain thermal conductivity, which can assist in dissipating the heat generated during device operation, preventing the accumulation of heat in the phase change material nano antenna layer, leading to thermal runaway.

[0044] The waveguide layer is used for directional transmission of signals, accurately transmitting optical signals in intelligent monitoring scenes to the ion plasma nano antenna layer and the phase change material, avoiding light scattering loss, and ensuring high fidelity of optical signal processing.

[0045] The isolation layer is used for electrical isolation and leakage current suppression: blocking interlayer leakage current, stabilizing the resistance state of the phase change material (avoiding false triggering caused by leakage current), and ensuring the accuracy of optical signal processing in intelligent monitoring. Thermal management cooperation; quickly export the excess heat of the upper functional layer to prevent heat accumulation from causing phase change disorder and ensure the stability of the device during continuous optical signal processing.

[0046] In this embodiment, the plasmonic nano antenna layer uses a gold nanorod + VO2 phase change material composite array. The gold nanorod compresses the photoelectric pulse light field to the nanoscale region through the surface plasmon effect, highly concentrates the light energy, improves the light-to-phase conversion efficiency, allows weak light to drive phase change, and ensures effective operation at all times. VO2 phase change material switches between insulating state and metallic state with temperature or electrical signal, dynamically adjusts the antenna resonance frequency, matches the monitoring multi-spectral requirements of visible light during the day and near-infrared light at night, and can also actively avoid strong light interference through electrical pulse to prevent false triggering. This layer cooperates with the waveguide layer to directionally transmit and optimize the optical signal, and cooperates with the phase change material to make the phase change triggering more sensitive and energy consumption lower, and is connected in series with the "light collection-processing-output" whole process.

[0047] Due to cost considerations, this structure can be adapted to lower performance materials. The waveguide layer can be made of titanium dioxide, indium phosphide, gallium nitride, aluminum nitride, or silicon, etc. In this embodiment, the waveguide layer is made of single crystal silicon, the electrode can be made of a general conductive metal material, and the substrate can be made of aluminum oxide, silicon carbide, or silicon dioxide, etc. In this embodiment, the substrate is made of silicon dioxide, and the isolation layer can be made of silicon nitride, boron nitride, silicon dioxide, or aluminum nitride, etc. In this embodiment, the isolation layer is made of boron nitride.

[0048] As a preferred embodiment of the present application, the bottom of the electrode is connected to the isolation layer, and one end of the electrode is close to the active layer. Although this non-embedded structure can also achieve the above effects, in this structure, the electrode acts as an external heater, and the heat is transferred to the phase change material through thermal conduction, which can cause a large range of heat diffusion.

[0049] In order to ensure better effect, the electrode is embedded into the active layer at one end on the basis of the connection of the electrode bottom and the isolation layer. The embedded structure can form a nano-level constraint structure, the Joule heat is concentrated in a narrow area of the phase change material through geometric constraint, the heat distribution is accurately overlapped with the light field, the size of the phase change area can be controlled, and multi-level switching is realized. Moreover, the heat constraint effect is remarkable, the electric switching energy is ultra-low, and the energy consumption per modulation depth is low. The design of embedding the electrode into the phase change material can be integrated with the optical waveguide, the optical loss is reduced through optimizing the coupling interface, and the optoelectronic dual-mode reading is supported. The external electrode can introduce additional optical loss, or due to insufficient heat-light field overlap, the optical-electric coupling efficiency is low.

[0050] As a preferred embodiment of the present application, the electrode comprises a metal electrode layer 41 and an auxiliary electrode layer 42, wherein the metal electrode layer adopts copper, and the auxiliary electrode layer adopts titanium nitride. In the working process, on the basis that the metal electrode layer bears the core electric signal writing and reading function, the auxiliary electrode layer focuses on the environmental adaptation and weak light enhancement task. The metal electrode layer adopts copper, which has extremely low resistivity, can efficiently transmit electric signals, and reduce the energy loss and delay of signals in the electrode transmission process; at the same time, the metal has excellent chemical stability and is not easy to be oxidized and corroded in a complex environment, so that the light memory resistor can work stably for a long time, and meet the needs of long-term reliable operation of intelligent monitoring equipment. The auxiliary electrode layer adopts titanium nitride (TiN) as the auxiliary electrode layer material, which can convert weak light signals into local heat through the photo-thermal effect in weak light environment such as night light and overcast day, provide additional energy for the metal electrode layer for the phase change material, widen the response threshold of the light memory resistor to weak light signals, and enable the equipment to work normally under low illumination conditions; at the same time, the stable chemical properties can inhibit electrode oxidation and improve the service life and reliability of the light memory resistor in harsh environments.

[0051] As a preferred embodiment of the present application, the active layer is sequentially covered with a van der Waals hetero layer 8 and a covering layer 6. The van der Waals hetero layer is a heterojunction formed by stacking two-dimensional transition metal chalcogenide MoS2 and WSe2 through van der Waals force. The photoelectric response can be enhanced; MoS2 has high response rate to 1550nm light (common communication waveband of intelligent monitoring), can quickly absorb photoelectric pulses and generate carriers; WSe2 regulates carrier lifetime and transport path, accurately injects carriers into the phase change material, accelerates light-phase change conversion, and improves the real-time performance of intelligent monitoring. The covering layer adopts ZnS-SiO2, which can be physically protected; the internal functional layer is protected from environmental erosion.

[0052] As a preferred embodiment of the present application, a heterojunction photovoltaic layer 9 is covered on the bottom of the substrate. The heterojunction photovoltaic layer is formed by a heterojunction of silicon (Si) and perovskite, which is self-powered and adapted to absorb ambient light (such as sunlight, street lights, moonlight) to generate electricity and provide continuous and low-power electrical energy support for the photoreccurrent device. The photoelectricity cooperates to initialize: the generated electricity can be used to initialize the resistance state of the photoreccurrent device, so that the device is always in a "ready state to quickly respond to photoelectric pulses".

[0053] The structure is based on the characteristics of phase change materials. Phase change materials have two stable states: crystalline and amorphous, and their resistance and optical properties differ significantly in the two states. By applying a pulse voltage between the electrodes, the phase change material is heated using the Joule heating effect. When the temperature exceeds its melting point, the material quickly cools and changes to an amorphous state; when the temperature is near the crystallization temperature and is maintained for a certain period of time, the material will change to a crystalline state. Crystalline phase change materials have low resistance and high optical transmission, while amorphous phase change materials are the opposite, and the characteristics are used to realize the function of the memristor and the modulation of the optical signal. The working mode is to apply pulse voltages of different amplitudes, widths and quantities on the electrodes. The voltage pulse makes the current pass through the phase change material, generating Joule heat. When the pulse energy is high enough, the local temperature of the phase change material rises rapidly above the melting point (about 890K), and then cools rapidly, the material changes to an amorphous state, the resistance increases, and the optical transmission decreases; if a lower energy pulse is applied, the temperature is maintained near the crystallization temperature, the material changes from amorphous to crystalline, the resistance decreases, and the optical transmission increases. Silicon (Si) waveguide is used to transmit optical signals, and the transmission, reflection and other characteristics of the light will change when it passes through the phase change material region due to the phase change of the phase change material, thereby realizing the modulation of the optical signal.

[0054] The nanoscale contraction structure realizes the thermal confinement effect, which concentrates the Joule heat in the local area of the phase change material, reduces the heat diffusion, and reduces the energy required for phase change. The electrical switching energy can be as low as picojoules, and the optical switching energy is also low, and the power consumption is greatly reduced. The fast phase change characteristics of the phase change material itself, combined with precise thermal management structure, make the amorphization speed of the device reach tens of nanoseconds, and the crystallization speed is within a few hundred nanoseconds, which can meet the demand of high-speed signal processing and calculation. The structure has both memristor and optical modulation functions, and can be used to build a photoreccurrent device to realize photoelectric fusion information storage and processing. The silicon-based material and preparation process adopted are compatible with the existing complementary metal oxide semiconductor (CMOS) process, which is convenient for large-scale integrated manufacturing, reduces production cost, and is conducive to industrialization and popularization. In some power-sensitive monitoring scenarios, the photoreccurrent device can be in a low-power standby state for a long time, and only when there is a change in the optical signal or an electrical pulse is received will the state switching and signal processing be performed, prolonging the device's endurance time.

[0055] And adopt MoS2 / WSe2 van der Waals hetero layer and Ge2Sb2Te5 phase change material coordination, improve the speed of light-phase change conversion; Gold nanorod and VO2 composite form plasmonic antenna, realize light field localization and spectral adaptive control, weak light driving efficiency is improved and anti strong light interference; Integrated perovskite / Si heterojunction photovoltaic layer, realize environmental light self power and device initialization, get rid of external power supply dependence; Titanium nitride auxiliary electrode and gold electrode constitute double electrode structure, give consideration to low resistance signal transmission and photo-thermal assisted phase change, widen the working light intensity range and improve the stability in harsh environment; From waveguide layer to phase change material, form the closed loop of optical signal processing, optimize the whole process of transmission-enhancement-conversion, adapt to 1000fps high-speed video processing, through the system integration of materials and functions, realize the design of intelligent monitoring special light memristor in all time, anti-interference, self-powered, break through the functional limitation of traditional devices. The structure is innovated as follows: through "layered function integration + cross-layer coupling regulation + micro-nano precise matching", a "self-powered-light transmission-electric regulation-thermal management-light response" integrated three-dimensional structure is constructed, so that the physical structure is no longer a simple stack, but a "functional carrier" of light-electric-thermal multi-field cooperation.

[0056] Please refer to Figure 2 The embodiment also provides an intelligent monitoring method based on a light memristor. The method uses the light memristor described above, and the method comprises the following steps:

[0057] Step 1. Build several edge nodes in different monitoring areas, and deploy a multi-modal sensor array, a light memristor array, a GCN model (graph convolutional neural network) and a classifier in each edge node.

[0058] In this step, the light memristor is introduced and combined with the GCN model. The light memristor is a new non-volatile storage and computing element, which has the advantages of fast response speed, low energy consumption and high integration, and has great potential in intelligent monitoring systems. By introducing the light memristor, the computing efficiency and storage density of the system are further improved. The working principle of the light memristor is based on the interaction between light and matter, and its conductance state can be dynamically adjusted by light intensity to realize light-controlled storage and calculation. This feature enables the light memristor to balance between high-speed data processing and low-power storage, making it particularly suitable for real-time monitoring and data analysis tasks.

[0059] In the GCN-light memristor cooperative system, the light memristor is used to optimize the weight update and feature extraction process. Specifically, the light signal is inputted, the weight adjustment and feature calculation are performed through the light memristor array, the high-speed transmission characteristics of light are utilized, and the data processing delay is significantly reduced. At the same time, the low-power consumption characteristics of the light memristor help to prolong the running time of the system and improve the overall energy efficiency.

[0060] In addition, the introduction of the optical memristor also enhances the environmental adaptability of the system. In complex and variable monitoring scenarios, the optical memristor can utilize the sensitivity of the optical signal to external environmental changes to achieve more accurate weight adjustment and feature recognition, thereby improving the robustness and accuracy of the system.

[0061] Referring to Figure 3 As a preferred embodiment of the present application, the edge node further has a control unit, a Tile buffer, a data preprocessing module and an IMA unit array arranged therein;

[0062] The control unit is configured to coordinate data flow, allocate IMA unit tasks, and regulate overall processing flow; by receiving sensor array data, the control unit schedules data to the preprocessing module and the IMA unit array;

[0063] The data preprocessing module is configured to preprocess multi-modal data, such as format normalization, noise filtering, etc., and transmit the preprocessed standardized data to the Tile buffer;

[0064] The Tile buffer is configured to temporarily store preprocessed multi-modal data, alleviate the speed difference between data transmission and IMA unit calculation, and distribute data to the input registers of the IMA unit array according to the scheduling of the control unit;

[0065] The signal adaptation interface is configured to distribute preprocessed multi-modal data to the IMA unit array and adapt the input requirements of different modal signals (such as optical signals and electrical signals);

[0066] The IMA unit array is configured to store and calculate preprocessed multi-modal data;

[0067] The IMA unit array includes a plurality of IMA units, and the optical memristor array adopts a Crossbar cross array structure and is arranged in the IMA unit.

[0068] The IMA unit includes a digital-to-analog converter, an optoelectronic memristor converter, a sample-and-hold unit, an input register and an output register, a shift-add unit, a ReLU activation unit, an addressing unit, and an alarm and storage module;

[0069] The optoelectronic memristor converter is configured to convert optical signals into electrical signals or convert electrical signals into optical signals;

[0070] The digital-to-analog converter is configured to convert digital signals into analog signals to adapt to memristor calculation;

[0071] The analog-to-digital converter is configured to convert analog signals into digital signals for storage and output;

[0072] The optical memristor array is configured to: perform a multiply-add operation (such as a node aggregation calculation in a GCN model) in an integrated storage and calculation manner by using a conductance characteristic (corresponding to an adjacency matrix or a weight matrix value) of the optical memristor;

[0073] The sample-and-hold device is configured to: stabilize an output of the optical memristor array to generate a stabilized analog signal; prevent distortion caused by signal fluctuation by freezing a transient value of a current signal, ensure calculation accuracy, and provide a stable input for a subsequent accumulation operation of a shift-add unit to avoid a timing mismatch problem;

[0074] The input register and the output register are configured to: temporarily store input data and calculation results;

[0075] The shift-add unit is configured to: perform an analog signal accumulation operation on the stabilized analog signal to generate an accumulated analog signal;

[0076] The ReLU activation unit is configured to: perform a nonlinear activation process on the accumulated analog signal, and transmit an activated signal to the output register or the alarm and storage module;

[0077] The addressing unit is configured to: manage addresses of data in the optical memristor array, optimize access efficiency of a sparse graph data by a GCN model, and improve calculation speed;

[0078] The GCN coordination and control sub-module is configured to: rely on a graph convolutional network (GCN) algorithm, utilize an optical memristor array capability, cooperatively process visual data (images, target motion information, etc.) collected by a multi-modal sensor, fuse node data and mine correlations when tracking multiple targets, optimize calculation, and improve accuracy and efficiency of intelligent visual task processing (from a low order to a high order);

[0079] The CMOS routing and scheduling word module is configured to: connect the optical memristor array and external devices (input and output registers, digital-to-analog converters, analog-to-digital converters, etc.) of a system, plan a signal path, and efficiently circulate visual data; and according to a task type (simple recognition / complex tracking), schedule CMOS resources, coordinate array timing, and ensure an order of a sensing, storage, and calculation process;

[0080] The optical memristor material characteristic configuration module is configured to: adjust a material charge distribution by regulating a working mode (dynamic / non-volatile, etc.) of the optical memristor according to an intelligent visual task by using an optoelectronic excitation (light / voltage pulse, etc.), and adapt basic characteristics to sensing, storage, and calculation of visual information;

[0081] The alarm and storage module is configured to: process a final result, trigger a local physical alarm if an abnormality is detected, synchronize multi-modal data to a cloud through an incremental backup mechanism, and decide whether to send an alert information to a mobile terminal according to the result.

[0082] 1. Key interactive interfaces of the optical memristor conversion module

[0083] 1. The input and output relationship between the optical memristor conversion module and the external module is as follows:

[0084] 1.1 Input:

[0085] Optical signals: Optical channels or optical communication modules from multimodal sensor arrays.

[0086] Electrical signals: analog voltages from digital-to-analog converters (DACs) (such as node characteristic voltages) and pulse signals from control units.

[0087] 1.2 Output:

[0088] Electrical signal: The current signal is output to the sample-and-hold device, or the digital signal is transmitted to the output register through the analog-to-digital converter (ADC).

[0089] Optical signal: The modulated optical signal is output through the waveguide layer to the optical detector or communication link.

[0090] 2. The collaborative process between the optical memristor conversion module and the internal modules of the IMA unit is as follows:

[0091] 2.1. Interaction with the digital-to-analog converter: The DAC converts digital signals (such as the weight parameters of the GCN model) into analog voltages, which are input into the row ends of the optical memristor array to drive the calculation.

[0092] 2.2. Interaction with the sample-and-hold device: After the current signal output by the optical memristor array is stabilized by the sample-and-hold device, it enters the shift-and-add unit for accumulation.

[0093] 2.3. Interaction with the control unit: The control unit sends a sequence of voltage pulses to regulate the phase state (crystalline / amorphous) of the phase change material and dynamically adjust the conductance of the memristor (such as updating the adjacency matrix weights).

[0094] 2. Signal Interaction Process of Optical Memristor Conversion Module

[0095] 1. Conversion path from optical signal to electrical signal

[0096] 1.1. Optical signal input:

[0097] Source: Optical sensors in a multimodal sensor array (e.g., camera, spectrometer) or external optical communication links.

[0098] Transmission: The light field is guided to the active layer through the silicon waveguide layer, where it interacts with the phase change material.

[0099] Photoelectric conversion process

[0100] 1.2 Mechanism:

[0101] The crystalline / amorphous state of the phase-change material has different transmittance of light (high transmittance in crystalline state, low transmittance in amorphous state), and the light signal is modulated when passing through, causing the material's electrical conductivity to change.

[0102] 1.4, Output: The change in electrical conductivity is converted into an electric current signal, which is output to the sample holder through the electrode.

[0103] 1.5, Electric signal processing:

[0104] Sample holder: stabilizes the current signal to prevent fluctuations (such as "stable memristor output signal").

[0105] Subsequent flow: transmitted to the shift-add unit, participating in node aggregation calculation of the GCN model (such as matrix multiplication).

[0106] 2, Conversion path of electric signal to optical signal

[0107] 2.1, Electric signal input:

[0108] Source: analog voltage output by digital-to-analog converter (DAC), or control signal of control unit.

[0109] Processing: voltage is applied to the phase-change material through the electrode, generating Joule heat and driving the material to change phase (crystalline to amorphous or vice versa).

[0110] 2.2, Memristor characteristic regulation:

[0111] Mechanism:

[0112] High-voltage pulse: makes the temperature of the phase-change material exceed the melting point, and after rapid cooling, it turns into an amorphous state, increasing resistance and reducing light transmittance.

[0113] Low-voltage pulse: temperature maintained at crystallization temperature, material turns into crystalline state, resistance decreases, light transmittance increases.

[0114] Output: phase change causes changes in light transmittance of waveguide layer, realizing modulation of optical signal (such as light intensity, phase control).

[0115] 2.3, Optical signal output:

[0116] Destination: transmitted to external optical detector through waveguide layer, or used for optical communication link (such as cloud-synchronized optical signal transmission).

[0117] 3. Collaborative computing process of optical memristor conversion module and GCN model

[0118] 3.1, Graph structure mapping:

[0119] Adjacency matrix: mapped to the conductivity distribution of the optical memristor array, each memristor conductivity corresponds to the edge weight in the matrix.

[0120] Node characteristics: converted into analog voltages and applied to the row ends of the memristor array (as described in the document "Node characteristics are converted into analog voltages and applied to the row ends of the optical memristor array").

[0121] 3.2. The calculation process of storage and computing integration:

[0122] The voltage is input at the row end, multiplied and added through the memristor conductance (corresponding to the adjacency matrix or weight matrix), and the current (intermediate result) is output at the column end.

[0123] After the current is stabilized by the sample-and-hold device, it is input into another memristor array (corresponding to the weight matrix or adjacency matrix) again to complete the node aggregation of the GCN.

[0124] 3.3. Output: The feature vector of the GCN model is transmitted to the classifier for abnormal behavior detection.

[0125] See also Figure 3 , the TILE area on the left: consists of a multimodal sensor array, a control unit, a Tile buffer, a data preprocessing module, a signal adapter interface, and an array composed of multiple IMA units. The internal structure of the IMA on the right: includes a digital-to-analog converter, a photo-memristor array, a photoelectric memristor converter, a sample-and-hold unit, an input register and an output register, a shift-and-add unit, a ReLU activation unit, an addressing unit, a photo-memristor material property configuration module, a CMOS routing scheduling submodule, a GCN collaborative control submodule, and an alarm and storage module. It works as follows: first, the multimodal sensor array collects external data and transmits it to the data preprocessing module for formatting, normalization, and other operations; the processed data is dispatched by the control unit and enters the array composed of multiple IMA units on the left.

[0126] Inside the right IMA unit, after the processed data enters the register for temporary storage, each module cooperates in sequence to complete the storage and calculation and control closed loop: the addressing unit as the "spatial positioning center", after receiving the instruction of the control unit, accurately locates the cross-point address used for calculation in the optical memory resistor array, and feeds back the positioning result to the control unit to synchronize the calculation timing, solving the address confusion problem caused by the sparsity of graph data; the optical memory resistor material characteristic configuration module is the "dynamic parameter adjustment core", according to the task type issued by the control unit, the phase change state (crystalline / amorphous) of the optical memory resistor is adjusted through the photoelectric pulse to change the conductance characteristic to adapt to the numerical requirement of the adjacent matrix or the weight matrix, and the resistance curve after adjustment is returned to the control unit to confirm the validity, realizing the hardware adaptation of "one device with multiple functions"; the CMOS routing scheduling submodule assumes the "path planning role", according to the task priority allocated by the control unit, the data in the input register is shunted to different processing paths, simple calculation tasks are directly guided to the shift-add unit, and complex multi-modal fusion tasks are processed through the GCN cooperative control submodule, and the load state of each path is fed back to the control unit in real time to avoid congestion; the GCN cooperative control submodule as the "multi-array cooperative engine", based on the graph convolution algorithm, the large-scale graph calculation task is decomposed into subtasks, which are allocated to multiple optical memory resistor arrays for parallel processing of optical signal conversion into electrical signal, while the calculation progress of each array is synchronized and returned to the control unit, ensuring the timing consistency of cross-array cooperation.

[0127] In addition, after the photoelectric memory resistor converter completes the conversion of optical and electrical signals, the digital signal is converted into analog voltage to drive the optical memory resistor array through the digital-to-analog converter, the current signal output by the array is stabilized by the sample and hold unit, the accumulation operation is completed by the shift-add unit, the nonlinear transformation is realized by the ReLU activation unit, and finally the result is stored in the output register and synchronized to the alarm and storage module. In the whole process, the control unit coordinates the data flow, IMA task allocation and system operation, and the Tile buffer temporarily stores data to match the transmission and calculation speed. This working mode realizes overall optimization in energy consumption, speed, accuracy and scalability through the optical memory resistor storage and calculation integrated architecture, multi-module cooperation and flexible scheduling.

[0128] Step 2, collecting multi-modal data through a multi-modal sensor array, and establishing a graph structure according to the function information of the module components involved in the edge node, the sensor type information in the multi-modal sensor array, and the data type in the preprocessed multi-modal data;

[0129] As a preferred embodiment of the present application, the step of establishing a graph structure according to the function information of the module components involved in the edge node, the sensor type information in the multi-modal sensor array, and the data type in the preprocessed multi-modal data specifically includes the following steps:

[0130] Preprocess the multi-modal data to obtain preprocessed multi-modal data;

[0131] According to the function information of the module components involved in the edge node, the sensor type information in the multi-modal sensor array, and the data type in the preprocessed multi-modal data, a function module node, a sensor node, and a data type node are established;

[0132] The sensor node and the corresponding data type node are connected by an edge, the data type node and the corresponding processing function module node are connected by an edge, and the sensor performance, data importance, and system feedback information are used as the edge weights, to obtain a graph structure.

[0133] In this step, the adjacency matrix is the core component of the graph neural network, and its construction and training are directly related to the performance of the intelligent monitoring system. In this embodiment, the GCN model is constructed based on the multi-dimensional data processing graph structure, uses the adjacency matrix as the input, and captures the correlation information between nodes through graph convolution operation. Specifically, the GCN model adopts a multi-layer structure, each layer contains a certain number of neurons, and introduces non-linear characteristics through the ReLU activation function. During model training, the cross-entropy loss function is used to measure the gap between the prediction result and the true label, and the Adam optimizer is used for parameter update to realize the fast convergence and performance optimization of the model. Through the design of the GCN model, the intelligent monitoring system can deeply mine the spatio-temporal correlation and abnormal behavior patterns in the monitoring scene, and provide strong feature support for subsequent abnormal behavior classification.

[0134] Step 3, input the graph structure into the GCN model for propagation, and in the propagation process, use the optical memristor array to accelerate calculation to obtain the output of the GCN model;

[0135] Please refer to Figure 4 to Figure 9 , as a preferred embodiment of the present application, the graph structure is input into the GCN model for propagation, and in the propagation process, the optical memristor array is used to accelerate calculation to obtain the output of the GCN model, which specifically includes the following steps:

[0136] According to the edges in the graph structure, an adjacency matrix is constructed, and a weight matrix is constructed according to the edge weights;

[0137] Map the adjacency matrix and the weight matrix into two independent optical memristor arrays respectively, and the conductance of each optical memristor in the two independent optical memristor arrays corresponds to the numerical value of the adjacency matrix and the weight matrix respectively;

[0138] Input the graph structure into the GCN model for node aggregation, and in the node aggregation process, convert the node features into an analog voltage and apply it to the row end of any one of the two independent optical memristor arrays for operation, and the column end of the optical memristor array outputs a current to obtain an intermediate result;

[0139] The intermediate result is applied to another row end of the optical memristor array for operation, the column end of the optical memristor array outputs current, and sampling is performed to obtain the output of the GCN model.

[0140] The GCN model design is as shown in Figure 5 . First, the graph structure is input (as shown in Figure 4 , including nodes and corresponding feature vectors, i.e., node features), in this embodiment, the nodes are divided into sensor nodes, data type nodes, and function module nodes. The connection relationship between nodes is described by an adjacency matrix A. If there is a direct connection between node i and node j (such as sensor i generating data type j), a non-zero weight is assigned; if there is no connection, A i,j =0; then enter the node aggregation stage, calculate using the adjacency matrix A, the current layer feature matrix X （l） , and the weight matrix W （l） , aggregate the feature information of the node itself and its neighbors; finally, after the feature extraction operation, the updated feature vector X （l+1） is output, completing the feature transformation of one layer of graph convolution. The following is the feature extraction process of a 7x3 weight matrix by A, B, C, D, E, F, and G seven nodes, which provides feature representation for subsequent large-scale data processing tasks.

[0141] The GCN model design constructs a graph structure based on the spatiotemporal correlation of sensors, the nodes correspond to cameras, and the edge weight is dynamically calculated by spatial distance (Euclidean distance) and temporal correlation (Pearson coefficient); a two-layer GCN network (128 neurons per layer, ReLU activation) is used for feature extraction, trained by cross-entropy loss and Adam optimizer (learning rate 0.001), and finally outputs the feature to a support vector machine (SVM) classifier to realize abnormal behavior recognition. This design dynamically adjusts the synaptic weight through the storage-computing integrated characteristics of optical memristors, integrates multi-modal data to improve recognition accuracy, reduces energy consumption, and supports long-term offline operation of edge devices. Its core advantages include real-time dynamic adaptation to complex scenarios, multi-dimensional data fusion, and green low-power architecture. The workflow diagram is as shown in Figure 6 .

[0142] Among them, GCN-Crossbar is designed to address the problems of frequent memory access caused by large input data, irregular calculation characteristics, sparse graph data, and poor data locality when the GCN model processes graph structure data. By analyzing the calculation and memory access characteristics of the GCN model operands, the weights and adjacency matrices are mapped to the optical memristor array. The mapping process is as follows: in the node aggregation stage of the GCN model, the input feature vector is calculated with the weight matrix stored in the optical memristor array, and the multiplication and accumulation operations are completed in the array through the storage and calculation characteristics of the optical memristor. The highly sparse adjacency matrix is ​​optimally divided into sub-matrices and compressed mapping, and the COO compressed format feature vector input is supported to achieve sparse computing. The accelerator finally constructed (such as RGCN) has significant advantages in speed improvement and energy saving compared to CPU and GPU, providing a high-computing power and low-power computing platform for graph neural networks. Its structure Figure 7 As shown, Figure 7 The multiplication and addition operation mechanism using the Crossbar array is shown in the figure: the word line input voltage V1, V2, through the optical memristor G 11 , G 12 Components such as I1=V1G output current at the bit line end 11 +V2G 21 ), completing the multiplication and addition calculation of voltage and conductance - this process is the basis for the optical memristor to realize storage and computing integration. It can perform calculations while storing data (represented by conductance state), greatly reducing the energy consumption and time of data transfer.

[0143] Figure 8 In the figure, two mapping schemes of the GCN model are shown on the Crossbar cross array architecture. T represents the transpose of the eigenvector, A T represents the transpose of the adjacency matrix, (*) T Indicates that the matrix transpose operation is performed on the output part. Figure 8 As can be seen from a, the first option is to first T and the transpose of the adjacency matrix A T The elements in A correspond to the input voltage and conductance of the optical memristor array, and the intermediate result is obtained through the array calculation. After that, the intermediate result is calculated and output with the weight matrix W. In this process, the optical memristor array not only stores the adjacency matrix A and the weight matrix W, but also uses its multiplication and addition operation capability to efficiently complete the operation between the matrices. Figure 8 As can be seen in b, the second solution is to first process the feature matrix X and the weight matrix W. Similarly, using the optical memristor array, the elements of X are used as input voltages and the elements of W are used as conductances, and the intermediate results are obtained through the multiplication and addition operation of the optical memristor. Then, this intermediate result is combined with the transpose A of the adjacency matrix A. TThe operation output is performed. In this way, the calculation process of the GCN is optimized, and the mapping strategy of the GCN operation on the cross-array hardware is embodied. In order to fuse the multi-modal monitoring of images, sounds and environmental parameters (different modal feature matrices X need to be processed in parallel). Figure 8 The mapping scheme of the middle b first performs weight transformation on the feature matrix X (X·W). Different modal features (such as temperature and humidity data of the image sensor) can be mapped to different input channels of the opto-memristive array, and feature extraction can be completed in parallel. Then, the adjacency matrix transpose A T Integrate cross-modal correlation. Implement "dynamic weight update" + "multi-modal fusion", so, select Figure 8 The mapping scheme of the middle b.

[0144] As shown in Figure 9 The opto-memristive array converts the optical signal into a resistance variable signal containing spatial distribution information, and the CMOS transmission gate is activated to select the target node (corresponding to the activation of the GCN node). The operational amplifier regulates and converts the analog-digital signal to realize the connection between the analog-digital signals. The feature encoding unit adds spatial coordinates, connection states and other attributes to the digital signal to construct the graph node features and topological relations required by the GCN. The GCN module performs graph convolution based on the physical graph structure to mine the spatial correlation of the optical signal and output the control strategy. The control feedback circuit converts the strategy into a hardware control signal to reversely adjust the resistance state of the opto-memristive array and the on-off state of the CMOS transmission gate, so that the system adapts to the optical environment, realizes the intelligent closed loop from optical perception to hardware dynamic optimization, has the advantages of low delay and high energy efficiency, and is suitable for intelligent imaging.

[0145] An opto-memristive device (integrating light-sensitive and resistance variable characteristics) is used in combination with a CMOS transmission gate matrix. A single unit has the functions of optical signal perception (light intensity to resistance state mapping) and programmable computing node. Without software intervention, the construction of "optical spatial distribution-physical graph structure" is directly completed at the hardware layer to realize the hardware integration of perception, storage and calculation. Through the feature encoding unit, the resistance variable signal, physical coordinates and connection state of the opto-memristive device are directly converted into input features recognizable by a graph convolution network (GCN) to construct a direct mapping between the physical space and the algorithm space. The control instructions output by the GCN are reversely affected by the feedback circuit on the hardware to dynamically adjust the resistance state of the opto-memristive device (optimize the light-sensitive characteristics) and the on-off state of the CMOS transmission gate (reconstruct the node connection), so that the system has the task-driven hardware self-optimization capability and adapts to complex environments. The opto-memristive array supports multi-physical quantity perception expansion, shares the CMOS gating and GCN computing resources, and natively realizes multi-modal graph computation. At the same time, the gating mechanism of the CMOS transmission gate naturally supports hardware-level graph sparsification calculation, thereby realizing the deep cooperation between the opto-memristive array and the GCN module, and constructing a "perception-computation-optimization" closed loop.

[0146] Step 4, input the output of the GCN model into a classifier to detect abnormal behaviors in the current edge node monitoring area;

[0147] Step 5, synchronizing the multi-modal data to the cloud through the incremental backup mechanism, and determining whether to issue a physical alarm for the current edge node monitoring area and whether to issue a warning message to the mobile terminal through the cloud according to the abnormal behavior detection result.

[0148] It should be understood that although each step in the flowchart of each embodiment of the present application is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in each embodiment can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or sub-steps or stages of other steps.

[0149] It should be understood that each part of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-described embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized by hardware, as in another embodiment, it can be realized by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic function on data signal, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0150] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0151] The above-described embodiments only express several implementation manners of the present application, which are described in detail and specifically, but cannot be understood as a limitation on the scope of the patent of the present application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, which are within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An optical memristor, characterized in that, The application relates to a substrate, a waveguide layer, an isolation layer and an active layer, the substrate, the waveguide layer and the isolation layer are sequentially stacked, an isolated nano antenna layer is arranged between the waveguide layer and the isolation layer, the active layer is made of a phase change material, the active layer is arranged on the isolation layer, and electrodes are arranged on both sides of the isolation layer; wherein the isolation layer is made of silicon nitride, boron nitride, silicon dioxide or aluminum nitride, the active layer is made of Ge2Sb2Te, and the isolated nano antenna layer is made of a gold nanorod+VO2 phase change material composite array.

2. The optical memristor of claim 1, wherein, The bottom of the electrode is connected with the isolation layer, and one end of the electrode is close to the active layer. 3.The optical memristor of claim 1, wherein, The bottom of the electrode is connected with the isolation layer, and one end of the electrode is embedded in the active layer.

4. The optical memristor according to any one of claims 2 or 3, wherein, The active layer is externally covered with a van der Waals hetero layer and a covering layer; wherein the van der Waals hetero layer is a heterojunction formed by stacking two-dimensional transition metal chalcogenide MoS2 and WSe2 through van der Waals force.

5. The optical memristor of claim 4, wherein, A heterojunction photovoltaic layer is covered on the bottom of the substrate.

6. An intelligent monitoring method based on an optical memristor, using the optical memristor of any one of claims 1 to 5, characterized in that, The method comprises the following steps: Step 1, several edge nodes are built in different monitoring areas, and a multi-modal sensor array, an optical memristor array, a GCN model and a classifier are deployed in each edge node; Step 2, multi-modal data is collected through the multi-modal sensor array, and a graph structure is established according to the function information of the module components involved in the edge node, the sensor type information in the multi-modal sensor array and the data type in the preprocessed multi-modal data; Step 3, input the graph structure into the GCN model for propagation, and use the optical memristor array to accelerate calculation in the propagation process to obtain the output of the GCN model; Step 4, input the output of the GCN model into the classifier to detect the abnormal behavior of the current edge node monitoring area; Step 5, synchronize the multi-modal data to the cloud through the incremental backup mechanism, and according to the abnormal behavior detection result, decide whether to send a physical alarm to the current edge node monitoring area and whether to send a warning information to the mobile terminal through the cloud. 7.The intelligent monitoring method based on optical memristor according to claim 6, wherein, In the step 2, the graph structure is established according to the function information of the module components involved in the edge node, the sensor type information in the multi-modal sensor array and the data type in the preprocessed multi-modal data, which comprises the following steps: Preprocess the multi-modal data to obtain preprocessed multi-modal data; According to the function information of the module components involved in the edge node, the sensor type information in the multi-modal sensor array and the data type in the preprocessed multi-modal data, establish function module nodes, sensor nodes and data type nodes; Connect the sensor nodes and the corresponding data type nodes with edges, connect the data type nodes and the corresponding processing function module nodes with edges, and take the sensor performance, data importance and system feedback information as the edge weights of the corresponding edges to obtain the graph structure. 8.The intelligent monitoring method based on the optical memristor according to claim 7, wherein, In the step 3, the graph structure is input into the GCN model for propagation, and the optical memristor array is used for acceleration calculation in the propagation process to obtain the output of the GCN model, which comprises the following steps: According to the edges in the graph structure, construct an adjacency matrix, and construct a weight matrix according to the edge weights. The adjacency matrix and the weight matrix are mapped to two independent optical memristor arrays, respectively. The conductance of each optical memristor in the two independent optical memristor arrays corresponds to the value of the adjacency matrix and the weight matrix respectively. The graph structure is input into the GCN model for node aggregation. During the node aggregation process, the node features are converted into analog voltages and applied to the row ends of any of the two independent optical memristor arrays for calculation. The column ends of the optical memristor arrays output current to obtain intermediate results. The intermediate result is then applied to the row end of another optical memristor array for calculation. The column end of the optical memristor array outputs current and is sampled to obtain the output of the GCN model. 9.The intelligent monitoring method based on the optical memristor according to claim 8, wherein, The edge node also has a control unit, tile buffer, data pre-processing module, and IMA unit array deployed; Control unit, used to: coordinate data flow and task allocation; Data preprocessing module, used to: preprocess multimodal data; Tile buffer, used to: temporarily store pre-processed multimodal data; Signal adapter interface, used to: distribute pre-processed multimodal data to the IMA unit array; IMA unit array, used to: store and perform calculation operations on pre-processed multimodal data; The IMA unit array includes a plurality of IMA units. The optical memristor array adopts a crossbar array structure and is arranged in the IMA unit.

Images