Photon memristor for audio-visual dual-mode identification, preparation method, storage pool mapping method and application

Abstract

The invention belongs to the technical field of electric bionic synaptic devices, and particularly relates to a photon memristor for audio-visual dual-mode identification, a preparation method, a storage pool mapping method and application. The memristor comprises a bottom electrode, a hole layer, a light absorption layer, a passivation layer and a top electrode which are sequentially arranged from bottom to top, the hole layer is a ZnO thin film, the light absorption layer is a Cs2AgBiBr6 thin film, and the ZnO thin film and the Cs2AgBiBr6 thin film form a heterojunction; the passivation layer is a PMMA film. According to the photon memristor provided by the invention, the conductivity value can be regulated and controlled through ultraviolet pulse programming, and the conductivity state can provide abundant storage state space for physical storage pool calculation. In letter and audio recognition, the accuracy rates of 91.7% and 95.0% can be respectively realized based on physical storage pool calculation of the photon memristor. The application of the method can greatly improve the application of the photoelectric bionic synaptic device in audio-visual multi-mode identification, and provides a referable device instance for the implementation of physical storage pool calculation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of electric bionic synapse devices, and particularly relates to a photonic memristor for audio-visual bimodal recognition, a preparation method, a reservoir mapping method and application. BACKGROUND

[0002] Vision and hearing are the two most important senses for human beings to perceive the world, accounting for more than 90% of environmental information intake, and are the basis for cognition, learning and interaction. It is undeniable that endowing machines with similar sensory processing capabilities is crucial for achieving accurate perception and autonomous decision-making in complex environments, which is a key step in developing true artificial intelligence.

[0003] Building a bio-inspired computing system based on photonic synapses is crucial for overcoming the computational bottlenecks of traditional von Neumann computing architecture and promoting the development of artificial intelligence towards low energy consumption and high response speed. However, existing photonic synapse devices are mainly applied in the field of artificial vision, and there is a lack of effective reservoir mapping methods for their combination with artificial hearing, which greatly limits their application development in multi-modal brain-like computing. SUMMARY

[0004] To solve the above problems, the application provides a photonic memristor for audio-visual bimodal recognition, a preparation method, a reservoir mapping method and application, which is based on the capture of photo-generated carriers by a hole layer, the efficient absorption of photons by an absorbing layer, and the passivation of perovskite defects by a passivation layer, forming a memristor sensitive to optical signals. The conductance value of the memristor can be regulated by ultraviolet light pulse programming, and different conductance states can provide rich reservoir space for physical reservoir computing, which is used to map high-dimensional input vectors to low-dimensional outputs. In the letter and audio recognition tasks, the physical reservoir computing based on the photonic memristor can achieve an accuracy of up to 91.7% and 95.0%, respectively. The application of this method can greatly improve the application of optoelectronic bionic synapse devices in audio-visual multi-modal recognition, and provides a reference device example for the implementation of physical reservoir computing.

[0005] The application solves the above technical problems through the following technical solutions.

[0006] The first object of the application is to provide a photonic memristor for audio-visual bimodal recognition, which comprises, from bottom to top along the thickness direction, a bottom electrode, a hole layer, an absorbing layer, a passivation layer and a top electrode, the hole layer is a ZnO thin film for capturing photo-generated carriers; the absorbing layer is a Cs2AgBiBr6 thin film for absorbing photons; the ZnO thin film and the Cs2AgBiBr6 thin film form a heterojunction; the passivation layer is a PMMA thin film for passivating perovskite defects; the bottom electrode is a conductive glass, and the top electrode is a conductive metal; the photonic memristor adjusts the current state characteristics by adjusting the ultraviolet light pulse during programming, thereby adjusting the responsivity of the memristor.

[0007] Further, the thickness of the hole layer is 70nm-110nm, the thickness of the light absorption layer is 120nm-200nm, and the thickness of the passivation layer is 60nm-100nm.

[0008] Further, the bottom electrode is an FTO electrode, and the top electrode is an Ag electrode.

[0009] A second object of the present application is to provide a preparation method of the above-mentioned photonic memristor for audio-visual dual-modal recognition, comprising the following steps: S1. A conductive glass is provided as a bottom electrode, and a ZnO thin film is deposited on the bottom electrode under an oxygen-free atmosphere to form a hole layer.

[0010] S2. A Cs2AgBiBr6 thin film is prepared on the hole layer by a two-step spin coating method to obtain a light absorption layer.

[0011] S4. A metal is deposited on the passivation layer to obtain a top electrode, thereby obtaining the photonic memristor for audio-visual dual-modal recognition.

[0012] A third object of the present application is to provide a reservoir mapping method of the above-mentioned photonic memristor for audio-visual dual-modal recognition, comprising the following steps: S1. A voltage is applied between the top electrode and the bottom electrode of the photonic memristor, and an ultraviolet light pulse sequence is applied to the photonic memristor to output a current response curve of the photonic memristor. S2. The current response curve is feature-sampled to obtain a main feature SMP1 and a secondary feature SMP2. The main feature SMP1 is an average value of all sampling point currents in a sampling parameter as the SMP1 value, and the sampling parameter is a sampling delay of 0 seconds and a sampling time of 5 seconds. The sampling parameter of the secondary feature SMP2 is to directly sample the current value at the 13th second. After testing the 4-bit ultraviolet light pulse sequence, 4-bit SMP1 main feature values and 4-bit SMP2 secondary feature values are obtained, respectively. S3. A physical reservoir computing system is constructed based on the main feature SMP1 and the secondary feature SMP2. The physical reservoir computing system includes an input layer, a physical reservoir layer, and an output layer. The input layer receives the 4-bit ultraviolet light pulse sequence, the physical reservoir layer maps the results represented by SMP1 and SMP2, and the output layer is trained under a supervised learning framework using a linear classifier and a Softmax output function. S4. The physical reservoir computing system is applied to train and recognize letter data sets and audio data sets. The signal features of the letter data sets and the audio data sets are all encoded as multiple groups of 4-bit binary codes, and each group corresponds to a string of 4-bit ultraviolet light pulse sequences.

[0013] Further, the recognition of the letter data set comprises: performing binaryzation on the original image, obtaining a binary matrix, performing maximum pooling using a 2*2 kernel to reduce the dimension, and flattening the output after pooling into a one-dimensional vector, from which a plurality of groups of 4-bit binary codes are extracted as inputs of the input layer.

[0014] Further, the recognition of the audio data set comprises: performing pre-emphasis processing, windowing, framing, FFT transformation, MEL filtering and MFCC feature extraction on the initial audio signal, calculating the mean, standard deviation, maximum value and minimum value after obtaining the MFCC feature, and quantizing them to logic 1 or 0 according to whether they exceed the median, and finally generating a plurality of groups of 4-bit binary codes as inputs of the input layer.

[0015] Further, the voltage is 0.05V, and is used to read the current response of the photonic memristor.

[0016] Further, the ultraviolet light pulse sequence is a 4-bit pulse sequence, and the light intensity is 12.3mW / cm -2 .

[0017] A fourth object of the present application is to provide the application of the above-mentioned photonic memristor for audio-visual dual-modal recognition in visual and auditory information processing.

[0018] Compared with the prior art, the present application has the following beneficial effects: The memristor provided by the present application comprises, from bottom to top along the thickness direction, a bottom electrode, a hole layer, a light absorption layer, a passivation layer and a top electrode, the hole layer captures photo-generated carriers, the light absorption layer efficiently absorbs photons, and the passivation layer passivates perovskite defects, forming a memristor sensitive to optical signals, a photonic memristor for audio-visual dual-modal recognition is prepared, the rich current state under different light pulse programming is used as the feature mapping of the reservoir layer, and the linear classifier is combined to execute the physical reservoir computing task, and the letter or audio recognition can be performed. This helps to realize multi-modal brain-like computing and provides a reference reservoir model for the development of artificial audio-visual systems. The development of a physical reservoir computing system based on a photonic memristor not only provides an efficient hardware solution for processing multi-modal information, but also significantly reduces the power consumption and time cost of traditional computing architecture in complex pattern recognition tasks. The photonic synapse can accurately regulate the conductance state of the memristor by using different ultraviolet light pulse sequences, thereby generating rich and high-dimensional feature responses in the reservoir layer. This ability to directly convert optical signals into electrical features enables the system to achieve high parallelism in information processing without complex front-end preprocessing circuits. In addition, the device structure is simple, compatible with existing semiconductor processes, and has good scalability and stability, providing an important device foundation for the development of low-power, high-intelligence embedded perception systems in the future. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Current response of the photonic memristor of the embodiment 1 and the comparative example 1 to 2 of the present application under the stimulation of a single ultraviolet light pulse.

[0020] Figure 2 Current response graphs of the photonic memristor of the present application induced by four representative light pulse programming, Figure 2 In the figure, a is the current response graph of the photonic synapse induced by four representative light pulse programming (“0011”, “1011”, “1111” and “0010”), and b is the current characteristic sampling method of the device.

[0021] Figure 3 Main feature SMP1 and secondary feature SMP2 output result graphs of the photonic memristor prepared by the present application under the stimulation of 4-bit ultraviolet light pulse,

[0022] Figure 4 Training and recognition graphs of the physical reservoir computing system constructed by the present application on the letter data set, Figure 4 In the figure, a is the recognition graph of the letter “j”, and b is the training process graph of the readout network.

[0023] Figure 5 Training and recognition graphs of the physical reservoir computing system constructed by the present application on the audio data set, Figure 5 In the figure, a is the audio processing flowchart, and b is the training and recognition graph of the readout network (audio “0-9” recognition task). DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0025] It should be noted that the professional terms used in the present application are only for the purpose of describing the specific embodiments, and are not intended to limit the scope of protection of the present application. Unless otherwise specified, the various raw materials, reagents, instruments and equipment used in the following embodiments of the present application can be purchased from the market or prepared by the existing method.

[0026] In one aspect, the application provides a photonic memristor for audio-visual bimodal recognition, the memristor comprises, from bottom to top, a bottom electrode, a hole layer, a light absorption layer, a passivation layer and a top electrode along the thickness direction, the hole layer is a ZnO thin film for capturing photo-generated carriers, the light absorption layer is a Cs2AgBiBr6 thin film for absorbing photons, the ZnO thin film and the Cs2AgBiBr6 thin film form a heterojunction, the passivation layer is a PMMA (polymethyl methacrylate) thin film for passivating perovskite defects, the bottom electrode is a conductive glass, and the top electrode is a conductive metal; the photonic memristor adjusts the current state characteristics by adjusting the ultraviolet light pulse during programming, so as to adjust the responsivity of the memristor.

[0027] In the application, the bottom electrode serves as a support substrate and can be made of a rigid material such as conductive glass to achieve mechanical stability, and in a preferred embodiment, the bottom electrode is an FTO electrode; the top electrode is a metal electrode, and in a preferred embodiment, the top electrode is an Ag electrode, which guarantees signal acquisition accuracy due to its high conductivity, wherein the hole layer is a ZnO thin film deposited by magnetron sputtering in an oxygen-deficient atmosphere for capturing photo-generated carriers; the light absorption layer is a Cs2AgBiBr6 thin film prepared by a two-step spin coating method for efficient absorption of photons; and the passivation layer is a PMMA thin film prepared by a two-step spin coating method for passivating perovskite defects. The memristor provided by the application is a bionic photonic synapse device, the current of which can be regulated by an applied ultraviolet light pulse sequence to realize multiple current states and give the physical reservoir layer a rich reservoir space. The memristor can be directly used as a reservoir layer for physical reservoir computing, and in combination with a linear classifier, it can perform letter and audio recognition tasks.

[0028] In some embodiments, the thickness of the hole layer is 70 nm to 110 nm, the thickness of the light absorption layer is 120 nm to 200 nm, and the thickness of the passivation layer is 60 nm to 100 nm.

[0029] In another aspect, the application provides a preparation method of the above-mentioned photonic memristor for audio-visual bimodal recognition, comprising the following steps: S1, providing a conductive glass as a bottom electrode, depositing a ZnO thin film on the bottom electrode in an oxygen-deficient atmosphere to form a hole layer.

[0030] The conductive glass is sequentially placed in acetone, ethanol and deionized water for ultrasonic treatment for 20 min to 30 min before use, and then the surface of the FTO glass is blown dry with N2 as the bottom electrode of the photonic memristor.

[0031] The ZnO thin film is deposited by magnetron sputtering, a ZnO target is installed on an alternating target of a magnetron sputtering machine, the target-substrate distance is set to 10 cm to 14 cm, the base vacuum degree of the sputtering chamber is extracted to less than 4×10 -4Pa, pure argon gas with a purity of 99.999% is introduced as a working atmosphere, the argon gas flow is 10 sccm during sputtering, the gas pressure in the sputtering cavity is controlled at 0.6 Pa to 0.8 Pa, the sample table rotation speed is kept at 10 rpm / min, and the sputtering power is 60 W / cm 2 , and the sputtering time is 60 min.

[0032] S2, a Cs2AgBiBr6 film is prepared on the hole layer in a two-step spin coating manner to obtain an absorbing layer.

[0033] The two-step spin coating manner is specifically that 200 μL of Cs2AgBiBr6 precursor is dropped on the surface of the ZnO film on a spin coater, chlorobenzene, a reverse solvent, is dropped at 10 s before the spin coating operation is completed to accelerate perovskite crystallization, and finally annealing is performed at 280°C for 5 min.

[0034] The preparation method of the Cs2AgBiBr6 precursor is to dissolve CsBr (0.852 g), AgBr (0.376 g) and BiBr3 (0.916 g) in 4 mL of DMSO (dimethyl sulfoxide). The two-step spin coating parameters are running at 1000 r / min for 10 s and then running at 4000 r / min for 50 s.

[0035] S3, a PMMA film is prepared on the absorbing layer in a two-step spin coating manner to obtain a passivation layer.

[0036] The two-step spin coating manner is specifically that 200 μL of PMMA precursor is dropped on the surface of the Cs2AgBiBr6 film on a spin coater, and then annealing is performed at 180°C for 5 min. The two-step spin coating parameters are running at 2000 r / min for 10 s and then running at 6000 r / min for 40 s.

[0037] The preparation method of the PMMA precursor is to dissolve 0.025 g of PMMA in 5 mL of chlorobenzene.

[0038] S4, a conductive metal Ag is deposited on the passivation layer to obtain a top electrode; and a photonic memristor for audio-visual dual-mode recognition is obtained.

[0039] The deposition electrode Ag is in a magnetron sputtering manner, specifically, a metal Ag target is installed on a direct current target of a magnetron sputtering, the target-substrate distance is set to 10 cm to 14 cm, the base vacuum degree of the sputtering chamber is extracted to be lower than 4×10 -4 Pa, pure argon gas with a purity of 99.999% is introduced as a working atmosphere, the argon gas flow is 10 sccm during sputtering, the gas pressure in the sputtering cavity is controlled at 0.6 Pa to 0.8 Pa, the sample table rotation speed is kept at 10 rpm / min, and the sputtering power is 60 W / cm 2The sputtering time is 8 minutes.

[0040] Furthermore, the present invention also provides the above-mentioned cell mapping method for photonic memristors used for audiovisual dual-modal recognition, comprising the following steps: S1. Apply a voltage between the top and bottom electrodes of the photonic memristor and apply a sequence of ultraviolet light pulses to the photonic memristor to output the current response curve of the photonic memristor.

[0041] In this process, the top and bottom electrodes of the photonic memristor are connected to a source meter, ensuring a good and stable connection. A voltage of 0.05V is set to read the current response of the photonic memristor. Then, a specific sequence of ultraviolet light pulses is applied above the photonic memristor, maintained at 12.3 mW / cm². -2 To determine the light intensity, adjust the 4-bit pulse sequence. For example, apply an ultraviolet light pulse with the sequence 1101. The corresponding ultraviolet lamp switching state is: on, on, off, on, to observe the current response of the memristor.

[0042] S2. Perform feature sampling on the current response curve to obtain the main feature SMP1 and the secondary feature SMP2. The main feature SMP1 is the average value of the current at all sampling points within the sampling parameters, with a sampling delay of 0 seconds and a sampling time of 5 seconds. The sampling parameters for the secondary feature SMP2 are the current value directly sampled at the 13th second, without considering the sampling delay and sampling time. After testing the 4-bit ultraviolet light pulse sequence, 4-bit SMP1 main feature value and 4-bit SMP2 secondary feature value are obtained respectively.

[0043] S3. Based on the main feature SMP1 and the secondary feature SMP2, a physical reservoir computing system is constructed. The physical reservoir computing system includes an input layer, a physical reservoir layer and an output layer. The input layer receives a 4-bit ultraviolet light pulse sequence. The mapping result of the physical reservoir layer is represented by SMP1 and SMP2. The output layer is trained under a supervised learning framework using a linear classifier and a Softmax output function.

[0044] S4. The physical storage pool computing system is used to train and recognize the letter dataset and the audio dataset; the signal features of the letter dataset and the audio dataset are encoded into multiple sets of 4-bit binary codes, each set corresponding to a 4-bit ultraviolet light pulse sequence.

[0045] In some embodiments, the recognition of letter datasets includes: binarizing the original image to obtain a binary matrix, performing max pooling with a 2×2 kernel to reduce the dimensionality, flattening the pooled output into a one-dimensional vector, and extracting multiple sets of 4-bit binary codes from it as input to the input layer.

[0046] In some embodiments, the recognition of the audio dataset includes: pre-emphasis processing, windowing, framing, FFT transformation, MEL filtering, and MFCC feature extraction of the initial audio signal. After obtaining the MFCC features, the mean, standard deviation, maximum value, and minimum value are calculated, and the data are quantized to logic 1 or 0 depending on whether they exceed the median. Finally, multiple sets of 4-bit binary codes are generated as input to the input layer.

[0047] This invention presents a physical reservoir computing system based on a photonic memristor, which simultaneously achieves visual and auditory information processing. This facilitates multimodal neuromorphic computing and provides a reference reservoir model for the development of artificial audiovisual systems. Developing a physical reservoir computing system based on photonic memristors not only provides an efficient hardware solution for processing multimodal information but also significantly reduces the power consumption and time cost of traditional computing architectures in complex pattern recognition tasks. The photonic synapse, stimulated by different ultraviolet light pulse sequences, can precisely control the conductance state of the memristor, thereby generating rich and high-dimensional feature responses in the reservoir layer. This ability to directly convert optical signals into electrical features allows the system to achieve high-parallel information processing without complex front-end preprocessing circuits. Furthermore, the device has a simple structure, is compatible with existing semiconductor processes, and possesses good scalability and stability, providing an important device foundation for the future development of low-power, highly intelligent embedded sensing systems.

[0048] A fourth objective of this invention is to provide the application of the aforementioned photonic memristor for audiovisual bimodal recognition in visual and auditory information processing.

[0049] The following specific examples will provide further explanation.

[0050] Example 1 A photonic memristor for audiovisual dual-modal recognition, comprising, from bottom to top, a bottom electrode, a hole layer, a light-absorbing layer, a passivation layer, and a top electrode along its thickness direction. The hole layer is a ZnO thin film, the light-absorbing layer is a Cs₂AgBiBr₆ thin film, the passivation layer is a PMMA thin film, the bottom electrode is FTO conductive glass, and the top electrode is an Ag electrode. The thickness of the hole layer is approximately 90 nm, the thickness of the light-absorbing layer is approximately 150 nm, and the thickness of the passivation layer is approximately 80 nm.

[0051] The above-mentioned method for fabricating a photonic memristor for audiovisual dual-modal recognition includes the following steps: S1. Provide an FTO conductive glass, and immerse it in acetone, ethanol, and deionized water in sequence for 20 minutes of ultrasonication. Then, use N2 to dry the surface of the FTO glass to serve as the bottom electrode.

[0052] S2. A ZnO thin film is deposited on an FTO glass slide using magnetron sputtering. A ZnO target is mounted on the AC target for magnetron sputtering, with a target-to-substrate distance of 10 cm. The background vacuum in the sputtering chamber is evacuated to below 4 × 10⁻⁶. -4 The working atmosphere was 99.999% pure argon gas. During sputtering, the argon flow rate was 10 sccm, the gas pressure inside the sputtering chamber was controlled at 0.6 Pa, the sample stage rotation speed was maintained at 10 rpm / min, and the sputtering power was 60 W / cm². 2 The sputtering time was 60 minutes to obtain a hole layer.

[0053] S3. A Cs2AgBiBr6 film was prepared on the surface of the void layer using a two-step spin coating method. CsBr (0.852 g), AgBr (0.376 g), and BiBr3 (0.916 g) were dissolved in 4 mL of DMSO to obtain the Cs2AgBiBr6 precursor. 200 μL of the Cs2AgBiBr6 precursor was dropped onto the ZnO film surface on a spin coater and run at 1000 r / min for 10 s, then at 4000 r / min for 50 s. 10 s before the end of the spin coating operation, the antisolvent chlorobenzene was added to accelerate the perovskite crystallization. Finally, the film was annealed on a 280℃ heating plate for 5 min to obtain the light-absorbing layer.

[0054] S4. A PMMA film was prepared on the surface of the light-absorbing layer using a two-step spin coating method. 0.025 g of PMMA was dissolved in 5 mL of chlorobenzene to obtain a PMMA precursor. 200 μL of the PMMA precursor was dropped onto the surface of the Cs2AgBiBr6 film using a spin coater, and then annealed on a hot plate at 180 °C for 5 min. The two-step spin coating parameters were 2000 rpm for 10 s and 6000 rpm for 40 s to obtain a passivation layer.

[0055] S5. Electrode Ag is deposited on the passivation layer using magnetron sputtering. A metallic Ag target is mounted on the DC target for magnetron sputtering, with a target-substrate distance of 10 cm. The background vacuum in the sputtering chamber is evacuated to below 4 × 10⁻⁶. -4 The working atmosphere was 99.999% pure argon gas. During sputtering, the argon flow rate was 10 sccm, the gas pressure inside the sputtering chamber was controlled at 0.6 Pa, the sample stage rotation speed was maintained at 10 rpm / min, and the sputtering power was 60 W / cm². 2 The sputtering time was 8 minutes to obtain Ag electrodes. This yielded a photonic memristor for audiovisual dual-modal recognition, named the Ag / PMMA / Cs2AgBiBr6 / ZnO / FTO structured photonic memristor.

[0056] Comparative Example 1 A photonic memristor for audiovisual dual-modal recognition, comprising, from bottom to top, a bottom electrode, a hole layer, a passivation layer, and a top electrode arranged sequentially along the thickness direction. The hole layer is a ZnO thin film, the passivation layer is a PMMA thin film, the bottom electrode is FTO conductive glass, and the top electrode is an Ag electrode. The thickness of the hole layer is approximately 90 nm, and the thickness of the passivation layer is approximately 80 nm.

[0057] The above-mentioned method for fabricating a photonic memristor for audiovisual dual-modal recognition includes the following steps: S1. Provide an FTO conductive glass, and immerse it in acetone, ethanol, and deionized water in sequence for ultrasonication for 20-30 minutes. Then, use N2 to dry the surface of the FTO glass to serve as the bottom electrode.

[0058] S2. A ZnO thin film is deposited on an FTO glass slide using magnetron sputtering. A ZnO target is mounted on the AC target for magnetron sputtering, with a target-to-substrate distance of 10cm to 14cm. The background vacuum in the sputtering chamber is evacuated to below 4×10⁻⁶. -4 Argon gas with a purity of 99.999% was introduced as the working atmosphere. During sputtering, the argon gas flow rate was 10 sccm, the gas pressure in the sputtering chamber was controlled at 0.6 Pa to 0.8 Pa, the sample stage rotation speed was maintained at 10 rpm / min, and the sputtering power was 60 W / cm. 2 The sputtering time was 60 minutes to obtain a hole layer.

[0059] S3. A PMMA film was prepared on the surface of the void layer using a two-step spin coating method. 0.025 g of PMMA was dissolved in 5 mL of chlorobenzene to obtain a PMMA precursor. 200 μL of the PMMA precursor was dropped onto the surface of the void layer using a spin coater, and then annealed on a 180°C hot plate for 5 min. The two-step spin coating parameters were 2000 rpm for 10 s and 6000 rpm for 40 s to obtain a passivation layer.

[0060] S4. Electrode Ag is deposited on the passivation layer using magnetron sputtering. A metallic Ag target is mounted on the DC target for magnetron sputtering, with a target-substrate distance of 10cm to 14cm. The background vacuum in the sputtering chamber is evacuated to below 4×10⁻⁶. -4 Argon gas with a purity of 99.999% was introduced as the working atmosphere. During sputtering, the argon gas flow rate was 10 sccm, the gas pressure in the sputtering chamber was controlled at 0.6 Pa to 0.8 Pa, the sample stage rotation speed was maintained at 10 rpm / min, and the sputtering power was 60 W / cm. 2 The sputtering time was 8 minutes to obtain Ag electrodes. This yielded a photonic memristor for audiovisual dual-modal recognition, named the Ag / PMMA / ZnO / FTO structured photonic memristor.

[0061] Comparative Example 2 A photonic memristor for audiovisual dual-modal recognition, comprising, from bottom to top, a bottom electrode, a light-absorbing layer, a passivation layer, and a top electrode along its thickness direction. The light-absorbing layer is a Cs2AgBiBr6 thin film, the passivation layer is a PMMA thin film, the bottom electrode is FTO conductive glass, and the top electrode is an Ag electrode. The thickness of the light-absorbing layer is approximately 150 nm, and the thickness of the passivation layer is approximately 80 nm.

[0062] The above-mentioned method for fabricating a photonic memristor for audiovisual dual-modal recognition includes the following steps: S1. Provide an FTO conductive glass, and immerse it in acetone, ethanol, and deionized water in sequence for ultrasonication for 20-30 minutes. Then, use N2 to dry the surface of the FTO glass to serve as the bottom electrode.

[0063] S2. A Cs2AgBiBr6 thin film was prepared on the surface of the bottom electrode using a two-step spin coating method. CsBr (0.852 g), AgBr (0.376 g), and BiBr3 (0.916 g) were dissolved in 4 mL of DMSO to obtain the Cs2AgBiBr6 precursor. 200 μL of the Cs2AgBiBr6 precursor was dropped onto the surface of the bottom electrode using a spin coater. The spin coater was run at 1000 r / min for 10 s, and then at 4000 r / min for 50 s. 10 s before the end of the spin coat operation, the antisolvent chlorobenzene was added to accelerate the perovskite crystallization. Finally, the film was annealed on a 280 °C hot plate for 5 min to obtain the light-absorbing layer.

[0064] S3. A PMMA film was prepared on the surface of the light-absorbing layer using a two-step spin coating method. 0.025 g of PMMA was dissolved in 5 mL of chlorobenzene to obtain a PMMA precursor. 200 μL of the PMMA precursor was dropped onto the surface of the Cs2AgBiBr6 film using a spin coater, and then annealed on a hot plate at 180 °C for 5 min. The two-step spin coating parameters were 2000 rpm for 10 s and 6000 rpm for 40 s to obtain a passivation layer.

[0065] S4. Electrode Ag is deposited on the passivation layer using magnetron sputtering. A metallic Ag target is mounted on the DC target for magnetron sputtering, with a target-substrate distance of 10cm to 14cm. The background vacuum in the sputtering chamber is evacuated to below 4×10⁻⁶. -4 Argon gas with a purity of 99.999% was introduced as the working atmosphere. During sputtering, the argon gas flow rate was 10 sccm, the gas pressure in the sputtering chamber was controlled at 0.6 Pa to 0.8 Pa, the sample stage rotation speed was maintained at 10 rpm / min, and the sputtering power was 60 W / cm. 2 The sputtering time was 8 minutes to obtain Ag electrodes. This yielded a photonic memristor for audiovisual dual-modal recognition, named the Ag / PMMA / Cs2AgBiBr6 / FTO structured photonic memristor.

[0066] The current response of the photonic memristors in Example 1 and Comparative Examples 1 to 2 under single ultraviolet light pulse stimulation was tested. Figure 1 The current response of the photonic memristors in Embodiment 1 and Comparative Examples 1 to 2 of the present invention under a single ultraviolet light pulse stimulation is shown. Figure 1 As shown, monolayer zinc oxide devices exhibit postsynaptic current behavior, but the current decays rapidly (Ag / PMMA / ZnO / FTO structure photonic memristor prepared in Comparative Example 1); monolayer Cs2AgBiBr6 devices show no postsynaptic current behavior and cannot simulate biological synaptic characteristics (Ag / PMMA / Cs2AgBiBr6 / FTO structure photonic memristor prepared in Comparative Example 2); the Cs2AgBiBr6 / ZnO heterostructure significantly prolongs the decay time of the postsynaptic current (Ag / PMMA / Cs2AgBiBr6 / ZnO / FTO structure photonic memristor prepared in Example 1), which is very important for the subsequent output of 4-bit distinguishable current states. This can provide rich distinguishable storage states for the physical reservoir layer, ultimately improving the accuracy of image / speech recognition.

[0067] Based on the Ag / PMMA / Cs2AgBiBr6 / ZnO / FTO structure photonic memristor prepared in Example 1, a photonic memristor-based physical reservoir computing system was constructed. Various current states can be generated by stimulating the system with different sequences of ultraviolet light pulses, which can be directly used as the reservoir layer for physical reservoir computing. Combined with a linear classifier, letter and audio recognition tasks can be performed, as detailed below: A cell mapping method for photonic memristors used for audiovisual dual-modal recognition includes the following steps: S1. Connect the top and bottom electrodes of the photonic memristor to the source meter, ensuring a good and stable connection. Set the voltage to 0.05V to read the current response of the photonic memristor. Then, apply a specific sequence of ultraviolet light pulses above the photonic memristor, maintaining a voltage of 12.3mW / cm². -2 The light intensity was adjusted, and a 4-bit pulse sequence was applied. Ultraviolet light pulses with a sequence of 1101 were applied. The corresponding ultraviolet lamp switching status was: on, on, off, on, to observe the current response of the memristor.

[0068] S2. Perform feature sampling on the current response curve to obtain the main feature SMP1 and the secondary feature SMP2. Specifically, the sampling parameters for the main feature SMP1 are a sampling delay of 0s and a sampling time of 5s. The average value of the current at all sampling points within the sampling time is taken as the SMP1 value. The sampling parameters for the secondary feature SMP2 are to sample directly at 13s, without considering the sampling delay and sampling time. After testing the 4-bit ultraviolet light pulse sequence, 4-bit SMP1 main feature value and 4-bit SMP2 secondary feature value can be obtained respectively.

[0069] S3. Using Python-written pooling computation code, the pooling features directly utilize the principal features SMP1 and secondary features SMP2 of the fabricated photonic memristor. Combined with a linear classifier, a physical pooling computation system based on photonic synapses is constructed to train and recognize letter and audio datasets. The physical pooling computation system includes an input layer, a physical pooling layer, and an output layer. The input is a 4-bit ultraviolet light pulse sequence, and the pooling layer mapping result is represented by the readout signals SMP1 and SMP2 of the fabricated photonic memristor. The output layer is trained using a Python program within a supervised learning framework, employing a Softmax output function and a single-layer linear classifier. In the recognition task, the features of both letter and audio signals are encoded as multiple sets of 4-bit binary codes (4×i), each set corresponding to a 4-bit ultraviolet light pulse.

[0070] S4. Training and recognition of the letter dataset using a physical pooling computing system: In the letter "XJTU" recognition task, the original image (28×28 pixels) is first binarized (threshold=127) to obtain a 28×28 binary matrix. Then, max pooling with a 2×2 kernel is performed to reduce the dimension to 14×14. The pooled output is flattened into a one-dimensional vector, and 36 groups of 4-bit binary codes are extracted from it (discarding the last 52 pixels). The "XJTU" letter dataset contains 10,000 training images (2,500 per letter) and 2,000 test images (500 per letter).

[0071] S5. The audio dataset is trained and recognized using a physical storage pool computing system: In the audio "0-9" recognition task, the initial audio signal undergoes pre-emphasis processing, windowing, framing, FFT transformation, MEL filtering, and MFCC feature extraction to obtain complete MFCC features (13 MFCC coefficients × 57 time frames). Then, the mean, standard deviation, maximum, and minimum values ​​are calculated, and quantized to logic 1 or 0 depending on whether they exceed the median. Finally, 13 sets of 4-bit binary codes are obtained, used to generate the final storage pool input features. The audio "0-9" dataset contains 500 segments from 5 speakers, with 400 segments in the training set (4 speakers × 100 segments) and 100 segments in the test set (1 speaker × 100 segments).

[0072] Figure 2 These are the current response diagrams of four representative photonic memristors induced by optical pulse programming according to the present invention. Figure 2 In diagram a, the current response of a photonic synapse induced by four representative optical pulse programming techniques ("0011", "1011", "1111", and "0010") is plotted, and in diagram b, the current characteristic sampling method of the device is shown. Figure 2As shown, "0" and "1" represent the on and off states of the ultraviolet light pulse, respectively. The device is programmed using a 4-bit ultraviolet light pulse sequence. The device current is extracted as feature outputs (SMP1 and SMP2) by adjusting the sampling position. SMP1, as the primary feature, takes into account the effects of sampling delay and sampling time, while the secondary feature SMP2 is sampled directly at the 13th second.

[0073] Figure 3 The diagram shows the output results of the main feature SMP1 and secondary feature SMP2 of the photonic memristor fabricated under 4-bit ultraviolet light pulse stimulation according to this invention. Figure 3 As shown, the characteristic outputs of the photonic synapse are displayed under all 16 optical pulse input sequences. Each sequence was tested three times, and the mean and standard deviation were calculated. The photonic synapse successfully mapped each 4-bit input sequence to different SMP1 and SMP2 values, confirming its rich reservoir state.

[0074] Figure 4 The physical storage pool computing system constructed for this invention trains and recognizes the letter dataset. Figure 4 In the diagram, 'a' represents the recognition image for the letter "j", and 'b' represents the training process diagram of the reading network. For example... Figure 4 As shown in Figure a, taking the letter "J" as an example, the letter recognition task mainly consists of three components: software processing (from right to left: original image, binarized image, max pooling, and pixel rearrangement), PRC mapping, and linear regression. Figure 4 As shown in Figure b, the training process of the readout network demonstrates how the letter recognition accuracy changes with the increase of training rounds under different feature strategies.

[0075] Figure 5 The physical storage pool computing system constructed for this invention trains and recognizes audio datasets. Figure 5 In diagram a, the audio processing flow is illustrated, and in diagram b, the training and recognition diagrams of the readout network (audio "0-9" recognition task) are shown. Figure 5 As shown in Figure a, feature extraction, physical reservoir mapping based on photonic synapses, and linear regression training are performed. Figure 5 As shown in Figure b, the training process of the readout network demonstrates how the audio recognition accuracy changes with the increase of training rounds under different feature strategies.

[0076] The Physical Reservoir Computation (PRC) system consists of three main parts: an input layer, a physical reservoir layer, and an output layer. The input is a 4-bit UV light pulse, while the reservoir mapping result is represented by the readout signals SMP1 and SMP2 from the photonic memristor device. The output layer is programmed in Python and trained under a supervised learning framework, using a Softmax output function and a single-layer linear classifier. In letter and audio recognition, the photonic memristor-based physical reservoir computation achieves accuracies of up to 91.7% and 95.0%, respectively. This method significantly improves the application of photonic biomimetic synaptic devices in audiovisual multimodal recognition, providing a reference device example for the implementation of physical reservoir computation.

[0077] It should be noted that when numerical ranges are involved in this invention, it should be understood that both endpoints of each numerical range and any value between the two endpoints can be selected. Since the steps and methods used are the same as in the embodiments, preferred embodiments are described here to avoid redundancy. Although preferred embodiments of the invention have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this invention.

[0078] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A photonic memristor for audio-visual dual modal recognition, characterized in that, The memristor comprises, from bottom to top along the thickness direction, a bottom electrode, a hole layer, a light-absorbing layer, a passivation layer and a top electrode, the hole layer is a ZnO thin film for capturing photo-generated carriers, the light-absorbing layer is a Cs2AgBiBr6 thin film for absorbing photons, the ZnO thin film and the Cs2AgBiBr6 thin film form a heterojunction, the passivation layer is a PMMA thin film for passivating perovskite defects, the bottom electrode is a conductive glass, and the top electrode is a conductive metal; the photomemristor adjusts the current state characteristics by adjusting the ultraviolet light pulse during programming, thereby adjusting the responsivity of the memristor.

2. The photonic memristor for audio-visual dual modal recognition according to claim 1, wherein, The thickness of the hole layer is 70-110 nm, the thickness of the light-absorbing layer is 120-200 nm, and the thickness of the passivation layer is 60-100 nm.

3. The photonic memristor for audio-visual dual modal recognition according to claim 1, wherein, The bottom electrode is an FTO electrode, and the top electrode is an Ag electrode.

4. A method for preparing the photonic memristor for audio-visual dual modal recognition according to any one of claims 1-3, characterized in that, The method comprises the following steps: A conductive glass is provided as a bottom electrode, a ZnO thin film is deposited on the bottom electrode under an oxygen-free atmosphere to form a hole layer; A Cs2AgBiBr6 thin film is prepared on the hole layer by two-step spin coating to obtain a light-absorbing layer; A PMMA thin film is prepared on the light-absorbing layer by two-step spin coating to obtain a passivation layer; A conductive metal is deposited on the passivation layer to obtain a top electrode; A photomemristor for audio-visual dual-modal recognition is obtained.

5. A reservoir mapping method of the photonic memristor for audio-visual dual modal recognition according to any one of claims 1 to 3, characterized in that, The method comprises the following steps: A voltage is applied between the top electrode and the bottom electrode of the photomemristor, and an ultraviolet light pulse sequence is applied to the photomemristor to output a current response curve of the photomemristor; The current response curve is sampled to obtain a main feature SMP1 and a secondary feature SMP2; the main feature SMP1 is the average value of all sampling point currents in a sampling parameter as the SMP1 value, the sampling parameter is a sampling delay of 0 seconds and a sampling time of 5 seconds, and the sampling parameter of the secondary feature SMP2 is the direct sampling of the current value at the 13th second; after testing the 4-bit ultraviolet light pulse sequence, the 4-bit SMP1 main feature value and the 4-bit SMP2 secondary feature value are obtained respectively; Based on the main feature SMP1 and the secondary feature SMP2, a physical reservoir computing system is constructed; the physical reservoir computing system comprises an input layer, a physical reservoir layer and an output layer, the input layer receives a 4-bit ultraviolet light pulse sequence, the physical reservoir layer maps the results represented by SMP1 and SMP2, and the output layer is trained under a supervised learning framework using a linear classifier and a Softmax output function; The physical reservoir computing system is applied to train and recognize letter data sets and audio data sets; the signal features of the letter data sets and the audio data sets are encoded into multiple groups of 4-bit binary codes, and each group corresponds to a string of 4-bit ultraviolet light pulse sequences.

6. The physical reservoir computing system of claim 5, wherein, The recognition of the letter data set comprises: after the original image is binarized to obtain a binary matrix, a 2*2 kernel is used for maximum pooling to reduce the dimension, and the output after the pooling is flattened into a one-dimensional vector, from which multiple groups of 4-bit binary codes are extracted as the input of the input layer.

7. The physical reservoir computing system of claim 5, wherein, The identification of the audio data set comprises pre-emphasis processing, windowing, framing, FFT transformation, MEL filtering and MFCC feature extraction on the initial audio signal, after obtaining the MFCC features, calculating the mean, standard deviation, maximum value and minimum value, and quantifying them as logical 1 or 0 according to whether they exceed the median, and finally generating multiple groups of 4-bit binary codes as the input of the input layer.

8. The physical reservoir computing system of claim 5, wherein, The voltage is 0.05 V, and is used for reading the current response of the photonic memristor.

9. The physical reservoir computing system of claim 5, wherein, The UV light pulse sequence is a 4-bit pulse sequence, and the light intensity is 12.3 mW / cm -2 .

10. Application of the photonic memristor for audio-visual bimodal recognition according to any one of claims 1-3 in visual and auditory information processing.

Images