Multimodal vision sensing system and method of use, method of making a vision sensor

Abstract

The application relates to the technical field of visual sensing, and provides a multi-modal visual sensing system and a use method, and a preparation method of a visual sensor. The multi-modal visual sensing system comprises an array formed by a plurality of visual sensors, a power module, a VI conversion transimpedance amplification module, a single-chip microcomputer system and a result output module. A synapse transistor is used as the visual sensor. The structure of a photosensitive floating gate layer is a two-dimensional perovskite layer and a three-dimensional perovskite layer. The single-chip microcomputer system is used for receiving a voltage signal, converting the voltage signal into a digital signal, and processing the digital signal to obtain multi-modal visual information. The output module is used for outputting the multi-modal visual information processed by the single-chip microcomputer system. The information output by the output module comprises wavelength information, a motion state of an object, a spatial position, time and polarization state information of light. By using the system, the problem that the visual information capable of being detected and output by an existing visual sensing system is limited and cannot meet the demand for diversified information can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual sensing, in particular to a multi-modal visual sensing system and use method, and a preparation method of a visual sensor. BACKGROUND

[0002] The visual sensor uses a perovskite synapse transistor structure to exhibit great potential in improving the ability of computers to process visual information. The synapse transistor uses a non-volatile synapse oxide medium, which almost does not consume energy for each operation, thereby significantly reducing power consumption. Compared with traditional flash memory devices, the synapse transistor is more energy-saving and can significantly reduce the overall power consumption of the device. The synapse transistor has a simple and stable structure and is not easy to be damaged, and can withstand a high amount of electron injection, so it has higher reliability. At the same time, the synapse oxide transistor is not prone to failure phenomena such as "leakage", and has a longer service life. The synapse transistor is manufactured based on the CMOS process, so its manufacturing cost is relatively low, and the size of the components is relatively small. This makes the synapse transistor meet the requirements of various miniaturized devices, further expanding its application field.

[0003] Although the perovskite synapse transistor can simulate the synapses in the brain that are used to store memories, there are still challenges in realizing more complex neuromorphic computing. The high interconnectivity and complexity of the brain neural network make it extremely difficult to simulate its function, and current technology cannot achieve complete brain simulation. Therefore, when using the perovskite synapse transistor for visual detection, it may not be able to fully utilize its ability to improve the diversity of detection results. SUMMARY

[0004] The present application aims to solve the problem that the existing visual sensing system can only detect and output limited visual information, which cannot meet the demand for diverse information, and provides a multi-modal visual sensing system and use method, and a preparation method of a visual sensor. The synapse transistor has high light response sensitivity, and the separate gate can be independently regulated.

[0005] To achieve the above-mentioned purpose, the technical scheme adopted by the present application is:

[0006] A multi-modal visual sensing system comprises an array of several visual sensors, a power module, a VI conversion trans-impedance amplification module, a single-chip microcomputer system and a result output module, uses a synaptic transistor as a visual sensor, the structure of the visual sensor is sequentially connected substrate, bottom gate layer, insulating layer, photosensitive floating gate layer, tunneling layer, channel layer and metal electrode layer, the substrate adopts flexible material, the bottom gate layer and the metal electrode layer are used for connecting the power module, the metal electrode serves as a source electrode and a drain electrode, the photosensitive floating gate layer is a double-layer perovskite film, the structure of the photosensitive floating gate layer from top to bottom is a two-dimensional perovskite layer and a three-dimensional perovskite layer, the two-dimensional perovskite layer is used for responding to the polarization state of light, the two-dimensional perovskite layer and the three-dimensional perovskite layer are used for generating photo-generated carriers, and the channel layer is used for transmitting the carriers, and the tunneling layer is used for capturing the carriers and isolating the photo-generated carriers;

[0007] The power module is connected to the bottom gate layer, the metal electrode layer, the VI conversion trans-impedance amplification module, the single-chip microcomputer system and the result output module, the power module is used for applying a bias voltage to the visual sensor array and supplying power to the VI conversion trans-impedance amplification module, the single-chip microcomputer system and the result output module, the VI conversion trans-impedance amplification module is connected to the metal electrode layer, the VI conversion trans-impedance amplification module is used for amplifying the photocurrent signal and converting it into a voltage signal, the single-chip microcomputer system is connected to the VI conversion trans-impedance amplification module, the single-chip microcomputer system is used for receiving the voltage signal, converting it into a digital signal and processing it to obtain multi-modal visual information, and the output module is connected to the single-chip microcomputer system, and the output module is used for outputting the multi-modal visual information obtained by the single-chip microcomputer system.

[0008] The information output by the output module comprises color information, motion state information of an object, spatial position information of the object, time information and polarization state information of light.

[0009] The two-dimensional perovskite layer is composed of R configuration phenethylamine lead iodine, and the three-dimensional perovskite layer is composed of FAPbBr3.

[0010] The multi-modal visual sensing system provided by the application can detect photocurrent information more quickly and accurately by using an array of several visual sensors. When the photosensitive floating gate layer receives a light pulse, photo-generated excitons are generated, which are quickly separated into photo-generated holes and electrons under light. The photo-generated holes enter the channel layer through the tunneling layer, while the electrons are captured at the interface defects between the photosensitive floating gate layer and the tunneling layer in the initial state. In this process, a current is generated. Therefore, after the light pulse disappears, the current does not decrease to the dark state level at the first time, and the photocurrent signal can be used for detecting the action of generating current change.

[0011] Unlike traditional photodetectors with constant saturated photocurrent, the visual sensor realizes the relative position displacement between the interface state and quasi-Fermi level (E F ) by the gate voltage regulation under various light conditions, and when affected by positive (negative) bias V G , E F can be raised (lowered), which plays a leading role in releasing trapped electrons into the channel layer to promote (weaken) the recombination with holes, thereby completing the writing and erasing of optical signals and the sensing and processing of multi-modal information. The visual sensor can improve the light absorption and response using the two-dimensional perovskite layer and the three-dimensional perovskite layer as the light-sensitive floating gate layer, and the two-dimensional perovskite layer can have a good response to the polarization state of light, which can detect the polarization state digital information of light in cooperation with the single-chip microcomputer system.

[0012] The power module applies different bias voltages to the visual sensor to control the working state of the visual sensor, thereby outputting different photocurrent signals, so that the visual sensor can detect the photocurrent array in weak light, the photocurrent array detected by the visual sensor array is converted into a voltage signal by the VI conversion transimpedance amplification module and is amplified, the single-chip microcomputer system receives the voltage signal and converts it into a digital signal and processes it, and the output module can output color information, object motion direction information, object spatial position information, time information and light polarization state information according to the processing result.

[0013] The single-chip microcomputer system serves as the core control unit of the multi-modal visual sensing system, can coordinate the work of each part, and ensures the efficient operation of the entire multi-modal visual sensing system. At the same time, the powerful computing capability of the single-chip microcomputer system further improves the information processing capability of the multi-modal visual sensing system. The combination of the visual sensor and the single-chip microcomputer system enables the entire system to realize high integration in a small space. Such integration not only reduces the volume and weight of the system, but also improves the stability and reliability of the multi-modal visual sensing system.

[0014] The multi-modal visual sensing system adopted by the application has the advantages of efficient information processing, fast response speed, high integration, good compatibility and reliability, strong learning ability, and can output various information. These advantages make the system have wide application prospects in the fields of automatic driving, robot vision, security monitoring, etc.

[0015] As a preferred scheme of the application, the material of the tunneling layer comprises polymethyl methacrylate (PMMA).

[0016] The application relates to a method for using a multi-modal visual sensing system, which comprises the following steps: first, starting a power module, and then detecting and outputting a photoelectric current signal array by a visual sensor array; a VI conversion trans-impedance amplification module receives the photoelectric current signal of each visual sensor, converts the photoelectric current signal into a voltage signal, amplifies the voltage signal, and then transmits the voltage signal to a single-chip microcomputer system.

[0017] The wavelength information detection step is as follows:

[0018] A1. The single-chip microcomputer system converts the voltage signal into a digital signal, compares the digital signal with wavelength digital information of different colors of light recorded in a first database in sequence, and outputs a first digital signal.

[0019] A2. An output module receives the first digital signal and outputs color information corresponding to each visual sensor.

[0020] The polarization state information detection step is as follows:

[0021] B1. The single-chip microcomputer system converts the voltage signal into a digital signal and compares the digital signal with a plurality of polarization state digital information of light recorded in a second database in sequence, identifies the polarization information of the photoelectric current signal array, and outputs a second digital signal.

[0022] B2. An output module receives the second digital signal and outputs corresponding polarization state information of light.

[0023] The spatial position information and time information detection step is as follows:

[0024] C1. The single-chip microcomputer system converts the voltage signal into a digital signal and calculates the coordinates of each visual sensor in the visual sensor array as a mapping digital signal of a light source, compares the mapping digital signal with mapping digital information of the light source at different positions recorded in a third database in sequence, obtains spatial information of an object, and outputs a third digital signal; the single-chip microcomputer system continuously calculates the change trend of the photoelectric current signal array and compares the change trend with time digital information recorded in a fourth database, and outputs a fourth digital signal.

[0025] C2. An output module receives the third digital signal and outputs spatial position information of the object; the output module receives the fourth digital signal and outputs time information.

[0026] The motion state information detection step is as follows:

[0027] D1, the single-chip microcomputer system converts the voltage signal into a digital signal, uses the single-chip microcomputer system to scan the digital signal of the visual sensor array at a certain moment when the object is in motion, and compares with the pre-stored object motion direction information in the fifth database to obtain the motion information of the object at a certain moment and output a fifth digital signal;

[0028] D2, the output module receives the fifth digital signal and outputs the motion state information of the object at a certain moment.

[0029] The application provides a use method of a multi-modal visual sensing system, which can detect multiple information by using the visual sensor array, improve the detection efficiency in actual application, and reduce the detection cost.

[0030] When detecting the motion of an object, unlike the motion detection of a traditional visual sensing system which needs a high-speed camera to take multiple frames of the moving object and process the redundant information, the multi-modal visual sensing system only needs to scan the digital signal of each visual sensor in the visual sensor array at a certain moment when the object is in motion to obtain the motion direction of the object at that moment. When the object moves in space, it blocks the light irradiated onto the visual sensor array, so that the light signal received by the visual sensor array becomes weak, and the photoelectric current signal of the visual sensor array also becomes weak. Because the structure used by the visual sensor has a memory characteristic, when moving horizontally, the positions passed by the object first and last are mapped in the array to show a state of large to small device current; and when approaching and moving away, with the change of the relative position of the object, the light source and the visual sensor array, the area of the photoelectric current signal received by the visual sensor array also changes. Therefore, only the scanning result at a certain moment of the motion of the object can determine the motion direction of the object.

[0031] The use of a synaptic transistor as the visual sensor has better response to light, is sensitive to light changes, has fast detection time, color, spatial position of the object and polarization state of light, high accuracy, and improves the detection efficiency.

[0032] As a preferred scheme of the application, the wavelength digital information of light of different colors, the digital information of different polarization states of light, the mapping digital signal corresponding to the light source at different positions, the digital information at different times, and the digital information of the motion of the object in different directions are input into the single-chip microcomputer system as learning samples and fitted into a Bayesian regularization mathematical model, and then the corresponding first database, second database, third database, fourth database and fifth database are obtained through machine learning and neural network training.

[0033] The single-chip microcomputer system repeatedly performs optical programming and electrical erasing operations on a single visual sensor in the visual sensor array with light pulses and electrical pulses of different wavelengths, and records the measured photocurrent signals, taking the photocurrent signals of the visual sensor array as the mapping results of light pulses of different wavelengths, as mathematical model input data and fitting a Bayesian regularized mathematical model, through machine learning and neural network training, the visual sensor array has good recognition function for light of different wavelengths, and can be applied to color detection and color imaging field.

[0034] When the light source is located in different positions in space, the photocurrent signal distribution received by the visual sensor in the visual sensor array is also different, taking the coordinates of each visual sensor in the visual sensor array as the mapping results of the position of the light source, as mathematical model input data and fitting a Bayesian regularized mathematical model, through machine learning and neural network training, the visual sensor array can complete the detection of the spatial position of the object.

[0035] Due to the memory characteristics of the visual sensor, when the light pulse disappears, the photocurrent signal of the visual sensor array will not instantaneously decrease to the low level of the off state. Therefore, when the object is in the detection range of the visual sensor array, the photocurrent signal generated by the object will gradually decrease, so that the state of the object when moving can be detected in cooperation with the single-chip microcomputer system.

[0036] Taking the corresponding photocurrent signals at different times and the corresponding photocurrent signals of different light polarizations as mathematical model input data and fitting a Bayesian regularized mathematical model, through machine learning and neural network training, the visual sensor array can complete the detection of the position of the light source.

[0037] A preparation method of a visual sensor, comprising preparing a visual sensor in a multi-modal visual sensing system, the steps being as follows:

[0038] S1, cleaning, drying and ozone treatment of the substrate;

[0039] S2, making a bottom gate layer on the top surface of the substrate;

[0040] S3, preparing an insulating layer on the top surface of the bottom gate layer by atomic deposition method;

[0041] S4, a perovskite precursor solution is configured, the perovskite precursor solution is coated on the top surface of the insulating layer, and after annealing treatment, a photosensitive floating gate layer is obtained; FABr and PbBr2 are dissolved in a mixed solution of anhydrous dimethylformamide (DMF) and dimethyl sulfoxide (DMSO) in a volume ratio of 1:4 to obtain a first perovskite precursor solution; hydriodic acid aqueous solution and methylamine (MA) and an aqueous solution of R-configuration phenylethylamine lead iodine are added to a mixed solution of deionized water and PbI2 to obtain a second perovskite precursor solution, and the pH value of the second perovskite precursor solution is less than 4; the first perovskite precursor solution is blade-coated on the top surface of the bottom gate layer and annealed to obtain a three-dimensional perovskite layer, and then the second perovskite precursor solution is blade-coated on the top surface of the three-dimensional perovskite layer and annealed and separated to obtain a two-dimensional perovskite layer, and the preparation of the photosensitive floating gate layer is completed;

[0042] S5, a tunneling layer is prepared on the top surface of the photosensitive floating gate layer by an atomic deposition method;

[0043] S6, a channel layer is prepared on the top surface of the tunneling layer by a thermal evaporation plating method;

[0044] S7, a metal electrode layer is prepared on the top surface of the channel layer.

[0045] As a preferred scheme of the present application, in step S1, the substrate is first subjected to ultrasonic cleaning, then nitrogen is used to remove liquid residues on the surface of the substrate, and the substrate is vacuum dried and then subjected to ozone treatment.

[0046] As a preferred scheme of the present application, in step S5, the thickness of the tunneling layer is 8-15 nm.

[0047] As a preferred scheme of the present application, in step S6, the material of the channel layer is pentacene, and the thickness of the channel layer is 30-90 nm.

[0048] As a preferred scheme of the present application, in step S7, the metal electrode layer contains copper, silver or gold, the metal electrode layer is uniformly prepared on the top surface of the channel layer by a thermal evaporation plating method, and the thickness of the metal electrode layer is 80-120 nm.

[0049] In summary, due to the adoption of the above technical scheme, the present application has the following beneficial effects:

[0050] 1. A multi-modal visual sensing system uses a synaptic transistor as a visual sensor, has a good response to light, is sensitive to light changes, and has a fast detection time, wavelength, spatial position of an object, motion state of an object and polarization state of light, high accuracy, and improved detection efficiency.

[0051] 2. A method for using a multimodal visual sensing system, wherein the single-chip computer system compares the received voltage signal with the first database, the second database, the third database, the fourth database, and the fifth database, respectively, and can quickly determine the wavelength information array, the object's movement direction information, the object's spatial position information, time information, and the polarization state information of light.

[0052] 3. A method for preparing a visual sensor. The preparation method is simple and can be prepared using the preparation method of perovskite photodetectors, which reduces the demand for instruments and thus reduces the cost of introducing new equipment. The prepared visual sensor is sensitive and has high detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a connection diagram of a multimodal visual sensing system. DETAILED DESCRIPTION

[0054] The present invention will be described in detail below with reference to the accompanying drawings.

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] Example 1

[0057] like Figure 1 As shown, a perovskite synaptic transistor used in the present invention includes an array formed by several visual sensors, a power module, a VI conversion transimpedance amplifier module, a single-chip computer system and a result output module. The synaptic transistor is used as a visual sensor. The structure of the visual sensor is a substrate, a bottom gate layer, an insulating layer, a photosensitive floating gate layer, a tunneling layer, a channel layer and a metal electrode layer connected in sequence. The substrate is made of flexible material. The bottom gate layer and the metal electrode layer are used to connect the power module. The metal electrode serves as the source and drain. The photosensitive floating gate layer is a double-layer perovskite film. The structure of the photosensitive floating gate layer from top to bottom is a two-dimensional perovskite layer and a three-dimensional perovskite layer. The two-dimensional perovskite layer is used to respond to the polarization state of light. The two-dimensional perovskite layer and the three-dimensional perovskite layer are used to generate photogenerated carriers. The channel layer is used to transmit carriers. The tunneling layer is used to capture the carriers and isolate the photogenerated carriers.

[0058] The power module is connected to the bottom gate layer, the metal electrode layer, the VI conversion trans-impedance amplification module, the single-chip microcomputer system and the result output module, and is used to apply a bias voltage to the visual sensor array and to supply power to the VI conversion trans-impedance amplification module, the single-chip microcomputer system and the result output module.

[0059] The information output by the output module includes color information, motion state information of an object, spatial position information of an object, time information and polarization state information of light.

[0060] The two-dimensional perovskite layer is composed of R-configuration phenethylamine lead iodine, and the three-dimensional perovskite layer is composed of FAPbBr3.

[0061] Further, the material of the tunneling layer includes polymethyl methacrylate (PMMA).

[0062] The array formed by a plurality of the visual sensors can more quickly and accurately detect photocurrent information. The light-sensitive floating gate layer receives a light pulse to generate photo-generated excitons, which are quickly separated into photo-generated holes and electrons under light. The photo-generated holes enter the channel layer through the tunneling layer, while the electrons are initially captured at the interface defect between the light-sensitive floating gate layer and the tunneling layer. Therefore, after the light pulse disappears, the current does not immediately decrease to the dark state level, and the photoelectric signal can be used to detect the action of generating current change. The power module applies different bias voltages to the visual sensor to control the working state of the visual sensor, so as to output different photocurrent signals. After amplification processing by the VI conversion trans-impedance amplification module and processing by the single-chip microcomputer system, the output module can output time information, position information, motion trajectory information, spectral information or polarization state information of light according to the processing result.

[0063] Embodiment 2

[0064] The method for using the multi-modal visual sensing system comprises the following steps: first, starting the power module, the visual sensor array detects and outputs a photocurrent signal array; the VI conversion trans-impedance amplification module receives the photocurrent signal of each visual sensor, converts it into a voltage signal, and transmits it to the single-chip microcomputer system after amplification processing;

[0065] The step of detecting the wavelength information is as follows:

[0066] A1, the single-chip microcomputer system converts the voltage signal into a digital signal, compares it with the wavelength digital information of different colors of light recorded in the first database in sequence, and outputs a first digital signal;

[0067] A2, the output module receives the first digital signal and outputs the color information corresponding to each visual sensor;

[0068] The step of detecting the polarization state information of light is as follows:

[0069] B1, the single-chip microcomputer system converts the voltage signal into a digital signal and compares it with the digital information of several polarization states of light recorded in the second database in sequence, identifies the polarization information of the photocurrent signal array, and outputs a second digital signal;

[0070] B2, the output module receives the second digital signal and outputs the corresponding polarization state information of light;

[0071] The step of detecting the spatial position information and time information is as follows:

[0072] C1, the single-chip microcomputer system converts the voltage signal into a digital signal and calculates the coordinates of each visual sensor in the visual sensor array as a mapping digital signal of the light source, and the single-chip microcomputer system compares the mapping digital signal with the mapping digital information of the light source at different positions in the third database in sequence to obtain the spatial information of the object and output a third digital signal; the single-chip microcomputer system continuously calculates the change trend of the photocurrent signal array and compares it with the time digital information in the fourth database, and outputs a fourth digital signal;

[0073] C2, the output module receives the third digital signal and outputs the spatial position information of the object; the output module receives the fourth digital signal and outputs the time information;

[0074] The step of detecting the motion state information of the object is as follows:

[0075] D1, the single-chip microcomputer system converts the voltage signal into a digital signal, uses the single-chip microcomputer system to scan the digital signal of the visual sensor array at a certain moment when the object is in motion, and compares with the pre-stored object motion direction information in the fifth database to obtain the motion information of the object at a certain moment and output a fifth digital signal;

[0076] D2, the output module receives the fifth digital signal and outputs the motion state information of the object at a certain moment.

[0077] Further, the wavelength digital information of different color light, the digital information of different polarization states of light, the mapping digital signal corresponding to the light source at different positions, the digital information of different times and the digital information of the object moving in different directions are input into the single-chip microcomputer system as learning samples and a Bayesian regularization mathematical model is fitted, and the corresponding first database, second database, third database, fourth database and fifth database are obtained through machine learning and neural network training.

[0078] In the embodiment, the synapse transistor is used as the visual sensor, which has good response to light and is sensitive to light changes, and has fast detection time, color, spatial position of the object and polarization state of light, high accuracy and improved detection efficiency. Because the structure used by the visual sensor has memory characteristics, when moving horizontally, the positions of the object moving first and later are mapped in the array to show the state of the device current from large to small; and when approaching and moving away, as the relative position of the object and the light source and the visual sensor array changes, the area of the photoelectric current signal received by the visual sensor array also changes. Therefore, only the scanning result at a certain moment of the object motion can determine the motion direction of the object. The single-chip microcomputer system compares the received voltage signal with the first database, second database, third database, fourth database and fifth database, so as to determine the wavelength information array, the motion direction information of the object, the spatial position information of the object, the time information and the polarization state information of the light.

[0079] Embodiment 3

[0080] The preparation method of the visual sensor adopted in the application comprises the following steps of preparing the visual sensor in the multi-modal visual sensing system as described in embodiment 1:

[0081] S1, cleaning, drying and ozone treatment are performed on the substrate;

[0082] S2, a bottom gate layer is prepared on the top surface of the substrate;

[0083] S3, an insulating layer is prepared on the top surface of the bottom gate layer by using atomic deposition method;

[0084] S4, a perovskite precursor solution is configured, the perovskite precursor solution is coated on the top surface of the insulating layer, and after annealing treatment, a photosensitive floating gate layer is obtained; FABr and PbBr2 are dissolved in a mixed solution of anhydrous dimethylformamide (DMF) and dimethyl sulfoxide (DMSO) in a volume ratio of 1:4 to obtain a first perovskite precursor solution; hydriodic acid aqueous solution and methylamine (MA) and R-configuration phenethylamine lead iodine aqueous solution are added to a mixed solution of deionized water and PbI2 to obtain a second perovskite precursor solution, and the pH value of the second perovskite precursor solution is less than 4; the first perovskite precursor solution is blade-coated on the top surface of the bottom gate layer and annealed to obtain a three-dimensional perovskite layer, and then the second perovskite precursor solution is blade-coated on the top surface of the three-dimensional perovskite layer and annealed and separated to obtain a two-dimensional perovskite layer, and the preparation of the photosensitive floating gate layer is completed;

[0085] S5, a tunneling layer is prepared on the top surface of the photosensitive floating gate layer by an atomic deposition method;

[0086] S6, a channel layer is prepared on the top surface of the tunneling layer by a thermal evaporation plating method;

[0087] S7, a metal electrode layer is prepared on the top surface of the channel layer.

[0088] Further, in the step S2, the bottom gate layer is a silicon wafer with grown silicon dioxide, the bottom gate layer is first subjected to ultrasonic cleaning, then nitrogen is used to remove liquid residues on the surface of the bottom gate layer, the bottom gate layer is vacuum dried, and then subjected to ozone treatment, and the bottom gate layer is prepared on the top surface of the substrate by evaporation.

[0089] Further, in the step S5, the thickness of the tunneling layer is 8-15 nm.

[0090] Further, in the step S6, the material of the channel layer is pentacene, and the thickness of the channel layer is 30-90 nm.

[0091] Further, in the step S7, the metal electrode layer contains copper, silver or gold, the metal electrode layer is prepared on the top surface of the channel layer by a thermal evaporation plating method, and the thickness of the metal electrode layer is 80-120 nm.

[0092] The substrate is prepared: the substrate is taken out and sequentially subjected to ultrasonic cleaning with acetone, ethanol and deionized water for 15 min; the cleaned substrate is taken out and the residual deionized water on the surface is blown dry with N2; the dried substrate is placed in a vacuum drying box with a set temperature of 80℃ for 30 min until dry; the dried substrate is taken out of the vacuum drying box and placed in an ozone environment for 15 min, and then taken out for standby.

[0093] Preparation of the bottom gate layer: the bottom gate layer is prepared on the top surface of the substrate by evaporating copper to a thickness of 50 nm.

[0094] Preparation of the insulating layer: the insulating layer is prepared on the top surface of the bottom gate layer by ALD growth, and the material of the insulating layer is aluminum oxide. In this embodiment, the thickness of the insulating layer is 100 nm.

[0095] Preparation of the photosensitive floating gate layer: 100 mg of FABr and 293.6 mg of PbBr2 are dissolved in 4 ml of a mixed solution of anhydrous dimethylformamide (DMF) and dimethyl sulfoxide (DMSO) to obtain a perovskite precursor solution. The above-mentioned precursor solution is coated on the insulating layer by using a perovskite thin film coating device. After coating, annealing is performed at a temperature of 85°C for 10 min to obtain the photosensitive floating gate layer.

[0096] Preparation of the tunneling layer: the tunneling layer is prepared on the top surface of the photosensitive floating gate layer by ALD growth, and the material of the tunneling layer is PMMA. In this embodiment, the thickness of the tunneling layer is 10 nm.

[0097] Preparation of the channel layer: the channel layer is prepared on the top surface of the tunneling layer by thermal evaporation plating, and the material of the channel layer is pentacene. In this embodiment, the thickness of the channel layer is 50 nm.

[0098] Preparation of the metal electrode layer: in this embodiment, the material of the metal electrode layer is copper, and the metal electrode layer is prepared on the top surface of the channel layer by thermal evaporation plating. The thickness of the metal electrode layer is 100 nm.

[0099] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement and improvement within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multimodal visual sensing system, comprising an array of several visual sensors, a power supply module, a VI conversion transimpedance amplifier module, a single-chip microcomputer system, and a result output module, characterized in that: A synaptic transistor is used as a visual sensor. The structure of the visual sensor comprises a substrate, a bottom gate layer, an insulating layer, a photosensitive floating gate layer, a tunneling layer, a channel layer, and a metal electrode layer connected in sequence. The substrate is made of a flexible material. The bottom gate layer and the metal electrode layer are used to connect to the power module. The metal electrodes serve as the source and drain. The photosensitive floating gate layer is a double-layer perovskite film. The structure of the photosensitive floating gate layer from top to bottom comprises a two-dimensional perovskite layer and a three-dimensional perovskite layer. The two-dimensional perovskite layer is used to respond to the polarization state of light. The two-dimensional perovskite layer and the three-dimensional perovskite layer are used to generate photogenerated carriers. The channel layer is used to transmit carriers. The tunneling layer is used to capture the carriers and isolate the photogenerated carriers. The power supply module is simultaneously connected to the bottom gate layer, the metal electrode layer, the VI conversion transimpedance amplifier module, the single-chip computer system and the result output module. The power supply module is used to apply a bias voltage to the visual sensor array and power the VI conversion transimpedance amplifier module, the single-chip computer system and the result output module. The VI conversion transimpedance amplifier module is connected to the metal electrode layer. The VI conversion transimpedance amplifier module is used to amplify the photocurrent signal and convert it into a voltage signal. The single-chip computer system is connected to the VI conversion transimpedance amplifier module. The single-chip computer system is used to receive the voltage signal and convert it into a digital signal and process it to obtain multimodal visual information. The output module is connected to the single-chip computer system. The output module is used to output the multimodal visual information processed by the single-chip computer system. The information output by the output module includes color information, object motion state information, object spatial position information, time information and light polarization state information; The component of the two-dimensional perovskite layer is R-configuration phenylethylamine lead iodine, and the component of the three-dimensional perovskite layer is FAPbBr3.

2. A multimodal visual sensing system according to claim 1, characterized in that: The tunneling layer is made of polymethyl methacrylate (PMMA).

3. A method for using a multimodal visual sensing system, characterized in that: The method comprises using a multimodal visual sensing system according to any one of claims 1 to 2 for detection, firstly starting a power module, and the visual sensor array performs detection and outputs a photocurrent signal array; the VI conversion transimpedance amplifier module receives the photocurrent signal of each visual sensor, converts it into a voltage signal, amplifies it, and transmits it to the single-chip computer system; The steps for detecting wavelength information are as follows: A1. The single-chip microcomputer system converts the voltage signal into a digital signal, compares it with the digital information of wavelengths of different colors of light recorded in the first database, and outputs a first digital signal; A2, an output module receives the first digital signal and outputs color information corresponding to each of the visual sensors; The steps to detect the polarization state information of light are as follows: B1. The single-chip computer system converts the voltage signal into a digital signal and compares it with the digital information of the polarization states of light recorded in the second database in sequence, identifies the polarization information of the photocurrent signal array and outputs it as a second digital signal; B2. The output module receives the second digital signal and outputs corresponding light polarization state information; The steps for detecting spatial location information and time information are as follows: C1. The single-chip microcomputer system converts the voltage signal into a digital signal and calculates the coordinates of each of the visual sensors in the visual sensor array as a mapping digital signal of the light source. The single-chip microcomputer system sequentially compares the mapping digital signal with the mapping digital signals corresponding to the light sources at different positions in the third database to obtain spatial information of the object and outputs a third digital signal. The single-chip microcomputer system continuously calculates the change trend of the photocurrent signal array and compares it with the time digital information in the fourth database, and outputs a fourth digital signal. C2, the output module receives the third digital signal and outputs the spatial position information of the object; the output module receives the fourth digital signal and outputs time information; The steps to detect the motion state information of an object are as follows: D1. The single-chip microcomputer system converts the voltage signal into a digital signal, uses the single-chip microcomputer system to scan the digital signal of the visual sensor array at a certain moment when the object is moving, and compares it with the object movement direction information pre-stored in a fifth database to obtain the movement information of the object at a certain moment and output a fifth digital signal; D2. The output module receives the fifth digital signal and outputs the motion state information of the object at a certain moment.

4. The method for using a multimodal visual sensing system according to claim 3, wherein: The digital information of the wavelengths of light of different colors, the digital information of the different polarization states of light, the mapped digital signals corresponding to the light sources at different positions, the digital information at different times, and the digital information of the objects moving in different directions are respectively used as learning samples to fit the Bayesian regularized mathematical model, and then the corresponding first database, second database, third database, fourth database, and fifth database are obtained through machine learning and neural network training.

5. A method for preparing a visual sensor, characterized in that: The method comprises preparing a visual sensor in a multimodal visual sensing system according to any one of claims 1 to 2, comprising the following steps: S1. Cleaning, drying and ozone treatment of the substrate; S2, forming a bottom gate layer on the top surface of the substrate; S3, preparing an insulating layer on the top surface of the bottom gate layer by an atomic deposition method; S4. Preparing a perovskite precursor solution, coating the perovskite precursor solution on the top surface of the insulating layer and annealing the solution to obtain a photosensitive floating gate layer; dissolving FABr and PbBr2 in a mixed solution of anhydrous dimethylformamide (DMF) and dimethyl sulfoxide (DMSO) in a volume ratio of 1:4 to obtain a first perovskite precursor solution; adding an aqueous solution of hydroiodic acid and an aqueous solution of methylamine (MA) and R-configuration phenylethylamine lead iodine to a mixed solution of deionized water and PbI2 to obtain a second perovskite precursor solution, wherein the pH value of the second perovskite precursor solution is less than 4; Applying the first perovskite precursor solution to the top surface of the bottom gate layer by scraping and annealing to obtain a three-dimensional perovskite layer, and then applying the second perovskite precursor solution to the top surface of the three-dimensional perovskite layer by scraping and annealing and separating to obtain a two-dimensional perovskite layer, thereby completing the preparation of the photosensitive floating gate layer; S5, preparing a tunneling layer on the top surface of the photosensitive floating gate layer by atomic deposition; S6, preparing a channel layer on the top surface of the tunnel layer by a thermal evaporation coating method; S7. Prepare a metal electrode layer on the top surface of the channel layer.

6. The method for preparing a visual sensor according to claim 5, characterized in that: In step S5, the thickness of the tunneling layer is 8-15 nm.

7. The method for preparing a visual sensor according to claim 5, characterized in that: In step S6 , the material of the channel layer is pentacene, and the thickness of the channel layer is 30-90 nm.

8. The method for preparing a visual sensor according to claim 5, wherein: In step S7, the metal electrode layer comprises copper, silver or gold, and is formed on the top surface of the channel layer by thermal evaporation coating method. The thickness of the metal electrode layer is 80-120 nm.

Images