A synapse-like dual-mode perception device

Abstract

The application provides a synapse-like bimodal sensing device, which integrates a bimodal sensing device in one body, significantly simplifies the structure of the bimodal sensing device, and enables the bimodal sensing device to simulate the timely response and processing of nerve synapses to tactile signals and visual signals, thereby exhibiting the performance characteristics of a synapse-like bimodal sensing device. The synapse-like bimodal sensing device comprises a transparent substrate, a stress layer, a photosensitive layer, and an electrode connected to the photosensitive layer; the stress layer and the photosensitive layer are distributed on two side surfaces of the transparent substrate, wherein the projected area of the stress layer on the transparent substrate is not greater than the projected area of the photosensitive layer on the transparent substrate, and the photosensitive layer comprises a metal oxide layer.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to a synaptic-like dual-mode sensing device. Background Technology

[0002] With the development of artificial intelligence, more and more attention is being focused on simulating neuromorphic computing, using synapses as an analogy to build more sensitive machine perception systems. Examples include humanoid robots and barrier-free human-machine interfaces.

[0003] In the context of human-computer interaction, a modality is a classification of a single, independent sensory input / output between a device (such as a computer) and a human. In other words, a modality can simulate the signal processing of various human modalities. If a device implements only one modality, it is designated as a single-modality device. For example, if a device only implements a response to light signals, then that device is single-modal.

[0004] Current machine perception systems still primarily rely on single-sensory (modal) synapses. This is because multimodal systems can only be constructed through circuit connections to couple multiple single-modal synapses. Especially for bimodal perception systems combining vision and touch, a common approach is to first convert mechanical force into electrical signals using mechanoreceptors for tactile perception; then, use photodetectors to convert light signals into electrical signals for visual perception; and finally, use synaptic devices to process and memorize both tactile and visual perceptions. Therefore, existing bimodal perception devices suffer from structural complexity and inconvenience due to the physical separation of the detector and synaptic device. Summary of the Invention

[0005] This application provides a synaptic dual-mode sensing device that integrates dual-mode sensing devices into one unit. This significantly simplifies the structure of the dual-mode sensing device while enabling it to simulate the timely response and processing of tactile and visual signals by neural synapses, thereby exhibiting synaptic-like performance characteristics.

[0006] In a first aspect, embodiments of this application provide a synapse-like dual-mode sensing device, comprising:

[0007] A transparent substrate, a force-bearing layer, a photosensitive layer, and an electrode in contact with the photosensitive layer; the force-bearing layer and the photosensitive layer are distributed on both sides of the transparent substrate, wherein,

[0008] The projected area of ​​the stress-bearing layer on the transparent substrate is not greater than the projected area of ​​the photosensitive layer on the transparent substrate. The stress-bearing layer includes a mechanoluminescent material layer, and the photosensitive layer includes a metal oxide layer.

[0009] This application embodiment provides a force-receiving layer and a photosensitive layer on both sides of a transparent substrate, enabling the dual-mode sensing array of this type of synapse to process the received light and force signals in a coordinated manner and respond promptly. This allows the dual-mode sensing device of this type of synapse to have a total thickness of no more than 2 mm, so that the use of the dual-mode sensing device is not limited by size. Furthermore, because the mechanoluminescent material in the force-receiving layer has self-driving and self-recovering characteristics, the energy consumption of this type of synapse dual-mode sensing device (as low as 20 pJ) is significantly reduced.

[0010] In one possible implementation, the photosensitive layer further includes a perovskite layer, wherein the perovskite layer overlaps with the metal oxide layer.

[0011] In one possible implementation, the thickness of the photosensitive layer is 70-90 nm.

[0012] In one possible implementation, the perovskite layer covers one side surface of the metal oxide layer relative to the transparent substrate, and the photosensitive layer further includes a transparent encapsulation layer; the transparent encapsulation layer covers one side surface of the perovskite layer relative to the transparent substrate.

[0013] In one possible implementation, the thickness of the transparent encapsulation layer is no greater than 200 μm.

[0014] In one possible implementation, the photosensitive layer is an array of at least two spaced photosensitive units, each of which is connected to a set of electrodes.

[0015] In one possible implementation, the spacing between any two adjacent photosensitive units in the array is 3-5 mm.

[0016] In one possible implementation, the distance between any two points on the visible light receiving surface of the photosensitive unit in the array does not exceed 3 mm.

[0017] In one possible implementation, the electrode includes interdigitated electrodes; both the positive and negative electrodes of the interdigitated electrodes are in contact with the photosensitive layer.

[0018] In one possible implementation, the thickness of the transparent substrate does not exceed 1 mm. Attached Figure Description

[0019] Figure 1 A schematic diagram illustrating the composition of a synapse-like dual-mode sensing device provided in an embodiment of this application;

[0020] Figure 2 A partial schematic diagram of a cross-section of a synapse-like dual-mode sensing device provided in an embodiment of this application;

[0021] Figure 3A schematic diagram illustrating a method for fabricating a synapse-like dual-mode sensing device provided in an embodiment of this application;

[0022] Figure 4 A schematic diagram of the structure of another synapse-like dual-mode sensing device provided in an embodiment of this application;

[0023] Figure 5 A diagram showing the postsynaptic current changes of a synaptic-like dual-mode sensing device provided in this application when stimulated by light pulse signals of different durations and intensities;

[0024] Figure 6 The diagram shows the postsynaptic current changes of a synaptic-like dual-mode sensing device provided in this application under stimulation by light pulse signals of different wavelengths.

[0025] Figure 7 The diagram shows the postsynaptic current changes of a synaptic-like dual-mode sensing device provided in this application under stimulation by different numbers of light pulse signals.

[0026] Figure 8 The diagram shows the postsynaptic current changes of a synaptic-like dual-mode sensing device provided in this application under stimulation by light pulse signals of different frequencies.

[0027] Figure 9 A test diagram of the memory capacity of a synaptic dual-mode sensing device under light pulse signal stimulation, provided in an embodiment of this application;

[0028] Figure 10 Test diagram of the luminescence of the force-receiving layer in a synaptic dual-mode sensing device provided in this application embodiment under the stimulation of force pulse signals of different intensities;

[0029] Figure 11 The diagram shows the postsynaptic current changes of a synaptic-like dual-mode sensing device provided in this application under the stimulation of force pulse signals of different intensities.

[0030] Figure 12 The diagram shows the postsynaptic current change of a synaptic-like dual-mode sensing device provided in this application under stimulation by force pulse signals of different numbers;

[0031] Figure 13 The diagram shows the postsynaptic current changes of a synaptic-like dual-mode sensing device provided in this application under stimulation by force pulse signals of different frequencies.

[0032] Figure 14 A test diagram of the memory capacity of a synaptic dual-mode sensing device under force pulse signal stimulation, provided in an embodiment of this application;

[0033] Figure 15 Based on Pavlov's dog experiments, this diagram illustrates the postsynaptic current changes of a synaptic-like dual-mode sensing device provided in this application embodiment under simultaneous stimulation by force pulse signals and light pulse signals.

[0034] Figure 16 This is a diagram showing the postsynaptic current change of a single photosensitive unit in a synaptic dual-mode sensing device provided in this application embodiment under simultaneous stimulation by force pulse signal and light pulse signal;

[0035] Figure 17 This is a test image of the image memory capability of a synapse-like dual-mode sensing device provided in an embodiment of this application;

[0036] Figure 18 This is a test image showing the image recognition accuracy of a synapse-like dual-mode sensing device provided in an embodiment of this application.

[0037] Reference numerals: 10-Synaptic dual-mode sensing device; 11-Transparent substrate; 12-Force-receiving layer; 13-Photosensitive layer; 131-Metal oxide layer; 132-Perovskite layer; 14-Electrode. Detailed Implementation

[0038] To address the issue of complex structures in existing dual-mode sensing devices, this application provides a synaptic-like dual-mode sensing array: a force-receiving layer and a photosensitive layer are respectively disposed on opposite sides of a transparent substrate, enabling the photosensitive layer to respond and output an electrical signal when the force-receiving layer emits visible light under force, and to determine information such as intensity based on the electrical signal. This significantly reduces the structural complexity of the dual-mode sensing device while achieving visual and tactile dual-mode sensing, thereby solving the problem of limited application range (scenario) of dual-mode sensors due to their large size and complex structure.

[0039] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0040] This application provides a synapse-like dual-mode sensing device that integrates a force detector and a photodetector into one unit, reducing material costs while significantly reducing the structural complexity of the dual-mode sensing device. Figure 1 This application provides a synapse-like dual-mode sensing device. For example... Figure 1As shown, this type of synaptic dual-mode sensing device 10 includes: a transparent substrate 11, a photosensitive layer 13, a force-receiving layer 12, and electrodes 14. The force-receiving layer 12 and the photosensitive layer 13 are distributed on both sides of the transparent substrate 11. That is, as... Figure 1 As shown, the photosensitive layer 13 is disposed on the right side surface of the transparent substrate 11, and the force-bearing layer 12 is disposed on the left side surface of the transparent substrate 11. That is, the surface of the transparent substrate 11 where the photosensitive layer 13 is located is relative to the surface of the transparent substrate 11 where the force-bearing layer 12 is located.

[0041] The projected area of ​​the stress-bearing layer 12 on the transparent substrate 11 is not greater than the projected area of ​​the photosensitive layer on the transparent substrate 11, so as to ensure that the visible light emitted by the stress-bearing layer 12 after being stressed is conducted to the photosensitive layer 13 through the transparent substrate 11.

[0042] The aforementioned projected area refers to the area enclosed by the outer contour of the shadow of the corresponding detector (photosensitive layer 13 or force-receiving layer 12) projected vertically onto the transparent substrate 11. In other words, the force-receiving layer 12 and photosensitive layer 13 can remain in direct contact with the transparent substrate 11 without affecting visual or tactile sensation; however, when the force-receiving layer 12 and photosensitive layer 13 are in direct contact with the transparent substrate 11, the projected area is equal to the area occupied by the corresponding (force-receiving layer 12 or photosensitive layer 13) on the transparent substrate 11. Taking the force-receiving layer 12 as an example, when the material of the force-receiving layer 12 is solely a mechanoluminescent material, the force-receiving layer 12 does not directly contact the transparent substrate 11, and a transparent fixing device should be provided between the force-receiving layer 12 and the transparent substrate 11 to ensure that the force-receiving layer 12 does not fall off. When the force-receiving layer 12 is a mixture of mechanoluminescent material and transparent acrylic adhesive, the force-receiving layer 12 can be directly attached to one side of the transparent substrate 11 by spin coating, and the distributed area in this case is its projected area.

[0043] The photosensitive layer 13 is made of a metal oxide. Examples of the metal oxide include at least one of IGZO (indium gallium zinc oxide), ZnO, and IZO (indium zinc oxide).

[0044] The stress-bearing layer 12 can be a mechanoluminescent material layer. The mechanoluminescent material in the mechanoluminescent layer can be at least one of ZnS-CaZnOS, SrAl2O4:(Eu,Dy,Nd), and ZnS:(Mn,Cu,Te,Al).

[0045] To ensure that the light emitted by the stress-bearing layer 12 after being stressed reaches the photosensitive layer 13 through the transparent substrate 11, in one embodiment of this application, the thickness of the transparent substrate 11 does not exceed 1 mm. For example, it can be 10 μm, 50 μm, or 200 μm. The material of the transparent substrate 11 can be at least one of glass, PEN (Polyethylene naphthalate), PET (Polyethylene terephthalate), and PI (Polyimide).

[0046] The synaptic characteristics exhibited by the dual-mode sensing device 10, in which the photosensitive layer 13 and the force-receiving layer 12 are respectively disposed on both sides of the transparent substrate 11, mainly refer to the fact that the response to light / force stimulation is not immediate, but rather has the function of learning and memory. For example, during a first time period, a mechanical force of a set frequency and intensity is applied to the force-receiving layer 12 in the dual-mode synaptic sensing device 10. The photosensitive layer 12 responds to the visible light emitted after the force-receiving layer 12 is subjected to the force, outputs a current, and outputs an electrical signal through the electrode 14. After a period of learning (i.e., training), a mechanical force of less intensity and / or a mechanical force of less frequency than that used in the training phase can be used, so that the dual-mode synaptic sensing device 10, as described above, outputs an electrical signal of the same intensity as that used in the training phase.

[0047] Furthermore, although the metal oxide layer 131 exhibits good synaptic performance under illumination due to its oxygen vacancies, it primarily responds to ultraviolet light within the visible spectrum, rather than all visible light. Therefore, in one embodiment of this application, the photosensitive layer 13 further includes a perovskite layer 132 that is responsive to all visible light. Please refer to [reference needed]. Figure 2 The aforementioned metal oxide layer 131 was prepared by magnetron sputtering under argon conditions. This perovskite material can be, for example, MAPbI3 (chemical formula: CH3NH3PbI3, Chinese name: methylamine iodide perovskite), CH3NH3PbI... 3-x Cl or CsPbBr3.

[0048] Specifically, under illumination, perovskite materials generate photogenerated charge carriers. Since the energy level of perovskite materials is higher than that of metal oxides, the energy level difference between the two (absolute value not greater than 0.4 eV) allows electrons from the photogenerated charge carriers to be injected into the metal oxides, thereby increasing the conductivity of the metal oxides.

[0049] Therefore, continue to refer to Figure 2In this embodiment, the perovskite and metal oxide layers in the photosensitive layer 13 are each layered and overlapped in a direction away from the transparent substrate 11, so that electron migration can occur between the perovskite layer 132 and the metal oxide layer 131 based on the energy level difference. Preferably, both the perovskite layer 132 and the metal oxide layer 131 are regular cubes with the same area and shape of their contact surfaces.

[0050] The following example uses MAPbI3 as the perovskite material and IGZO as the metal oxide: MAPbI3 has an energy level distribution of -5.4 to -3.9 eV, while IGZO has an energy level distribution of -7.6 to -4.3 eV. (Continue to refer to...) Figure 2 Assuming the perovskite layer 132 is illuminated, photogenerated carriers are generated in the perovskite layer 132 (MAPbI3). Electrons from these carriers are injected into the metal oxide layer 131 (IGZO) under the influence of energy level differences. On one hand, only electrons from the photogenerated carriers generated in MAPbI3 enter the IGZO; holes remain trapped within MAPbI3. The positive potential carried by these holes forms a bias voltage, enabling the grating effect to form. This bias voltage from the grating effect enhances the sensitivity of the perovskite layer 132 to weak light, allowing MAPbI3 to respond to intensities as low as 1 μW / cm². 2 The wavelength is less than 600 nm, which is weak light. On the other hand, the electrons injected into the metal oxide layer 131, i.e., IGZO, increase the carrier concentration in IGZO, thereby improving the conductivity in IGZO. At the same time, the electrons injected into IGZO are bound by positively charged oxygen vacancies, which significantly prolongs the recombination time between the injected electrons and holes in MAPbI3, thus exhibiting good memory performance.

[0051] The thickness of the aforementioned photosensitive layer 13 can be 70-90 nm.

[0052] It should be noted that, Figure 2 The light source is an LED bulb, but for the photosensitive layer 13, it only needs to receive visible light, without any restrictions on the light source. This light source can also be emitted from the force-bearing layer 12 on the back of the transparent substrate 11 after being subjected to force.

[0053] Accordingly, the relative positions of the perovskite layer 132 and the metal oxide layer 131 are not specifically limited in the embodiments of this application; that is, they can be positioned as follows: Figure 2 The overlapping form shown is as follows: the perovskite layer 132 is attached to the surface of the metal oxide layer 131 away from the transparent substrate 11; alternatively, the perovskite layer 132 is attached to the surface of the metal oxide layer 131 facing the transparent substrate 11. When the relative positions of the perovskite layer 132 and the metal oxide layer 131 are... Figure 2In the relative positional relationship shown, to prevent the water-absorbing perovskite from becoming damp and failing, in one embodiment of this application, the photosensitive layer 13 further includes a transparent encapsulation layer ( Figure 2 (Not shown), the transparent encapsulation layer covers one side of the perovskite layer relative to the transparent substrate 11, that is, the side of the perovskite layer away from the transparent substrate 11 and facing the air.

[0054] The thickness of the aforementioned transparent encapsulation layer is no greater than 200 μm. The material of this encapsulation layer is a transparent polymeric organic material with adhesiveness and stretchability, such as polydimethylsiloxane (PDMS).

[0055] To further enhance the sensitivity of the dual-mode sensing device 10, enabling it to exhibit a more timely and accurate response to light / force / stimuli, in one embodiment of this application, the photosensitive layer 13 can be an array composed of at least two spaced photosensitive units. Each photosensitive unit in the array can include a transparent encapsulation layer, a perovskite layer 132, and a metal oxide layer 131. Each unit is connected to a set of electrodes, allowing an ammeter and a voltmeter to determine changes in conductivity, thereby achieving the sensing and memorization of mechanical and optical signals. The aforementioned voltmeter applies a readable voltage to the photosensitive unit through the positive and negative terminals connected to the electrodes 14, allowing the ammeter to test changes in conductivity within the photosensitive unit.

[0056] Furthermore, the distance between any two photosensitive units can be the same or not; if the corresponding sides of the photosensitive units are parallel, then the photosensitive unit can be at least one of a cube, a cuboid, and a frustum.

[0057] The spacing between any two units is 3-5 mm. The distance between any two points on the visible light receiving surface of the photosensitive unit, i.e., the side away from the transparent substrate 11 (relative to the transparent substrate 11), does not exceed 3 mm. For example, when the photosensitive unit is a cube, its long side can be 2-3 mm.

[0058] Obviously, when the photosensitive layer 13 is an array, the force-bearing layer 12, which is located on the other side of the transparent substrate 11, can also be an array; and the position of each unit in the force-bearing layer 12 on the transparent substrate 11 corresponds to that of the photosensitive unit.

[0059] Furthermore, in this embodiment, electrode 14 includes an external negative electrode, an external positive electrode, and interdigitated electrodes. The positive and negative electrodes of each interdigitated electrode are connected to the photosensitive unit, and the positive and negative electrodes are respectively connected to the external negative electrode and the external positive electrode. The area of ​​the interdigitated electrodes matches the area of ​​a single photosensitive unit. When the photosensitive layer 13 includes a metal oxide layer 131 and a perovskite layer 132, refer to... Figure 2The positive and negative electrodes of the interdigital electrodes are only connected to the metal oxide layer 131 to achieve a sensitive response to changes in low conductivity. Furthermore, when the photosensitive layer 13 is arranged in an array on one side of the transparent substrate 11, the interdigital electrodes are also arranged in an array accordingly, and together with their respective external negative and external positive electrodes, they form an interdigital electrode array circuit.

[0060] Based on the same inventive concept, this application provides a method for fabricating a synapse-like dual-mode sensing device 10 as described above, which is described in detail below:

[0061] Figure 3 This is a schematic diagram illustrating a method for fabricating a synapse-like dual-mode sensing device 10, as provided in an embodiment of this application. Please refer to... Figure 3 First, a (metal) circuit is fabricated on a cleaned transparent substrate 11 (e.g., glass) using photoresist and magnetron sputtering. When current is generated in the photosensitive layer 103, the change in conductivity can be detected in real time through this circuit. When the photosensitive layer 102 is an array of photosensitive units ( Figure 3 (Not shown) The interdigital electrodes are also arranged in an array on the transparent substrate 11 at positions corresponding to the photosensitive units. The circuit described above can be made of gold interdigital electrodes with a channel width of 100 μm and a thickness of 30 nm.

[0062] Then, apply a masking tape so that the tape covers the part of the circuit except for the interdigital electrodes, leaving only the interdigital electrodes exposed.

[0063] Next, the photosensitive layer 13 is attached above the interdigitated electrode. The perovskite layer 132 can be attached to the upper surface of the metal oxide layer 131 via a one-step spin-coating method, and the metal oxide layer 131 can be attached to the upper surface of the interdigitated electrode via a deposition method. Finally, a transparent encapsulation layer is used to cover the air-facing side of the photosensitive layer. The following example uses MAPbI3 for the perovskite layer 132 and IGZO for the metal oxide layer 131: their relative positions are as follows: IGZO is located below MAPbI3, i.e., MAPbI3, IGZO, and the interdigitated electrode form a "sandwich" structure, with the interdigitated electrode and MAPbI3 on ​​the top and bottom sides, and IGZO in the middle; to facilitate the application of a readable voltage, the interdigitated electrode below the IGZO is designed to expose at least the positive and negative electrodes for connection. First, an IGZO layer is deposited above the interdigitated electrode, and then the surface of the IGZO layer is treated with oxygen plasma to make it hydrophilic. MAPbI3 was then spin-coated onto a hydrophilic IGZO surface, with an antisolvent added dropwise during the spin-coating process, followed by annealing to obtain the MAPbI3 layer. The antisolvent includes, but is not limited to, acetone, chlorobenzene, chloroform, and ethyl acetate.

[0064] Finally, a material containing a mechanoluminescent material (e.g., ZnS-CaZnOS:Mn) is spin-coated onto the back side of the transparent substrate 11, i.e., the other side opposite to the layered photoreceptor, and dried to obtain a stress-bearing layer 12. This material may also include, for example, epoxy resin-based acrylic adhesive to promote efficient curing of the powdered mechanoluminescent material and ensure its stable adhesion to the surface of the transparent substrate. The mass ratio of acrylic adhesive to mechanoluminescent material is preferably 7:3. The drying temperature is preferably 75-85°C, and the drying time is preferably 3-5 hours. The spin-coating speed can be 1000 rpm, and the spin-coating time can be 60 seconds.

[0065] The following example illustrates the preparation of a mechanoluminescent material: (1) Eggshell powder, ZnS powder, and MnCO3 powder are mixed in a molar ratio of Ca:Zn:Mn = 1:2:0.01, anhydrous ethanol is added, the mixture is ball-milled for 30 minutes, and then dried. (2) The dried mixture is transferred to a corundum crucible. (3) The mixture is calcined in a furnace at 1100°C for 4 hours under 80 sccm of argon gas, and then ground to obtain ZnS-CaZnOS:Mn powder.

[0066] The following specific examples illustrate this point.

[0067] Prepared according to the aforementioned method, such as Figure 4 The illustrated synapse-like dual-mode sensing device 10 has a force-bearing layer 12 with a thickness of 200 μm and a photosensitive layer 13 arranged in a 6×6 array. Each photosensitive unit in the array comprises, from top to bottom, a transparent encapsulation layer (not shown), a perovskite layer 132 (200 nm thick, made of MAPbI3), and a metal oxide layer 131 (80 nm thick, made of IGZO). The force-bearing layer 12 is located on the other side of the transparent substrate 11 (1 mm thick, made of glass) opposite to the photosensitive layer 13, i.e., on the back side of the transparent substrate 11.

[0068] Now, assuming that the light source directly above the dual-mode sensing device 10 of this type of synapse is turned off, and the force layer 12 is scratched by mechanical force in the shape of "H", it will emit "H"-shaped light. Then, the photosensitive unit at the corresponding "H" position in the photosensitive layer 13 will respond, and the generated current will flow into the ammeter through the interdigital electrode, the external positive electrode, and the external negative electrode, so as to read the current change. Since each photosensitive unit in the array has its own set of electrodes 14 (interdigital electrodes, external positive electrode, and external negative electrode), a voltmeter (model: Keithley 4200SCS, Stanford DS345) can be used to apply a 1V reading voltage to each photosensitive unit during testing. Then, each photosensitive unit is connected to an ammeter (model: Keithley 4200SCS, Stanford SR570), or a single ammeter can be used to test the photosensitive units in the array sequentially to determine the change in conductivity at each point in the array under the stimulation of optical and mechanical signals, thereby realizing the perception and memory of dual-mode signals.

[0069] The following tests were performed on the dual-mode sensing device 10 for this type of synapse to verify its sensitivity:

[0070] Figure 5 To obtain the postsynaptic current change diagram of the aforementioned synaptic dual-mode sensing device 10, which was continuously irradiated for 10 seconds under different times and intensities of light (light pulse signal), the following was determined. Figure 5 It can be seen that under the stimulation of orange light with a wavelength of 600nm, the change in postsynaptic current increases continuously with the increase of light intensity and irradiation time, which shows that orange light can effectively regulate the synaptic weight of the artificial visual-tactile dual-mode sensory array.

[0071] The postsynaptic current change indicates the change in current over a corresponding time period after the aforementioned synaptic-like dual-mode sensing device 10 is stimulated. A larger postsynaptic current change means a larger change in the synaptic weight of the dual-mode sensing device 10, corresponding to a stronger memory effect.

[0072] Synaptic weight refers to the strength or amplitude of the connection between two nodes (equivalent to the two identical synaptic-like dual-mode sensing devices 10 in this embodiment), and in biology corresponds to the amount of influence of a neuron's firing on another node. Here, it represents the change in synaptic conductivity to simulate the connection strength between neurons in a living organism.

[0073] Furthermore, the dual-mode sensing device 10, which resembles a synapse, is illuminated with light of equal intensity but different wavelengths (light pulse signals) to obtain the postsynaptic current change, as shown below. Figure 6As shown, the synaptic-like dual-mode sensing device 10 provided in this application embodiment responds to light within the visible light range. Furthermore, under the same illumination conditions, the postsynaptic current change gradually increases as the wavelength of light decreases; and the postsynaptic current change gradually increases as the illumination time increases. Therefore, the synaptic weight of this synaptic-like dual-mode sensing device 10 can be controlled by multiple wavelengths.

[0074] Furthermore, using a wavelength of 600 nm and a light intensity of 1 μW / cm² 2 A light pulse signal (with a pulse width of 500 ms and a frequency of 1 Hz) is used to stimulate the aforementioned synaptic-like dual-mode sensing device 10 multiple times, resulting in the following: Figure 7 The diagram shows the changes in postsynaptic current. (From...) Figure 7 Thus, the synaptic dual-mode sensing device 10 has the characteristic of modulating synaptic weights by the number of light pulses; by increasing the number of light pulse signal stimuli, the synaptic dual-mode sensing device 10 can realize the conversion from short-term memory to long-term memory.

[0075] Furthermore, using different frequencies (wavelength 600nm, light intensity 1μW / cm²) 2 A light pulse signal with a pulse width of 500 ms is used to stimulate the aforementioned synapse-like dual-mode sensing device 10, resulting in the following: Figure 8 The diagram shows the changes in postsynaptic current. (From...) Figure 8 It can be seen that the synaptic dual-mode sensing device 10 has the characteristic of optical pulse frequency modulation of synaptic weight; as the received optical pulse frequency continues to increase and the postsynaptic current change value continues to increase, the synaptic dual-mode sensing device 10 can gradually realize the conversion from short-term memory to long-term memory.

[0076] Furthermore, the memory capacity of the aforementioned synaptic-like bimodal sensory array was tested by simulating the human body's secondary learning process using optical signals. Figure 9 As shown, in the first learning phase, the postsynaptic current of the synaptic-like dual-mode sensor 10 gradually increased from 1.5 nA to 3.5 nA under 36 consecutive light pulses of stimulation. Then, when illumination was stopped, the postsynaptic current decayed back to a midpoint of 1.5 nA within 50 seconds. This behavior is analogous to how a person partially forgets learned information over time. (Continue to reference...) Figure 2The second learning process only requires 16 light pulse signals of the same intensity, frequency, and wavelength as the first learning. This type of synaptic dual-mode sensor 10 can achieve the same postsynaptic current level as the first learning, corresponding to the shorter learning time required for humans to achieve the same memory effect during second learning. Furthermore, the decay of the postsynaptic current change after 50 seconds is less than that after the first learning; that is, the postsynaptic current change 50 seconds after the second learning is higher than that after the first learning. Therefore, this type of synaptic dual-mode sensor 10 demonstrates a better learning effect when relearning forgotten information than when initially learning it, similar to the human body. In other words, this type of synaptic dual-mode sensor 10 can effectively simulate the second learning process of human vision.

[0077] Figures 5-10 Since the force-bearing layer 12 was not subjected to mechanical force, the illumination parameters above the synaptic dual-mode sensing device 10 were controlled to test the visual perception and memory process simulated by the synaptic dual-mode sensing device 10 provided in this application embodiment under the stimulation of light pulse signals.

[0078] To further determine the coupling effect between the force-receiving layer 12 and the photosensitive layer 13 in the dual-mode sensing device 10 of this type of synapse, the dependence of the luminescence intensity of the force-receiving layer 12 on the applied force was tested, see [reference needed]. Figure 10 .Depend on Figure 10 It can be seen that the force-bearing layer 12 can respond to forces as small as 2N, and the luminescence intensity of the force-bearing layer 12 increases with the increase of the applied force intensity, with a peak wavelength of 600nm.

[0079] Furthermore, closing such Figure 4 The light source shown above applies mechanical forces (force pulse signals) of different intensities to the force-receiving layer 12 of the synaptic dual-mode sensing device 10, and the changes in postsynaptic current are obtained as follows. Figure 11 As shown, the postsynaptic current change of this type of dual-mode synaptic sensing device 10 increases continuously with the enhancement of the force pulse signal.

[0080] Furthermore, it remains closed, such as Figure 4 The light source shown above applies force pulse signals of different numbers and the same intensity (force of 5N, pulse width of 400ms, and frequency of 1Hz) to the force-receiving layer 12 of the synaptic dual-mode sensing device 10, resulting in the following: Figure 12 The diagram shows the postsynaptic current changes. (Still closed as shown) Figure 4 The light source shown above applies force pulse signals of different frequencies (force of 5N, pulse width of 400ms) to the force-receiving layer 12 of the synaptic dual-mode sensing device 10, resulting in the following... Figure 13 The diagram shows the changes in postsynaptic current. (From...) Figures 12-13 It is evident that this type of dual-mode synaptic sensing device 10 can realize the process of sensing, learning, and memorizing tactile signals.

[0081] Furthermore, it remains closed, such as Figure 4 The light source shown above simulates the human body's secondary learning process through mechanical signals to test the memory capacity of the aforementioned synapse-like dual-mode sensing device 10, and obtains... Figure 14 .like Figure 14 As shown, in the first learning process, stimulation with 32 force pulse signals resulted in a postsynaptic current change of 1.5 nA in the synaptic-like bimodal sensor 10, which decayed to 0.6 nA after a period of time. In the second learning process, only 14 force pulse signals were needed to achieve the same postsynaptic current change as in the first learning. Clearly, the synaptic-like bimodal sensor 10 demonstrates a better learning effect in relearning forgotten information than in the initial learning, meaning it effectively simulates the secondary learning process of human touch.

[0082] Furthermore, based on joint learning—Pavlov's dog experiment—the bimodal sensory (response) and memory capabilities of the bimodal sensory device 10 of this type of synapse were tested; that is, the visual and tactile bimodal response and memory capabilities of the bimodal sensory device 10 of this type of synapse were stimulated using force pulse signals and light pulse signals, respectively. See [link to relevant documentation] Figure 15 First, ten consecutive force pulse signals (force magnitude 5N, duration 400ms, frequency 1Hz) were applied to the force-receiving layer 12 to simulate a bell-shaped (conditioned) stimulus, resulting in a slight increase in postsynaptic current (1.86nA), below the salivation threshold (2nA). Then, ten consecutive light pulse signals (wavelength 600nm, light intensity 1μW cm⁻², width 500ms, frequency 1Hz) were applied to the photosensitive layer 13 to mimic food vision (unconditioned stimulus), resulting in a high postsynaptic current change (2.33nA), activating the salivation response. During the training phase, both the bell-shaped (conditioned) and food vision (conditioned) stimuli were applied simultaneously to the device. The postsynaptic current change in this type of bimodal sensory device was significantly larger (2.80nA), far exceeding the salivation threshold. After training, a single rattle (conditioned stimulus) elicited a postsynaptic current change (2.44 nA) above the salivation threshold, indicating that an association had been established between the rattle (conditioned stimulus) and the food visual / unconditioned stimulus. It should be noted that the rattle (conditioned stimulus) occurred within 100 seconds of the end of training to ensure that a salivation response could be induced.

[0083] Furthermore, the sensitivity of the photosensitive unit of the photosensitive layer 13 in the synaptic dual-modal sensing device to the response of light pulse signals and force pulse signals was tested, and the postsynaptic current change was obtained as follows: Figure 16 As shown, for a single photosensitive unit, both 10 consecutive light pulses and 10 consecutive force pulses can stimulate the synaptic bimodal sensing array to generate postsynaptic current changes. The postsynaptic current change resulting from the combined effect of the two signals is greater than the change resulting from either signal acting alone. Therefore, this synaptic bimodal sensing device possesses better memory capabilities under the combined effect of mechanical force and visible light.

[0084] Furthermore, the image memory capability of this type of synapse in a bimodal sensing device was tested. Combined with... Figure 4 Under stimulation by light pulse signals, force pulse signals, or a combination of both, after training, an "H" shaped letter pattern can be clearly displayed on both the force-receiving layer 12 and the photosensitive layer 11. Figure 17 As shown, after applying individual light pulse signals or force pulse signals for stimulation, the image gradually becomes blurred within 30s or 60s. When the two pulse signals are stimulated synergistically, the image can still be recognized even after 100s. This demonstrates that the memory capacity of this type of synaptic dual-mode sensing device 10 can be enhanced under synergistic stimulation. Furthermore, this type of synaptic dual-mode sensing array exhibits good perception (response), learning, and memory capabilities for the individual or synergistic patterning of light and mechanical signals.

[0085] Furthermore, the average recognition accuracy of the synaptic-like dual-mode sensing device 10 was tested under different training conditions. See [link to documentation]. Figure 18As can be seen, when light pulse signals and force pulse signals are fused, the recognition accuracy exhibits a high saturation recognition accuracy (~70%), which is higher than that of light pulses alone (~65%) or force pulses alone (~61%). Therefore, this type of synaptic bimodal sensory array can achieve high recognition accuracy under bimodal training. The following is a brief explanation of the aforementioned average recognition accuracy test: The supervised learning process of the synaptic bimodal sensory device 10 was simulated using a handwritten letter dataset and an artificial neural network (ANN). The ANN consists of an input layer, an output layer, and a synaptic connection layer. Each layer consists of 785 input neurons (i.e., 1 bias neuron and 28×28 neurons), 10 output neurons (from 0 to 9), and 785×10=7850 synapses. The simulation process of the ANN is divided into a feedforward process and a backpropagation process. In the feedforward process, the training image is divided into 28×28 pixels, and each pixel is converted into a voltage vector (V), which is multiplied by the weight matrix (W) of the synaptic array to obtain a current vector (I). The current vector is then transformed into an output vector (y) through a sigmoid activation function y = f(I). The deviation between the output vector and the label value (k) of the input image is determined and backpropagated to the synaptic layer of the ANN to guide the weight update process, which is performed by applying a corresponding number of light / force pulses. Here, the conductivity change of a synaptic-like dual-mode sensing device 10 is used to simulate the weight update process of the neural network, ultimately affecting the recognition rate. Using this method, we successfully completed the recognition of the 26 letters "A" to "Z" and obtained the recognition accuracy by expanding the number of MNIST training datasets (188958) and test datasets (31346).

[0086] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A synapse-like dual-mode sensing device, characterized in that, include: A transparent substrate, a force-bearing layer, a photosensitive layer, and an electrode in contact with the photosensitive layer; the force-bearing layer and the photosensitive layer are distributed on both sides of the transparent substrate, wherein, The projected area of ​​the stress-bearing layer on the transparent substrate is not greater than the projected area of ​​the photosensitive layer on the transparent substrate. The photosensitive layer includes a metal oxide layer, and the stress-bearing layer includes a mechanoluminescent material.

2. The dual-mode sensing device as described in claim 1, characterized in that, The photosensitive layer also includes a perovskite layer, which is disposed overlapping the metal oxide layer.

3. The dual-mode sensing device as described in claim 2, characterized in that, The thickness of the photosensitive layer is 70-90 nm.

4. The dual-mode sensing device as described in claim 2, characterized in that, If the perovskite layer covers one side of the metal oxide layer relative to the transparent substrate, then the photosensitive layer further includes a transparent encapsulation layer; the transparent encapsulation layer covers one side of the perovskite layer relative to the transparent substrate.

5. The dual-mode sensing device as described in claim 4, characterized in that, The thickness of the transparent encapsulation layer is no greater than 200 μm.

6. The dual-mode sensing device according to any one of claims 1-5, characterized in that, The photosensitive layer is an array of at least two spaced photosensitive units, and each photosensitive unit is connected to a set of electrodes.

7. The dual-mode sensing device as described in claim 6, characterized in that, The spacing between any two adjacent photosensitive units in the array is 3-5 mm.

8. The dual-mode sensing device as described in claim 6, characterized in that, The distance between any two points on the visible light receiving surface of the photosensitive unit in the array does not exceed 3 mm.

9. The dual-mode sensing device according to any one of claims 1-5.7-8, characterized in that, The electrode includes interdigitated electrodes; both the positive and negative electrodes of the interdigitated electrodes are in contact with the photosensitive layer.

10. The dual-mode sensing device according to any one of claims 1-5.7-8, characterized in that, The transparent substrate is no more than 1 mm thick.

Images