Neuro-morphic tactile and proximity fusion sensor and signal processing method thereof

Abstract

The present application relates to the technical field of haptic and proximity sensation fusion detection devices, and particularly relates to a neuro-mimetic haptic and proximity sensation fusion receptor and a signal processing method thereof. The receptor comprises a multi-modal perception array layer and a neuro-mimetic processing array layer. The multi-modal perception array layer comprises a haptic sensing array layer formed by a plurality of haptic sensing unit arrays and a proximity sensing array layer formed by a plurality of proximity sensing unit arrays. The neuro-mimetic processing array layer comprises a plurality of array-arranged memristor units. Each memristor unit is electrically connected with at least one haptic sensing unit and at least one proximity sensing unit. The corresponding disclosed signal processing method is used. The fusion receptor uses multi-modal receptors and memristors to simulate biological neural circuits. The memristor array is used as a front-end processing unit to perform pulse coding, space-time fusion and feature extraction on the sensing signals of the haptic sensing unit and the proximity sensing unit.

Description

Technical Field

[0001] This invention relates to the field of tactile and proximity fusion detection devices, and in particular to a neuromorphic tactile and proximity fusion sensor and its signal processing method. Background Technology

[0002] Enabling robots and intelligent prosthetics to possess human-like or even superhuman environmental perception and interaction capabilities is a current challenge in the fields of robotics and electronic skin. Among these, tactile perception is the foundation for dexterous manipulation, while proximity perception (non-contact perception) enables early warning and prediction. The integration of the two can significantly improve the safety and intelligence of operations.

[0003] Existing tactile sensors are mostly based on resistance, capacitance or piezoelectric principles. Although their sensitivity is constantly improving, they generally have the following problems: they are mostly single-modal sensing and lack hardware-level integration and information fusion with proximity sensing; signal processing relies on the traditional von Neumann computing architecture, which requires transmitting massive amounts of sensing data to the central processing unit, resulting in large latency and high power consumption; rigid or semi-flexible designs are difficult to adapt to complex curved surfaces.

[0004] Neuromorphic computing, mimicking the event-driven and in-memory computing characteristics of the biological brain, offers a solution for efficiently processing sparse and asynchronous signals generated by sensors. Memristors, as synaptic biomimetic devices, are ideal components for realizing hardware neural networks. However, how to deeply integrate high-performance flexible multimodal sensors with flexible neuromorphic processing circuits (such as memristor arrays) both physically and functionally to form a compact, efficient, and flexible perception-decision closed-loop system remains a pressing technical challenge. Summary of the Invention

[0005] The purpose of this invention is to provide a neuromorphic tactile and proximity sensor fusion sensor and its signal processing method. The sensor is flexibly integrated with multimodal perception and front-end neuromorphic processing, and its signal processing method completes information processing from perception to preliminary decision-making in a low-power and low-latency manner.

[0006] To achieve the above objectives, the present invention provides a neuromorphic tactile and proximity sensor fusion sensor, comprising a multimodal sensing array layer and a neuromorphic processing array layer independently formed on a flexible substrate. The multimodal sensing array layer includes a tactile sensing array layer formed by an array of tactile sensing units and a proximity sensing array layer formed by an array of proximity sensing units. The neuromorphic processing array layer includes a plurality of memristor units arranged in an array, and each memristor unit is electrically connected to at least one tactile sensing unit and at least one proximity sensing unit.

[0007] The tactile sensing unit, the proximity sensing unit, and the memristor unit are connected in the substrate plane via flexible interconnecting wires to form a local neural circuit integrating perception and processing. The tactile and proximity sensing units are arranged in two layers, sharing some substrate and fabrication processes. Each or each group of sensing units is directly connected to a memristor unit below or adjacent to it, forming a distributed receptor-synaptic basic unit. When the proximity sensing unit detects an approaching object, the photocurrent signal it generates is used as a pre-pulse input to the connected memristor unit, modulating its synaptic weights, thereby pre-activating or changing the sensitivity of the signal processing channel of the connected tactile sensing unit.

[0008] Furthermore, the proximity sensing unit includes interdigital electrodes and a graphene thin film photosensitive layer. The interdigital electrodes are formed on the inner wall of the flexible substrate covering the proximity sensing unit by laser direct writing, and the graphene thin film photosensitive layer is coated on the gaps and surfaces of the interdigital electrodes.

[0009] The photosensitive layer of the graphene film is a conductive network obtained by laser reduction or thermal reduction of graphene oxide.

[0010] The proximity sensing unit operates on the principle of photoconductivity. Utilizing the broad-spectrum light absorption and efficient photogenerated carrier characteristics of graphene, when a nearby object reflects or blocks ambient light, it causes a change in the resistance of the photosensitive layer of the graphene film, thereby detecting the approach of the object non-contactly.

[0011] Furthermore, the tactile sensing unit includes a first upper electrode, a sensitive layer, a dielectric layer, and a first lower electrode. The sensitive layer is disposed on both sides of the dielectric layer. On one side, the first upper electrode is distributed between the sensitive layer and the dielectric layer, and on the other side, the first lower electrode is distributed between the sensitive layer and the dielectric layer.

[0012] The sensitive layer is made of a PDMS template with a sandpaper-like negative microstructure on its surface and an MXene material sprayed onto the microstructure surface of the PDMS template. The MXene material sprayed onto the microstructure surface of the PDMS template also forms the first upper electrode and the first lower electrode.

[0013] The MXene material is Ti3C2T. x .

[0014] The dielectric layer is PVA to provide a stable capacitive response.

[0015] The microstructure of the sensitive layer of the tactile sensing unit is prepared by the following method: using sandpaper as a template, PDMS template with negative microstructure is cast and cured to form a PDMS template; then, MXene material dispersion is sprayed onto the surface of the microstructure of the PDMS template; after drying, it is composited with a planar PDMS substrate; and after peeling, an MXene / PDMS composite sensitive layer with a sandpaper-like rough microstructure is obtained. The sandpaper template method replicates a micron-level rough structure. MXene, utilizing its conductivity, serves as both the first upper electrode and the first lower electrode; furthermore, the rough microstructure formed by MXene / PDMS significantly increases the effective contact area and deformation space under stress, thereby significantly improving the sensor's sensitivity and linear range. PVA (polyvinyl alcohol) serves as the dielectric layer, providing a stable capacitive response.

[0016] Furthermore, the memristor unit includes a second upper electrode, a second lower electrode, and a hafnium dioxide resistive switching functional layer. The second upper electrode and the second lower electrode are formed on the inner wall of the flexible substrate covering the memristor unit by laser direct writing. The second upper electrode and the second lower electrode are electrically connected to the tactile sensing unit and the proximity sensing unit.

[0017] The second upper electrode, the second lower electrode, the interdigitated electrode, and the interconnecting wires used for electrical connections are all formed in one step on a PET substrate using laser direct writing technology.

[0018] The memristor unit site adopts an Ag / HfO2 / Ag metal-insulator-metal structure. HfO2 is a mature resistive switching material, and its conductivity can be continuously adjusted under voltage pulses, perfectly simulating the enhancement and inhibition of biological synaptic weights. All materials can be deposited on PET using a low-temperature process, ensuring overall flexibility. The thickness of the hafnium dioxide resistive switching functional layer is 5-50 nm.

[0019] The present invention also provides a signal processing method for a neuromorphic tactile and proximity fusion receptor as described above, comprising the following steps: S1: When the proximity sensing unit detects a nearby object, it generates a simulated photocurrent signal. The simulated photocurrent signal is input to the memristor unit connected to it to pre-modulate the conductance of the memristor unit. At the same time, the simulated photocurrent signal is converted into the first characteristic pulse sequence to realize the spatiotemporal encoding of the proximity event. S2: When the tactile sensing unit is subjected to pressure, the change in capacitance is converted into a voltage signal by the readout circuit. The voltage signal is input to the connected memristor unit and converted into a second characteristic pulse sequence. S3: The memristor unit acts as a simulated synapse. By changing its conductance state, it performs spatiotemporal integration and correlation learning on the first and second characteristic pulse sequences, and outputs the fused pulse signal. S4: The fused pulse signal is directly used to drive the actuator, or transmitted to a higher-level neuromorphic processor for object recognition and decision-making.

[0020] In this signal processing method, the proximity signal serves as a warning input, first reaching the memristor and pre-modulating its conductance (synaptic weight). Subsequent tactile signals, passing through the modulated synapses, undergo altered transmission efficiency. This mechanism enables attentional focusing and pre-adaptation in biological neural systems, allowing the system to optimize its processing strategy for upcoming tactile information based on proximity data.

[0021] This invention discloses a neuromorphic tactile and proximity sensor fusion sensor and its signal processing method. The fusion sensor uses hardware circuitry to simulate biological neural circuits, enabling the system to have predictive and adaptive learning capabilities, significantly improving the interactive intelligence of robots in unstructured environments. Its signal processing method uses the memristor array as the front-end processing unit to perform pulse coding and spatiotemporal fusion on the sensory signals of the tactile and proximity sensing units, greatly reducing the amount of data that needs to be uploaded and redundant calculations, and realizing low-power, low-latency brain-like processing. At the same time, it uses flexible materials and low-temperature processes, with laser direct writing as the core graphics method, to achieve flexible, thin, and high-density integration of the entire system from perception to processing, and can be attached to any curved surface. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the structure of a neuromorphic tactile and proximity sensory fusion sensor without a flexible substrate according to the present invention.

[0024] Figure 2 This is a schematic diagram of the upper electrode connection of a neuromorphic tactile and proximity sensory fusion receptor according to the present invention.

[0025] Figure 3 This is a schematic diagram of the lower electrode connection of a neuromorphic tactile and proximity sensory fusion receptor according to the present invention.

[0026] Figure 4 This is a schematic diagram of the interdigital electrode of a neuromorphic tactile and proximity fusion sensor of the present invention.

[0027] Figure 5This is a schematic diagram showing the distribution of the electrical wires of the lower electrode in the sensor of a neuromorphic tactile and proximity fusion sensor according to the present invention.

[0028] Figure 6 This is a schematic diagram showing the distribution of the electrical wires of the upper electrode in the sensor of a neuromorphic tactile and proximity fusion sensor according to the present invention.

[0029] Figure 7 This is a flowchart of a signal processing method for a neuromorphic tactile and proximity sensory fusion receptor according to the present invention.

[0030] 1. Multimodal sensing array layer; 2. Neuromorphic processing array layer; 3. Flexible substrate; 4. Tactile sensing array layer; 5. Proximity sensing array layer; 41. Tactile sensing unit; 51. Proximity sensing unit; 61. Memristor unit; 411. First upper electrode; 412. Sensitive layer; 413. Dielectric layer; 414. First lower electrode; 511. Interdigital electrode; 512. Graphene film photosensitive layer; 611. Second upper electrode; 612. Second lower electrode; 613. Hafnium dioxide resistive switching functional layer. Detailed Implementation

[0031] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0032] Please see Figures 1 to 5 The present invention provides a neuromorphic tactile and proximity sensor fusion sensor, comprising a multimodal sensing array layer 1 and a neuromorphic processing array layer 2 independently formed on a flexible substrate 3. The multimodal sensing array layer 1 includes a tactile sensing array layer 4 formed by an array of several tactile sensing units 41 and a proximity sensing array layer 5 formed by an array of several proximity sensing units 51. The neuromorphic processing array layer 2 includes several memristor units 61 arranged in an array. Each memristor unit 61 is electrically connected to at least one tactile sensing unit 41 and at least one proximity sensing unit 51.

[0033] The proximity sensing unit 51 includes interdigitated electrodes 511 and a graphene thin film photosensitive layer 512. The interdigitated electrodes 511 are formed on the inner wall of the flexible substrate 3 covering the proximity sensing unit 51 by laser direct writing. The graphene thin film photosensitive layer 512 is coated on the gaps and surfaces of the interdigitated electrodes 511.

[0034] The graphene film photosensitive layer 512 is a conductive network obtained by laser reduction or thermal reduction of graphene oxide.

[0035] The tactile sensing unit 41 includes a first upper electrode 411, a sensitive layer 412, a dielectric layer 413, and a first lower electrode 414. The sensitive layer 412 is disposed on both sides of the dielectric layer 413, and the first upper electrode 411 and the first lower electrode 414 are respectively distributed between the sensitive layer 412 and the dielectric layer 413 on both sides.

[0036] The MXene material is Ti3C2T. x .

[0037] The memristor unit 61 includes a second upper electrode 611, a second lower electrode 612, and a hafnium dioxide resistive switching functional layer 613. The second upper electrode 611 and the second lower electrode 612 are formed on the inner wall of the flexible substrate 3 covering the memristor unit 61 by laser direct writing. The second upper electrode 611 and the second lower electrode 612 are electrically connected to the tactile sensing unit 41 and the proximity sensing unit 51.

[0038] The fusion sensor is further coated with a protective encapsulation layer, which is formed by spin coating of PDMS or polyurethane.

[0039] See Figure 5 and Figure 6 Specifically, the electrical connection refers to connecting the output terminals of the tactile sensing unit 41 and the proximity sensing unit 51 to the input terminals of the corresponding memristor unit 61 using silver paste or anisotropic conductive adhesive via wire bonding or micro-soldering processes. See also... Figure 2 and Figure 3 The interdigital electrode 511, the first lower electrode 414 and the second lower electrode 612 are electrically connected, and the interdigital electrode 511, the first upper electrode 411 and the second upper electrode 611 are also electrically connected through electrical wires. In the proximity sensing array layer 5 and the tactile sensing array layer 4, the units located in the middle are electrically connected through a PET film.

[0040] Please see Figure 7 The present invention also discloses a signal processing method for a neuromorphic tactile and proximity fusion receptor, comprising the following steps: S1: When the proximity sensing unit 51 detects a nearby object, it generates a simulated photocurrent signal. The simulated photocurrent signal is input to the memristor unit 61 connected to it to pre-modulate the conductance of the memristor unit 61. At the same time, the simulated photocurrent signal is converted into a first characteristic pulse sequence to realize the spatiotemporal encoding of the proximity event. S2: When the tactile sensing unit 41 is subjected to pressure, the change in capacitance is converted into a voltage signal by the readout circuit. The voltage signal is input to the connected memristor unit 61 and converted into a second characteristic pulse sequence. S3: Memristor unit 61 acts as a simulated synapse. By changing its conductance state, it performs spatiotemporal integration and correlation learning on the input first characteristic pulse sequence and second characteristic pulse sequence, and outputs the fused pulse signal. S4: The fused pulse signal is directly used to drive the actuator, or transmitted to a higher-level neuromorphic processor for object recognition and decision-making.

[0041] The specific implementation method is as follows: Substrate preparation and electrode fabrication.

[0042] A clean PET film (125µm thick) is used as a common flexible substrate 3. A pre-set circuit pattern is scanned on the PET surface using a laser direct writing system to form the lower electrodes of all tactile sensors, the interdigital electrodes 511 of the proximity sensors, the lower electrode of the memristor, and the interconnecting wires between them in one step. The electrode material is a network of silver nanowires.

[0043] Fabricate flexible neuromorphic processing array layer 2 (memristor array).

[0044] In the lower electrode region of the formed memristor, a 50nm Ag layer, a 20nm HfO2 layer, and a 50nm Ag layer are sequentially deposited by magnetron sputtering to form an Ag / HfO2 / Ag memristor stack. The stack is then patterned into an array of independent memristor cells 61 using photolithography or laser ablation techniques.

[0045] Fabrication of a flexible multimodal sensing array layer 1.

[0046] Fabrication of tactile sensing unit 41: a) Microstructure template fabrication: Mix PDMS prepolymer with curing agent, pour onto sandpaper of different mesh sizes, and peel off after curing to obtain a PDMS template with negative microstructure.

[0047] b) Fabrication of MXene / PDMS sensitive layer 412: Multilayer Ti3C2T x MXene (titanium carbide) was ultrasonically exfoliated into a few-layer dispersion in an aqueous solution. This dispersion was then uniformly sprayed onto the microstructure surface of the aforementioned PDMS template. Simultaneously, a liquid PDMS prepolymer was spin-coated onto another flat PDMS substrate. Before the PDMS was fully cured, the MXene-coated template microstructure was imprinted onto the spin-coated PDMS layer and then cured by heating. After the template was peeled off, a composite sensitive layer 412 film with a sandpaper-like microstructure and an embedded MXene conductive network was obtained.

[0048] c) Device Integration: A PVA aqueous solution is spin-coated onto the tactile lower electrode array prepared in step 1 to form a dielectric layer 413. Subsequently, the composite sensitive layer 412 film obtained in step b) is precisely aligned and bonded onto the PVA dielectric layer 413. Finally, a tactile upper electrode array is formed on top of the sensitive layer 412 by laser direct writing or mask evaporation, completing the encapsulation of the capacitive tactile sensing unit 41.

[0049] Fabrication of proximity sensing unit 51: In the fabricated interdigitated electrode 511 region, an aqueous dispersion of graphene oxide is drop-coated. After drying, a laser direct writing system is used to locally scan and reduce the graphene oxide film, forming highly conductive graphene sensing channels in the gaps between the interdigitated electrodes 511. The laser power needs to be precisely controlled to simultaneously complete the reduction and patterning.

[0050] Step 4: Interconnection and encapsulation.

[0051] Using silver paste or anisotropic conductive adhesive, the output terminals of the tactile sensing unit 41 and the proximity sensing unit 51 are electrically connected to the input terminals of the corresponding memristor units 61 through wire bonding or micro-soldering processes, forming a local neural circuit. Finally, a thin layer of PDMS or polyurethane is spin-coated onto the entire system surface as a protective encapsulation layer.

[0052] Example 2: System Operation and Integration Testing The prepared system is attached to the fingertip of the robot's finger.

[0053] Proximity sensing function test: A ping-pong ball is slowly moved towards the fingertip. When the ball enters a 2cm range in front of the sensor, the graphene proximity sensing unit changes its resistance due to the change in the intensity of the reflected light received, generating a weak current. This current signal is input to a connected memristor, causing a small, reversible change in its conductance (short-term plasticity), and the system records the event of an object approaching.

[0054] Haptic and Fusion Function Test: The ping-pong ball continues to move until it contacts the finger. The tactile sensing unit 41 experiences a capacitance change due to pressure, generating a strong voltage signal. This signal is input to the same memristor. Because the memristor's conductance has been pre-altered, the tactile signal is amplified and reaches the pulse firing threshold more quickly, thereby accelerating the recognition response to the contact event.

[0055] Learning function demonstration: Repeat the process of approaching at a specific speed and then lightly touching multiple times. The conductance change of the memristor unit 61 is enhanced by the fixed timing relationship between the proximity and tactile pulses (pulse timing depends on plasticity). After learning, the system can predict the impending light touch based solely on the specific proximity speed pattern and adjust the gain of the grasping command in advance.

[0056] The above-disclosed embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of the invention. Those skilled in the art will understand that implementing all or part of the above-described embodiments and making equivalent changes in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A neuromimetic tactile and proximity sensory fusion receptor, characterized in that, It includes a multimodal sensing array layer (1) and a neuromorphic processing array layer (2) formed independently on a flexible substrate (3). The multimodal sensing array layer (1) includes a tactile sensing array layer (4) formed by an array of several tactile sensing units (41) and a proximity sensing array layer (5) formed by an array of several proximity sensing units (51). The neuromorphic processing array layer (2) includes several memristor units (61) arranged in an array. Each memristor unit (61) is electrically connected to at least one tactile sensing unit (41) and at least one proximity sensing unit (51). The proximity sensing unit (51) includes interdigital electrodes (511) and a graphene thin film photosensitive layer (512). The interdigital electrodes (511) are formed on the inner wall of the flexible substrate (3) covering the proximity sensing unit (51) by laser direct writing. The graphene thin film photosensitive layer (512) is coated on the gaps and surfaces of the interdigital electrodes (511). The graphene film photosensitive layer (512) is a conductive network obtained by laser reduction or thermal reduction of graphene oxide.

2. The neuromimetic tactile and proximity sensory fusion receptor as described in claim 1, characterized in that, The tactile sensing unit (41) includes a first upper electrode (411), a sensitive layer (412), a dielectric layer (413), and a first lower electrode (414). The sensitive layer (412) is disposed on both sides of the dielectric layer (413). The first upper electrode (411) is distributed between the sensitive layer (412) and the dielectric layer (413) on one side, and the first lower electrode (414) is distributed between the sensitive layer (412) and the dielectric layer (413) on the other side.

3. The neuromimetic tactile and proximity sensory fusion receptor as described in claim 2, characterized in that, The sensitive layer (412) includes a PDMS template with a sandpaper-like negative microstructure on its surface and an MXene material sprayed onto the microstructure surface of the PDMS template. The MXene material sprayed onto the microstructure surface of the PDMS template also forms the first upper electrode (411) and the first lower electrode (414).

4. The neuromimetic tactile and proximity sensory fusion receptor as described in claim 3, characterized in that, The MXene material is Ti3C2T. x .

5. The neuromimetic tactile and proximity sensory fusion receptor as described in claim 1, characterized in that, The memristor unit (61) includes a second upper electrode (611), a second lower electrode (612), and a hafnium dioxide resistive switching functional layer (613). The second upper electrode (611) and the second lower electrode (612) are formed on the inner wall of the flexible substrate (3) covering the memristor unit (61) by laser direct writing. The second upper electrode (611) and the second lower electrode (612) are electrically connected to the tactile sensing unit (41) and the proximity sensing unit (51).

6. The neuromimetic tactile and proximity fusion receptor as described in claim 1, characterized in that, The fusion sensor is further coated with a protective encapsulation layer, which is formed by spin coating of PDMS or polyurethane.

7. The neuromimetic tactile and proximity sensory fusion receptor as described in claim 1, characterized in that, Specifically, the electrical connection refers to connecting the output terminals of the tactile sensing unit (41) and the proximity sensing unit (51) to the corresponding input terminals of the memristor unit (61) using silver paste or anisotropic conductive adhesive through wire bonding or micro-welding processes.

8. A signal processing method for a neuromimetic tactile and proximity fusion receptor as described in any one of claims 1 to 7, characterized in that, Includes the following steps: S1: When the proximity sensing unit (51) detects a nearby object, it generates a simulated photocurrent signal. The simulated photocurrent signal is input to the memristor unit (61) connected to it, and the conductance of the memristor unit (61) is pre-modulated. At the same time, the simulated photocurrent signal is converted into the first characteristic pulse sequence to realize the spatiotemporal encoding of the proximity event. S2: When the tactile sensing unit (41) is subjected to pressure, the capacitance change is converted into a voltage signal by the readout circuit. The voltage signal is input to the connected memristor unit (61) and converted into a second characteristic pulse sequence. S3: The memristor unit (61) acts as a simulated synapse. By changing its conductance state, it performs spatiotemporal integration and correlation learning on the first and second characteristic pulse sequences, and outputs the fused pulse signal. S4: The fused pulse signal is directly used to drive the actuator, or transmitted to a higher-level neuromorphic processor for object recognition and decision-making.

Images