A wearable neuromorphic haptic perception system
By integrating a flexible piezoelectric sensing module, a bionic pulse coding module, and a dual-gate synaptic transistor into wearable devices, the hardware redundancy and energy consumption problems of existing systems are solved, enabling real-time, low-power multimodal tactile perception and supporting the recognition and interaction of multiple tactile modes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV
- Filing Date
- 2026-04-16
- Publication Date
- 2026-06-23
AI Technical Summary
Existing electronic skin or tactile sensing systems suffer from hardware redundancy, high energy consumption, and data transmission delays due to their separate sensing-transmission-processing architecture. This makes it difficult to meet the requirements of wearable devices for miniaturization, low power consumption, and real-time response. Furthermore, artificial tactile systems fail to realize the core processing principles of biological systems at the hardware level.
By employing a flexible piezoelectric sensing module, a biomimetic pulse coding module, and a dual-gate synaptic transistor, the biomimetic coding and feature extraction of tactile signals are integrated into a micro wearable platform. The biomimetic pulse coding module converts analog voltage signals into pulse sequences, and the dual-gate synaptic transistor performs spatiotemporal integration. Finally, the recognition module performs real-time recognition.
It achieves high system integration and miniaturization, reduces computational complexity and energy consumption, possesses bio-inspired versatility and robustness, can efficiently process multiple tactile modalities, achieves extremely low power consumption and computing power requirements, and supports real-time tactile pattern recognition.
Smart Images
Figure CN122263997A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a tactile sensing system, and more particularly to a neuromorphic tactile sensing system. Background Technology
[0002] Tactile perception is crucial for robots, wearable devices, and smart prosthetics to physically interact with their environment. Existing electronic skin or tactile sensing systems typically rely on dense arrays of sensors to acquire spatial information and transmit massive amounts of raw data to external processors (such as CPUs and GPUs) for feature extraction and pattern recognition. This separate "sensing-transmission-processing" architecture leads to significant hardware redundancy, high power consumption, and data transmission latency, making it difficult to meet the stringent requirements of wearable devices for miniaturization, low power consumption, and real-time response.
[0003] Currently, most artificial tactile systems merely mimic biological forms, with their core signal processing and recognition functions still heavily reliant on complex software algorithms, failing to implement the core processing principles of biological systems at the hardware level. Therefore, developing a neuromorphic tactile system that integrates bio-inspired sensing, encoding, and computational functions onto a micro-wearable platform is of great significance. Summary of the Invention
[0004] Purpose of the invention: In view of the above-mentioned prior art, a wearable neuromorphic tactile perception system is proposed, which realizes biomimetic encoding and feature extraction of tactile signals at the hardware level, and achieves real-time multimodal tactile perception.
[0005] Technical solution: A wearable neuromorphic tactile sensing system, comprising:
[0006] A flexible piezoelectric sensing module is used to convert external tactile stimuli into analog voltage signals;
[0007] The biomimetic pulse coding module is used to convert the analog voltage signal into two independent pulse sequences, namely a fast-adaptive pulse sequence and a slow-adaptive pulse sequence, according to a preset differential threshold and an absolute threshold.
[0008] A dual-gate synaptic transistor, wherein the first gate and the second gate respectively receive the fast-adaptive pulse sequence and the slow-adaptive pulse sequence, and are used to perform spatiotemporal integration of the two received pulse sequences to output a postsynaptic current signal characterizing tactile features;
[0009] The identification module is used to extract features from the two pulse sequences output by the biomimetic pulse coding module: the fast-adaptive pulse average firing rate f. FA Slow-adaptive pulse average firing rate f SAFeatures are extracted from the postsynaptic current signal output by the dual-gate synaptic transistor: the postsynaptic current change ΔPSC; based on the three extracted features, a classification algorithm is used to identify tactile patterns in real time.
[0010] Furthermore, the biomimetic pulse coding module is configured to: generate the fast-adaptive pulse sequence when the rate of change of the analog voltage signal exceeds the differential threshold; and generate the slow-adaptive pulse sequence when the amplitude of the analog voltage signal exceeds the absolute threshold and continues for a preset duration.
[0011] Furthermore, the bionic pulse coding module includes:
[0012] An analog-to-digital converter unit is used to sample the analog voltage signal into a discrete digital sequence;
[0013] A digital processor is used to perform fast adaptation channel and slow adaptation channel detection in parallel on the digital sequence: the fast adaptation channel obtains the signal change rate by calculating the difference between adjacent sampling points, and when the change rate exceeds the differential threshold, the fast adaptation pulse output unit is triggered to output a high-level pulse; the slow adaptation channel compares the amplitude of the sampled signal with the absolute threshold, and when the amplitude continues to exceed the threshold for a preset de-jitter duration, the slow adaptation pulse output unit is triggered to output a high-level pulse.
[0014] Furthermore, the dual-gate synaptic transistor includes: a semiconductor channel layer, a source, a drain, a first gate, a second gate, and an ion gel electrolyte layer; the source, drain, first gate, and second gate are located in the same plane; and the coupling distances between the first gate, the second gate, and the channel in the semiconductor channel layer are different; the ion gel electrolyte layer covers the surface of the transistor, forming an electrical double layer through ion migration at the interface between the electrolyte and the channel; wherein, the first gate and the second gate respectively receive the fast-adaptive pulse sequence and the slow-adaptive pulse sequence from the biomimetic pulse coding module; the source serves as a current output terminal for outputting the postsynaptic current signal.
[0015] Furthermore, the semiconductor channel layer is made of indium gallium zinc oxide, and the ion gel electrolyte layer is made of chitosan matrix ion conductor.
[0016] Furthermore, the tactile modes include vibration patterns, object geometry, surface texture, or handwriting trajectories.
[0017] Furthermore, the flexible piezoelectric sensing module is integrated onto a flexible substrate that can be worn on a nail.
[0018] Furthermore, the recognition module is also used to generate corresponding interactive instructions based on the tactile pattern.
[0019] Beneficial Effects: This invention integrates flexible piezoelectric sensing, biomimetic pulse coding, and neuromorphic synaptic computation into a single micro wearable platform, achieving high system integration and miniaturization, eliminating reliance on sensor arrays and external processors. The system employs event-driven pulse coding and hardware-level feature extraction, reducing computational complexity and energy consumption by several orders of magnitude, achieving extremely low power consumption and computing power requirements. Simultaneously, mimicking the dual-path processing mechanism of biological tactile receptors, the system possesses bio-inspired versatility and robustness, enabling efficient processing of various tactile modalities such as vibration, contour, and texture. Attached Figure Description
[0020] Figure 1 This is a structural block diagram of the wearable neuromorphic tactile sensing system of the present invention;
[0021] Figure 2 This is a schematic diagram of the dual-gate synaptic transistor in this invention;
[0022] Figure 3 This is a schematic diagram of the actual wearing of the wearable neuromorphic tactile sensing system of the present invention;
[0023] Figure 4 This is a schematic diagram illustrating the system's performance in a vibration pattern recognition task.
[0024] Figure 5 This is a performance diagram of the system in the contour recognition task;
[0025] Figure 6 This is a performance diagram of the system in a texture recognition task.
[0026] Figure 7 A schematic diagram of the neuromorphic human-computer interaction results based on handheld writing;
[0027] Figure label:
[0028] 301 - Semiconductor channel layer, 302 - Source, 303 - Drain, 304 - First gate, 305 - Second gate, 306 - Ion gel electrolyte layer. Detailed Implementation
[0029] The present invention will now be described in detail with reference to the accompanying drawings.
[0030] like Figure 1 As shown, a wearable neuromorphic tactile sensing system is described. This system mimics the tentacle pathway of rodents, constructing a hierarchical structure of "perception-encoding-preprocessing-recognition". The entire system is integrated onto a flexible substrate that can be worn on a fingernail.
[0031] The flexible piezoelectric sensing module, acting as a mechanoreceptor, is attached to a custom-made artificial fingernail to convert external tactile stimuli into analog voltage signals. This module uses a flexible piezoelectric material as the sensing element; in this embodiment, a polyvinylidene fluoride piezoelectric film is used as the response layer, flexible printed silver paste as the electrodes, and it is encapsulated and protected by a polyethylene terephthalate film to achieve good flexibility, wearability, and signal response characteristics.
[0032] A biomimetic pulse coding module is electrically connected to the output of the flexible piezoelectric sensing module. Implemented by a microcontroller (such as an ESP32) and peripheral circuitry, this module converts the analog voltage signal output from the flexible piezoelectric sensing module into sparse FA and SA pulse sequences in real time by setting a differential threshold for the fast-adaptive (FA) pulse sequence and an absolute threshold for the slow-adaptive (SA) pulse sequence. A fast-adaptive pulse sequence is generated when the rate of change of the analog voltage signal exceeds the differential threshold; a slow-adaptive pulse sequence is generated when the amplitude of the analog voltage signal exceeds the absolute threshold and remains so for a preset duration.
[0033] The working process of the bionic pulse coding module is as follows: The analog voltage signal is sampled into a discrete digital sequence by the analog-to-digital converter. The digital processor executes two parallel detection logics in real time: For the fast-adaptive (FA) channel, it obtains the signal change rate by calculating the difference between adjacent sampling points and compares this change rate with a preset positive differential threshold; once the change rate exceeds the threshold, the FA pulse output unit is triggered, causing it to generate a high-level pulse within a set period, after which it returns to a low level until the next trigger condition is met. For the slow-adaptive (SA) channel, the instantaneous amplitude of the sampled signal is compared with a preset absolute threshold, and a timing mechanism is introduced; when the signal amplitude continues to exceed the absolute threshold for a preset de-jittering duration, the SA pulse output unit is triggered, causing it to generate a high-level pulse, after which it also returns to a low level; if the signal amplitude falls below the threshold before reaching the de-jittering duration, the timing is reset, and no pulse is generated. The de-jittering duration is 0.5 seconds. At any time when the trigger conditions of each of the above are not met, both pulse output units remain in a low-level stable state.
[0034] One of the core components of this system is the dual-gate synaptic transistor, whose structure is as follows: Figure 2As shown, its structure includes: a semiconductor channel layer 301, a source 302, a drain 303, a first gate 304, a second gate 305, and an ion gel electrolyte layer 306. The source 302, drain 303, first gate 304, and second gate 305 are located in the same plane. The coupling distance between the first gate 304 and the channel in the semiconductor channel layer 301 is different from the coupling distance between the second gate 305 and the channel, thereby providing asymmetric synaptic weights for the two pulse sequences. The ion gel electrolyte layer 306 covers the source 302, drain 303, first gate 304, second gate 305, and semiconductor channel layer 301, forming an electrical double layer through ion migration to induce short-range and long-range synaptic plasticity. The first gate 304 and the second gate 305 of the dual-gate synaptic transistor receive fast-adaptive pulse sequences and slow-adaptive pulse sequences from the bionic pulse coding module, respectively; the drain 303 is configured to be connected to a constant bias voltage Vdd, and the source 302 serves as a current output terminal to output the postsynaptic current signal after spatiotemporal integration by the transistor.
[0035] In this embodiment, the dual-gate synaptic transistor is fabricated on a flexible polyimide substrate, using IGZO as the semiconductor channel and chitosan-based ionogel as the ionogel electrolyte layer. The key design element is that the coupling distances from the two coplanar gates to the channel are precisely designed to be asymmetrical, thus naturally endowing the FA and SA pathways with different synaptic weights. When a pulse is applied to the gate, protons form an electrical double layer at the electrolyte / channel interface under the influence of the field effect, modulating the channel conductivity state and generating rich synaptic plasticity behaviors, such as double-pulse facilitation and pulse frequency-dependent plasticity, thereby achieving the spatiotemporal integration function of the FA and SA pulse sequences.
[0036] Specifically, the spatiotemporal integration function of the device manifests as follows: when a pulse is applied to the gate, protons in the electrolyte migrate and form an electric double layer at the interface. This process and subsequent relaxation are time-dependent, giving the device pulse timing and frequency-dependent plasticity, thereby encoding and accumulating pulse intervals, sequences, and frequencies in the time dimension. This temporal dynamics, coupled with inherent spatial asymmetric weighting, means that pulse sequences from the FA and SA pathways are not simply superimposed, but rather undergo nonlinear, time-dependent weighted fusion based on their input gate weights, arrival order, and time intervals. Ultimately, this physical process directly maps the sparse pulse flow characterizing dynamic and static tactile features into a unique, steady-state analog current signal. This signal simultaneously compresses and encodes the modal proportions, dynamic timing, and intensity information of the tactile stimulus, thus completing the feature extraction and integration of the original tactile information at the hardware level.
[0037] The recognition module identifies tactile patterns based on the two pulse sequence signals output by the bionic pulse coding module and the postsynaptic current signal output by the dual-gate synaptic transistor. Specifically, the recognition module extracts features from the two pulse sequence signals output by the bionic pulse coding module: the average firing rate f of the FA. FA SA average distribution rate f SA The system extracts features from the output current of the dual-gate synaptic transistor: the postsynaptic current change ΔPSC. These three features are then identified in real time using a lightweight classification algorithm (such as a nearest neighbor classifier based on Euclidean distance) pre-stored in the recognition module. Tactile patterns include vibration patterns, object geometry, surface texture, or handwriting trajectories.
[0038] The recognition module is also used to generate corresponding interactive instructions based on the tactile pattern.
[0039] like Figure 3 The image shown is a schematic diagram of the wearable neuromorphic tactile sensing system of this embodiment.
[0040] like Figure 4 The image shows the system's ability to recognize five iPhone vibration modes (alarm, heartbeat, rapid, SOS, and symphony) with an accuracy rate of 96%. Figure 4 In the table, (a) is the fast-adaptive pulse sequence for the five default vibration ringtones (alert, heartbeat, rapid, SOS, symphony) of the iPhone; (b) is the transistor postsynaptic current response corresponding to the five vibration ringtones; (c) is a two-dimensional mapping of the average firing rate of the fast-adaptive pulses corresponding to the five vibration ringtones and the change in postsynaptic current; and (d) is the recognition rate confusion matrix corresponding to the five vibration ringtones.
[0041] like Figure 5 The image shows the system's ability to recognize geometric contours (circles, pentagons, squares, trapezoids, and triangles), with an accuracy rate exceeding 90%. Figure 5 In the image, (a) is the slow-adaptive pulse sequence obtained from scanning the five geometric objects shown in the figure; (b) is the postsynaptic current response of the transistors corresponding to the five geometric objects; (c) is a two-dimensional mapping of the average firing rate of the slow-adaptive pulses corresponding to the five geometric objects and the change in postsynaptic current; and (d) is the confusion matrix of the recognition rate corresponding to the five geometric objects.
[0042] like Figure 6The figure shows the system's ability to recognize the texture of natural plant leaves. In the figure, (a) is the fast and slow adaptation pulse sequence obtained by scanning five types of leaves; (b) is the transistor postsynaptic current response corresponding to the five types of leaves; (c) is a three-dimensional mapping of the average firing rate of the fast and slow adaptation pulses corresponding to the five types of leaves and the change in postsynaptic current; and (d) is the recognition rate confusion matrix corresponding to the five types of leaves.
[0043] like Figure 7 The diagram illustrates the system's human-computer interaction capabilities: (a) the fast and slow adaptive pulse sequences obtained by drawing five writing patterns (dot, dash, triangle, square, and circle) on the palm with a fingertip; (b) the transistor postsynaptic current response corresponding to the five writing patterns; (c) a two-dimensional graph of the average firing rate of the fast and slow adaptive pulses corresponding to the five writing patterns; (d) a three-dimensional mapping graph of the average firing rate of the fast and slow adaptive pulses corresponding to the five writing patterns and the change in postsynaptic current; and (e) the recognition rate confusion matrix corresponding to the five writing patterns. Users can control the on / off state, brightness, and color of smart lights in real time by writing simple patterns (dot, dash, triangle, square, and circle) on their palms, with an accuracy exceeding 90%, verifying its feasibility as a natural, low-power interactive interface.
[0044] This system enables natural and intuitive human-computer interaction based on handwriting, providing users with a private control channel that is screen-independent and requires no visual feedback.
[0045] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A wearable neuromorphic tactile sensing system, characterized in that, include: A flexible piezoelectric sensing module is used to convert external tactile stimuli into analog voltage signals; The biomimetic pulse coding module is used to convert the analog voltage signal into two independent pulse sequences, namely a fast-adaptive pulse sequence and a slow-adaptive pulse sequence, according to a preset differential threshold and an absolute threshold. A dual-gate synaptic transistor, wherein the first gate and the second gate respectively receive the fast-adaptive pulse sequence and the slow-adaptive pulse sequence, and are used to perform spatiotemporal integration of the two received pulse sequences to output a postsynaptic current signal characterizing tactile features; The identification module is used to extract features from the two pulse sequences output by the biomimetic pulse coding module: the fast-adaptive pulse average firing rate f. FA Slow-adaptive pulse average firing rate f SA Features are extracted from the postsynaptic current signal output by the dual-gate synaptic transistor: the postsynaptic current change ΔPSC; based on the three extracted features, a classification algorithm is used to identify tactile patterns in real time.
2. The system according to claim 1, characterized in that, The biomimetic pulse coding module is configured to generate the fast-adaptive pulse sequence when the rate of change of the analog voltage signal exceeds the differential threshold. When the amplitude of the analog voltage signal exceeds the absolute threshold and continues for a preset duration, the slow-adaptive pulse sequence is generated.
3. The system according to claim 2, characterized in that, The biomimetic pulse coding module includes: An analog-to-digital converter unit is used to sample the analog voltage signal into a discrete digital sequence; A digital processor is used to perform fast adaptation channel and slow adaptation channel detection in parallel on the digital sequence: the fast adaptation channel obtains the signal change rate by calculating the difference between adjacent sampling points, and when the change rate exceeds the differential threshold, the fast adaptation pulse output unit is triggered to output a high-level pulse; the slow adaptation channel compares the amplitude of the sampled signal with the absolute threshold, and when the amplitude continues to exceed the threshold for a preset de-jitter duration, the slow adaptation pulse output unit is triggered to output a high-level pulse.
4. The system according to claim 1, characterized in that, The dual-gate synaptic transistor includes: a semiconductor channel layer (301), a source (302), a drain (303), a first gate (304), a second gate (305), and an ion gel electrolyte layer (306); the source (302), drain (303), first gate (304), and second gate (305) are located in the same plane; and the coupling distances between the first gate (304), the second gate (305), and the channel in the semiconductor channel layer (301) are different; the ion gel electrolyte layer (306) covers the surface of the transistor and forms an electric double layer through ion migration at the interface between the electrolyte and the channel; wherein, the first gate (304) and the second gate (305) respectively receive the fast-adaptive pulse sequence and the slow-adaptive pulse sequence from the biomimetic pulse coding module; the source (302) serves as a current output terminal for outputting the postsynaptic current signal.
5. The system according to claim 4, characterized in that, The semiconductor channel layer (301) is made of indium gallium zinc oxide, and the ion gel electrolyte layer (306) is made of chitosan matrix ion conductor.
6. The system according to claim 1, characterized in that, The tactile modes include vibration patterns, object geometry, surface texture, or handwriting patterns.
7. The system according to any one of claims 1 to 6, characterized in that, The flexible piezoelectric sensing module is integrated onto a flexible substrate that can be worn on a nail.
8. The system according to any one of claims 1 to 6, characterized in that, The recognition module is also used to generate corresponding interactive instructions based on the tactile pattern.