Gesture recognition intelligent glove based on PAM hydrogel flexible strain sensor
The smart glove for gesture recognition based on PAM hydrogel flexible strain sensor solves the problems of poor wearing comfort and large real-time response delay of traditional sensors, and achieves high sensitivity and low latency gesture recognition, which is suitable for long-term use and expands the application in fields such as medical rehabilitation and smart home.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- ZHENGZHOU UNIV
- Filing Date
- 2025-07-17
- Publication Date
- 2026-05-19
AI Technical Summary
Existing rigid sensors are uncomfortable to wear and cannot adapt to the dynamic deformation of the skin with high curvature. Visual gesture recognition is easily affected by ambient light and cannot capture subtle muscle activity signals. Traditional solutions have large real-time response delays in embedded devices, and insufficient synchronization of multi-channel signals leads to semantic misjudgment. Flexible sensors degrade in performance under the corrosive effects of sweat. They are difficult to meet the high sensitivity, low latency and strong environmental adaptability requirements of medical, industrial and consumer electronics.
Design a gesture recognition smart glove based on PAM hydrogel flexible strain sensors. The glove uses a PAM-MXene hydrogel flexible strain sensor array and combines a lightweight convolutional neural network for signal processing at the edge to achieve efficient real-time gesture recognition. The sensor is connected to the signal conversion module through flexible wires, and the power supply module uses a rechargeable lithium battery. The whole device is lightweight and can adapt to the high-frequency bending of the five fingers.
It achieves highly sensitive gesture recognition, real-time processing latency of less than 100ms, good wearability, and is suitable for long-term use, thus broadening its applications in fields such as medical rehabilitation, smart homes, and virtual reality.
Smart Images
Figure CN224263598U_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, specifically to a gesture recognition smart glove based on a PAM hydrogel flexible strain sensor. Background Technology
[0002] The combination of flexible strain sensors and gesture recognition technology has broad application prospects in fields such as medical rehabilitation, human-computer interaction, and intelligent prosthetic control. For example, in the medical field, hand motion monitoring can assist in the rehabilitation training of stroke patients and assess joint range of motion in real time; in the consumer electronics field, gesture interaction can enhance the immersive experience of AR / VR devices and replace traditional controller operation. However, existing technologies face significant bottlenecks in practical applications: traditional rigid sensors are uncomfortable to wear and cannot adapt to the high curvature dynamic deformation of hand skin, resulting in discontinuous data acquisition; while vision-based gesture recognition solutions (binocular cameras) are easily affected by ambient light and cannot capture subtle muscle activity signals. In addition, the needs of remote control of robotic arms in industrial scenarios and robot collaboration in hazardous environments urgently require a flexible sensing system that combines high sensitivity, low latency, and strong environmental adaptability.
[0003] As smart wearable devices evolve towards lighter weight and lower power consumption, users' expectations for natural interaction methods continue to rise. For example, sign language translation systems need to accurately recognize the continuous movements of complex gestures, but existing sensors are prone to semantic misinterpretation due to insufficient synchronization of multi-channel signals. In smart home scenarios, users expect to control multiple devices through simple gestures, but traditional solutions are limited by algorithmic computing power, making it difficult to achieve real-time response on embedded devices (latency is typically >200ms). Furthermore, the demand for health monitoring in an aging society is driving the development of wearable devices towards long-term, non-intrusive operation, but the performance degradation of most flexible sensors due to sweat corrosion and repeated bending limits their actual lifespan. Therefore, it is necessary to design a simple, easy-to-use wearable smart glove that can provide a robust, low-cost solution for medical, industrial, and consumer electronics scenarios. Utility Model Content
[0004] To address the problems existing in the prior art, this invention proposes a gesture recognition smart glove based on a PAM hydrogel flexible strain sensor to solve the problems mentioned in the background art.
[0005] To achieve the above objectives, this utility model provides the following technical solution:
[0006] A gesture recognition smart glove based on a PAM hydrogel flexible strain sensor, including a wearable flexible sensor and a gesture recognition system deployed at the edge;
[0007] The wearable flexible sensor includes an elastic substrate that fits the five fingers and a PAM-MXene hydrogel flexible strain sensor array. The gesture recognition system includes a signal conversion module, a power supply module, a signal processing module, and a microcontroller terminal.
[0008] The PAM-MXene hydrogel flexible strain sensor array consists of 5 independent sensors, which are integrated into the joints of the thumb, index finger, middle finger, ring finger and little finger respectively. The sensors are connected to the signal conversion module through flexible wires.
[0009] The signal conversion module converts the resistance signal output by the sensor into a voltage signal, which is then input into the signal processing module.
[0010] The power supply module provides power to the signal conversion module, the signal processing module, and the microcontroller terminal.
[0011] The signal processing module preprocesses and performs frequency domain conversion on the voltage signal, and the microcontroller terminal deploys a lightweight convolutional neural network to classify the processed signal and output the gesture recognition result.
[0012] As a further technical solution of this utility model: the PAM-MXene hydrogel flexible strain sensor is a thin sheet structure with a thickness of 0.5mm-2mm.
[0013] As a further technical solution of this utility model: the PAM-MXene hydrogel flexible strain sensor is prepared by ammonium persulfate-initiated polymerization, using acrylamide as monomer, N,N'-methylenebisacrylamide as crosslinking agent, tetramethylethylenediamine as catalyst, and MXene solution as conductive filler, and a three-dimensional network structure sensing layer is formed by solution casting.
[0014] As a further technical solution of this utility model: the two ends of the sensing layer formed by casting are wrapped with carbon nanotube sponge, and then wrapped with copper foil to form an electrode.
[0015] As a further technical solution of this utility model: the wearable flexible sensor is connected to the microcontroller terminal via a flexible wire.
[0016] As a further technical solution of this utility model: the power supply module adopts a rechargeable lithium battery with a capacity of ≥150mAh, supports continuous operation for ≥1 hour, and the overall device weight is ≤50g.
[0017] The technical effects and advantages provided by this utility model in the above technical solution are as follows:
[0018] 1. This flexible gesture recognition smart glove based on PAM-MXene composite hydrogel has the characteristics of high strain range sensing. It uses PAM hydrogel as the substrate of flexible strain sensor to solve the problems of poor biocompatibility and lag in deformation response of traditional sensors. Its linear sensing range covers the entire joint movement of the fingers (0°-90° bending) and can recognize micro-movements as small as 5°, providing high-quality data input for accurate gesture recognition.
[0019] 2. This utility model features efficient real-time processing. It deploys a lightweight CNN algorithm based on the nRF52840 chip of the Nano33BLESenseRev2 development board. It does not rely on cloud servers. A single device can complete the entire process of signal acquisition, processing and recognition with a latency of ≤100ms, which meets the real-time interaction needs of wearable devices in offline and mobile scenarios.
[0020] 3. This invention offers excellent wearability and adaptability. The sensor is only 0.5-2mm thick, lightweight, and conforms to the skin, adapting to high-frequency bending movements of the five fingers. It provides comfortable wear even after extended periods, expanding the application scenarios of wearable gesture recognition devices in fields such as medical rehabilitation, smart homes, and virtual reality. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of this utility model or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this utility model. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a schematic diagram of the structure of this utility model;
[0023] Figure 2 This is a schematic diagram of the structure of the PAM hydrogel sensor described in this utility model;
[0024] Figure 3 This is a circuit diagram of the signal conversion module described in this utility model;
[0025] Figure 4 This invention relates to a diagram showing some gestures used for gesture recognition and their corresponding acquired voltage signals.
[0026] Figure 5 This is a confusion matrix diagram of the CNN model for gesture letter recognition described in this utility model.
[0027] In the figure: 11-carbon nanotube seam, 12-coating copper foil, 13-flexible sensing layer. Detailed Implementation
[0028] The technical solutions of the present utility model will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present utility model, and not all embodiments. Based on the embodiments of the present utility model, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present utility model.
[0029] It should be noted that the terms "vertical," "horizontal," "up," "down," "left," "right," and similar expressions used in this article are for illustrative purposes only and do not represent the only possible implementation.
[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0031] Example 1:
[0032] This utility model provides, for example Figures 1-5 The drawings shown specifically depict a gesture recognition smart glove based on a PAM hydrogel flexible strain sensor, including a flexible wearable sensor and a gesture recognition system deployed at the edge:
[0033] The wearable flexible sensing device includes: an elastic substrate that conforms to the five fingers, and a PAM hydrogel flexible strain sensor array. The PAM hydrogel flexible strain sensor array consists of five independent sensors, which are integrated into the joints of the thumb, index finger, middle finger, ring finger, and little finger on the elastic substrate, respectively. Utilizing the excellent biocompatibility, high deformation sensitivity, and cyclic stability of PAM hydrogel, it can sense the strain signals generated by the movement of the finger joints in real time.
[0034] The gesture recognition system deployed at the edge includes a signal conversion module, a power supply module, a signal processing module, and a microcontroller terminal. The power supply module supplies power to the edge computing microcontroller terminal and the signal acquisition module. The signal conversion module converts the resistance signal from the sensor into a voltage signal, which is then input to the signal processing module. The signal processing module converts the time-domain signal into a frequency-domain signal using the onboard DSP unit of the edge computing microcontroller terminal. The microcontroller terminal deploys a convolutional neural network (CNN) to complete gesture recognition. The microcontroller terminal is an Arduino Nano 33 BLE Sense Rev2, which integrates a wireless communication module and a storage module. The wireless communication module can connect the glove to other wireless terminals, and the storage module pre-stores trained CNN model parameters. The microcontroller performs a Fast Fourier Transform (FFT) on the real-time acquired one-dimensional time-series strain signal and extracts features. The gesture recognition result is then output after classification by a fully connected layer using the edge-deployed CNN algorithm.
[0035] Furthermore, the PAM hydrogel flexible strain sensor is prepared by ammonium persulfate-initiated polymerization, using acrylamide as monomer, N,N'-methylenebisacrylamide as crosslinking agent, and tetramethylethylenediamine as catalyst, and is formed by casting the solution into a mold to form a sensing layer with a three-dimensional network structure.
[0036] Furthermore, the lightweight CNN neural network algorithm includes one convolutional layer, two pooling layers, and one fully connected layer; the algorithm improves the edge recognition generalization ability by using the personalized gesture data of the target user (each gesture is collected ≥30 times).
[0037] The wearable flexible sensor and signal conversion module are connected to the edge computing microcontroller terminal via flexible wires. The power supply module uses a rechargeable lithium battery (capacity ≥150mAh), which supports continuous operation for ≥1 hour. The overall device weighs ≤50g, is adapted to the natural bending movement of the five fingers, and has excellent wearing comfort.
[0038] The wearable flexible sensor comprises a PAM hydrogel flexible strain sensor array. The sensor is prepared using an ammonium persulfate-initiated polymerization process to create a 1mm thick PAM hydrogel. Both ends are wrapped with carbon nanotube sponges, and then coated with copper foil to form electrodes. DuPont wires are then soldered onto the copper foil. Five independent sensors are integrated into the proximal phalanges of the thumb, index finger, middle finger, ring finger, and little finger, respectively, and connected to a signal conversion module via flexible wires. The signal conversion module converts the sensor output resistance signals into voltage signals suitable for Arduino acquisition. A 3.7V rechargeable lithium battery (150mAh capacity) powers the sensors and conditioning module via a flexible circuit board, supporting continuous operation for 1 hour.
[0039] The edge computing microcontroller terminal is based on the Arduino Nano 33 BLE Sense Rev2 development board, integrating a Bluetooth wireless communication module and a storage module. It achieves stable data transmission within 20 meters and stores trained CNN model parameters and user gesture labels. The lightweight CNN algorithm is deployed in the Arduino's built-in memory, with 150 features in the input layer and a reshaping layer adjusted to 10 columns. Next, a 1D convolutional pooling layer with 16 neurons and a 9-cell kernel performs one layer of 1D convolution. A dropout layer with a dropout rate of 0.2 randomly discards 20% of neurons to prevent overfitting. Afterward, a flattening layer flattens the multidimensional data into a one-dimensional vector, inputting it into a fully connected layer with 30 neurons for feature integration. The final output layer outputs 13 categories. The total model parameters are 55KB, and the single inference time is 15ms, meeting real-time processing requirements.
[0040] Signal acquisition and processing flow
[0041] When a finger joint bends, the PAM hydrogel sensor deforms, causing a change in resistance. This change is converted into a voltage signal by the signal conversion module and then sampled through the analog pins of the Arduino. The terminal's built-in data processing program performs exponential smoothing and cross-difference enhancement on the signal. Subsequently, the onboard DSP unit performs a Fast Fourier Transform (FFT) to extract statistical and spectral features, which are then input into the CNN model Softmax classifier to output the gesture recognition result.
[0042] Example 2:
[0043] The wearable device in this embodiment is a five-finger glove, and its structure is as follows: Figure 1 As shown, the specific parameters are as follows:
[0044] Sensor fabrication: 3g of acrylamide (30wt%) was dissolved in 9ml of deionized water, and 1ml of 20mg / ml MXene solution was added. Then, 500 μL of 10mg / ml N,N'-methylenebisacrylamide (0.2wt%) and 500 μL of 30mg / ml ammonium persulfate (0.5wt%) were added and poured into a 3D-printed PLA mold with dimensions of 30mm × 16mm × 2mm. 10 μL of tetramethylethylenediamine was added as a catalyst to obtain a sensing layer with a thickness of 1mm. The sensor structure is then as follows. Figure 2 As shown, it includes a flexible sensing layer 13, with both ends wrapped by carbon nanotube sponge 11, and then wrapped with a copper foil 12 to lead out electrodes. DuPont wires are then soldered onto the copper foil to connect to the signal acquisition module.
[0045] Sensor layout: Five sensors correspond to the thumb (MCP joint) and the index to little fingers (PIP joints), respectively, and are arranged in an array.
[0046] Signal acquisition: such as Figure 3 As shown, the signal conversion module uses a voltage divider method to convert the resistance change signal caused by sensor deformation into a voltage signal, and then acquires the original signals F1-F5 of the five flexible strain sensors from the five analog signal input terminals of the Arduino.
[0047] Gesture Dataset: Strain signals were collected for 12 Chinese finger letter patterns from the "Chinese Finger Alphabet Scheme" (1 pattern for single-finger bending, 2 patterns for two-finger bending, and 9 patterns for three or more fingers). Each gesture was recorded 50 times, resulting in a total of 600 samples. These samples were divided into training and testing sets in an 8:2 ratio. Figure 4 The image shows the initial state, the gesture of the letter "ad", and the corresponding collected voltage data.
[0048] The acquired 5D voltage data was exponentially smoothed to eliminate high-frequency fluctuations, and cross-difference was used to enhance input features, forming a 15D original time-domain signal. A sliding detection window of 2 seconds and 200ms was added to each time-domain signal. For each window, three statistical features (RMS, skewness, and kurtosis) were generated, along with a frequency-domain feature including amplitude values at seven frequency points after time-frequency transformation. A total of 150 features were generated from the 15D original time-domain signal. The features generated from each window were then input into a CNN model, as follows: Figure 5 As shown, the average accuracy of gesture recognition reaches 95.2%.
[0049] Compared to traditional rigid sensors, the PAM hydrogel sensor in this embodiment can detect micro-bending movements of up to 135°; the edge computing terminal has lower latency than cloud solutions, and its wearability has passed ergonomic testing, with no skin redness or pressure after continuous wear for 4 hours.
[0050] It will be apparent to those skilled in the art that this invention is not limited to the details of the exemplary embodiments described above, and that it can be implemented in other specific forms without departing from the spirit or essential characteristics of this invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within this invention.
[0051] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment have been appropriately combined to form other embodiments that are easy for those skilled in the art to understand.
Claims
1. A gesture recognition smart glove based on a PAM hydrogel flexible strain sensor, characterized in that, This includes wearable flexible sensors and gesture recognition systems deployed at the edge; The wearable flexible sensor includes an elastic substrate that fits the five fingers and a PAM-MXene hydrogel flexible strain sensor array. The gesture recognition system includes a signal conversion module, a power supply module, a signal processing module, and a microcontroller terminal. The PAM-MXene hydrogel flexible strain sensor array consists of 5 independent sensors, which are integrated into the joints of the thumb, index finger, middle finger, ring finger and little finger respectively. The sensors are connected to the signal conversion module through flexible wires. The signal conversion module converts the resistance signal output by the sensor into a voltage signal, which is then input into the signal processing module. The power supply module provides power to the signal conversion module, the signal processing module, and the microcontroller terminal. The signal processing module preprocesses and performs frequency domain conversion on the voltage signal, and the microcontroller terminal deploys a lightweight convolutional neural network to classify the processed signal and output the gesture recognition result.
2. The gesture recognition smart glove based on a PAM hydrogel flexible strain sensor according to claim 1, characterized in that, The PAM-MXene hydrogel flexible strain sensor is a thin sheet structure with a thickness of 0.5mm-2mm.
3. The gesture recognition smart glove based on a PAM hydrogel flexible strain sensor according to claim 1, characterized in that, The PAM-MXene hydrogel flexible strain sensor was prepared by ammonium persulfate-initiated polymerization, using acrylamide as the monomer and N,N ' Methylenebisacrylamide is used as a crosslinking agent, tetramethylethylenediamine as a catalyst, and MXene solution as a conductive filler. A three-dimensional network structure sensing layer is formed by solution casting.
4. The gesture recognition smart glove based on a PAM hydrogel flexible strain sensor according to claim 1, characterized in that, The sensing layer formed by casting is wrapped at both ends with carbon nanotube sponge, and then wrapped with copper foil to form electrodes.
5. The gesture recognition smart glove based on a PAM hydrogel flexible strain sensor according to claim 1, characterized in that, The wearable flexible sensor is connected to the microcontroller terminal via a flexible wire.
6. The gesture recognition smart glove based on a PAM hydrogel flexible strain sensor according to claim 1, characterized in that, The power supply module uses a rechargeable lithium battery with a capacity of ≥150mAh, supports continuous operation for ≥1 hour, and the overall device weight is ≤50g.