Gesture recognition device based on instantaneous transmission muscle sound signal
By using a gesture recognition device that transmits myosalpingography signals instantaneously, and utilizing vibration sensors and vibration sources on a ring-shaped armband, gestures can be quickly collected and recognized. This solves the problems of high latency and limited application scenarios in existing technologies, and achieves efficient and accurate gesture recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-22
- Publication Date
- 2026-04-07
AI Technical Summary
Existing gesture recognition methods based on biosignals require close contact with the skin surface, resulting in high latency and limited application scenarios, making it difficult to achieve efficient recognition in complex environments.
A gesture recognition device based on instantaneous transmitted myosalpingography (IMA) signals is used. Through vibration sensors and vibration sources on the ring armband, sinusoidal signals of different frequencies are sent to penetrate the muscles to collect the transmitted myosalpingography signals. The recognition processing unit performs rapid recognition and combines the KNN model for gesture recognition.
It achieves low latency and high accuracy gesture recognition, is suitable for various wearable scenarios, has strong anti-interference capabilities, is suitable for complex environments, and has a recognition accuracy of up to 99%.
Smart Images
Figure CN121807151A_ABST
Abstract
Description
Technical Field
[0001] This application relates to a gesture recognition device based on instantaneous transmitted myosophytic signals, belonging to the field of human-machine interface technology. Background Technology
[0002] With the rapid development of human-computer interaction (HCI) technology, there is an urgent need to improve the real-time performance, comfort, and privacy of HCI tasks to bring people a better interactive experience. Gesture recognition is a typical example of many HCI tasks. It can be applied in various scenarios such as games, medical care, prosthetic control, and robot operation, and is of great significance to human life. Biosignal-based recognition methods include electromyography (EMG) recognition, ultrasound imaging, and myosomal sounding (MMG) recognition. Among the many acquisition devices based on arm biosignals, wearable devices that collect EMG and MMG signals are the most common. Researchers have achieved gesture recognition accuracy of over 90% using commercially available EMG and MMG acquisition devices such as the Myo armband and Shimmer sensors.
[0003] The existing technology has the following drawbacks:
[0004] The recognition method based on the above signals requires hundreds of milliseconds to several seconds of raw data filtering before it can start recognizing gestures, resulting in a high latency.
[0005] The aforementioned biosignals must be in close contact with the skin surface, which limits the application scenarios of the sensor. Summary of the Invention
[0006] To address the issue that existing gesture recognition methods require biosignals to be in close contact with the skin surface, this application provides a gesture recognition device based on transient transmissive myosalpingography (TTM) signals.
[0007] This application discloses a gesture recognition device based on instantaneous transmitted myosalpingography signals, comprising N vibration sensors, at least one vibration source, a sound collector, and a recognition processing unit; the N vibration sensors and at least one vibration source form a ring-shaped armband, which is worn on the arm during recognition.
[0008] The vibration source is used to send sinusoidal signals modulated with different vibration frequencies to the arm muscles. The sinusoidal signals modulated with different vibration frequencies penetrate the muscles and generate transmitted myosal signals.
[0009] Each vibration sensor is used to collect transmitted myosalpingography signals at all vibration frequencies and send them to the sound acquisition unit;
[0010] The sound acquisition unit is used to amplify and convert the transmitted muscle sound signals collected by N vibration sensors to obtain N sets of data, and then send them to the recognition and processing unit.
[0011] The recognition and processing unit is used to process the received N sets of data and determine the corresponding gesture.
[0012] Preferably, the vibration source is located on the side of the flexor digitorum profundus muscle, and the N vibration sensors are located on the side of the flexor digitorum superficialis muscle, all of which avoid the radius and ulna.
[0013] Preferably, the device further includes mounting positions for installing vibration sensors and vibration sources. The two mounting positions are connected by a hinge, and the first and last mounting positions are connected by an elastic strap to form a ring-shaped armband.
[0014] Preferably, the mounting position, vibration sensor, vibration source, and hinge connection structure are made of 3D printed housing.
[0015] Preferably, different vibration frequencies are selected uniformly within the range of 400~2000Hz.
[0016] Preferably, the recognition processing unit processes the received N sets of data and determines the corresponding gesture using the following method:
[0017] Feature extraction is performed on each set of data to obtain the characteristic signals of transmitted myosophytic signals at different vibration frequencies. Each vibration sensor corresponds to a set of characteristic signals.
[0018] N sets of feature signals are input into a trained recognition model, and the recognition model outputs the corresponding gesture. The recognition model is a neural network model.
[0019] Preferably, the characteristic signal of the transmitted myosalpingography signal is:
[0020] ;
[0021] in, It is a characteristic signal of transmitted muscle acoustic signals. This represents the k-th vibration frequency. This represents the number of different vibration frequencies, where N is the number of sampling points in each data set. It is the sampling frequency of the vibration sensor. Let i be the i-th data in each set of data.
[0022] The preferred identification model is the KNN model.
[0023] Preferably, the vibration source is a bone conduction speaker or an electromagnetic speaker.
[0024] Preferably, the vibration sensor is an accelerometer or a microphone.
[0025] The beneficial effects of this application are: (1) Significantly reduced latency: The latency of the prior art ranges from hundreds of milliseconds to several seconds, while this application only requires tens of milliseconds to complete signal acquisition, processing and identification, with a latency of 1 / 10 to 1 / 100 of the prior art;
[0026] (2) Application scenario expansion: Breaking through the limitation that the sensor must be in close contact with the skin, it supports signal acquisition through intermittent fabric, and is suitable for more wearable scenarios (such as underwear, sun protection sleeves, etc.).
[0027] (3) High recognition accuracy: Based on multi-frequency complementary recognition of tMMG signals and combined with KNN model training, the gesture recognition accuracy can reach 99%, which is on par with existing commercial equipment and has better stability;
[0028] (4) Easy to wear and use: 3D printed shell + elastic nylon buckle strap, lightweight and easy to carry, hinge structure supports installation position adjustment, suitable for people with different arm circumferences (adults with arm circumference within 40cm can use it).
[0029] (5) Strong anti-interference ability: Compared with EMG signal, tMMG signal is not affected by sweat or electromagnetic interference; the multi-frequency modulation design further reduces external vibration interference and is suitable for complex environments (such as outdoor and industrial scenarios).
[0030] This application achieves gesture recognition by detecting the skin surface pressure modulation signal caused by muscle contraction, and is applicable to application scenarios that require rapid gesture recognition, such as medical rehabilitation, muscle status assessment, virtual reality, and intelligent prosthetic control. Attached Figure Description
[0031] Figure 1 This is a schematic diagram of the gesture recognition device of this application;
[0032] Figure 2 This is a diagram showing the location distribution of the sensors and vibration sources.
[0033] Figure 3 This is a diagram showing the structural dimensions of the armband.
[0034] Figure 4 and Figure 5 Schematic diagrams of armband assembly from different perspectives;
[0035] Figure 6 This is a flowchart illustrating the data collection and processing process for this application. Detailed Implementation
[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0037] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.
[0038] The present application will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the application.
[0039] The gesture recognition device based on instantaneous transmitted myosalpingography signals in this embodiment includes N vibration sensors, at least one vibration source, a sound collector, and a recognition processing unit; the N vibration sensors and at least one vibration source form a ring-shaped armband, which is worn on the arm during recognition.
[0040] The vibration source is used to send sinusoidal signals modulated with different vibration frequencies to the arm muscles. The sinusoidal signals modulated with different vibration frequencies penetrate the muscles and generate transmitted myosal signals.
[0041] Each vibration sensor is used to collect transmitted myosalpingography signals at all vibration frequencies and send them to the sound acquisition unit;
[0042] The sound acquisition unit is used to amplify and convert the transmitted muscle sound signals collected by N vibration sensors to obtain N sets of data, and then send them to the recognition and processing unit.
[0043] The recognition and processing unit is used to process the received N sets of data and determine the corresponding gesture.
[0044] Transient myosomatic signal (tMMG) is a modulated, non-stationary vibration signal that penetrates muscle. During finger movement, muscle fiber contraction leads to muscle shortening, an increase in the cross-sectional area of the muscle belly, and a corresponding increase in the diameter of the armband, resulting in increased pressure from the armband on the skin. A single vibration source of the armband can be approximated as a combination of a rigid circular surface and a lightweight spring column. As the pressure from the armband on the skin increases, the normal displacement of the contact area between the skin and the rigid circular surface increases. Simultaneously, the elastic vibrating element inside the rigid circular surface exerts pressure on the skin. In soft tissue mechanics, the stress-strain relationship of hyperelastic materials exhibits a J-curve. Due to the highly nonlinear nature of the skin, its acoustic impedance increases with increasing pressure under low load from the spring column, specifically manifested as a rapid decrease in sound pressure with increasing pressure. Under low-frequency sound waves, the sound field penetrating the muscle approximates a pressure field, and the sound pressure distribution within the muscle is relatively uniform. Besides the influence of sound source pressure, an increase in muscle circumference also directly leads to a slight decrease in sound pressure on the skin surface. By analyzing the modulated vibration signals collected from the skin surface, the muscle contraction state can be classified. Compared to EMG signals, tMMG is less susceptible to interference from sweat, electromagnetic fields, etc. The tMMG method in this application further reduces external vibration interference through complementary recognition of multiple frequency signals, exhibiting superior stability. This application achieves low-latency gesture recognition based on transmitted myosophytic signals. This method not only has low latency but can also acquire signals at intervals through fabric.
[0045] To ensure accurate signal acquisition, this application sets the vibration source on the side of the flexor digitorum profundus muscle, and N vibration sensors on the side of the flexor digitorum superficialis muscle, all avoiding the radius and ulna. Specifically, as shown... Figure 1 As shown, the number of vibration sensors and vibration sources is 4. The 4 vibration sensors need to be located above the superficial flexor digitorum and the 4 vibration sources need to be located below the deep flexor digitorum.
[0046] The detection component of this application is a wearable device, specifically an acoustic array armband. It features mounting positions for vibration sensors and vibration sources, connected by a hinge structure. The first and last mounting positions are then connected by an elastic strap, forming a loop armband. The elastic strap can be secured with nylon buckles, is adjustable, and applies initial pressure to the forearm to accommodate different arm circumferences. The hinge structure also supports mounting position adjustment, further increasing arm circumference adaptability; it can be used by adults with arm circumferences up to 40cm. of
[0047] install The mounting, vibration sensor, vibration source, and hinge connection structure are housed in a 3D-printed shell, making them lightweight and portable.
[0048] The different vibration frequencies used in this application are uniformly selected within the range of 400~2000Hz. If the frequencies are 400Hz, 600Hz, 800Hz, 1000Hz, or 500Hz, 700Hz, 900Hz, 1100Hz, it will not affect the recognition effect. However, the more different vibration frequencies there are, the better the recognition effect will be.
[0049] Specifically, the vibration source outputs a set of sinusoidal signals modulated with four frequencies on the side of the flexor digitorum profundus muscle at a high sampling rate, such as 48000Hz. For example, a set of signals consisting of 400Hz, 600Hz, 800Hz, and 1000Hz has been verified to have excellent recognition performance under 10ms grouping conditions.
[0050] The vibration sensor located on the other side of the finger flexor muscle acquires the transmitted myosalpingography signal that penetrates the muscle at a high sampling rate, such as 48,000 Hz.
[0051] The sound acquisition device divides the acquired raw signal into groups of at least 10ms. It then digitally mixes each of the four frequencies with a set of pure sine or cosine reference signals that are completely synchronized with the target frequency. Finally, it extracts the amplitude and phase information of the frequency component through low-pass filtering to obtain four sets of data.
[0052] The recognition processing unit of this application processes the received four sets of data and determines the corresponding gesture using the following method:
[0053] Feature extraction is performed on each set of data to obtain the characteristic signals of transmitted myosophytic signals at different vibration frequencies. Each vibration sensor corresponds to a set of characteristic signals, and each set of characteristic signals includes features at four vibration frequencies.
[0054] Four sets of feature signals are input into the trained recognition model, and the recognition model outputs the corresponding gesture. The recognition model is a neural network model.
[0055] Before recognition, a training set is constructed, with feature signals as input and corresponding gestures as output to train the recognition model. The recognition module can use the KNN (K nearest neighbor) model. After training, feature signals are collected and extracted in real time and substituted into the trained recognition model to output the corresponding gesture recognition result.
[0056] The characteristic signal of the transmitted myosalpingography signal in this embodiment is:
[0057] ;
[0058] in, It is a characteristic signal of transmitted muscle acoustic signals. This represents the k-th vibration frequency. This indicates the number of different vibration frequencies; for example, four different vibration frequencies. N is the number of sampling points in each data set. It is the sampling frequency of the vibration sensor. Let i be the i-th data in each set of data.
[0059] Based on multi-frequency complementary recognition of tMMG signals, combined with KNN model training, the gesture recognition accuracy can reach 99%, which is on par with existing commercial equipment and has better stability. Specific Implementation
[0060] (a) Equipment parameters and assembly requirements:
[0061] Armband structure: The outer shell consists of 8 units, each unit being a mounting position, which respectively mounts 4 vibration sensors and 4 vibration sources. Each mounting position is fixed by 3 M2 screws. The mounting positions are connected by hinges. The elastic strap (fixed with nylon buckles) is fixed at both ends to the first and last mounting positions. The initial wearing pressure is 0.5~1kPa.
[0062] Core component selection: The vibration sensor uses a 14mm electret microphone, the vibration source is a bone conduction speaker, the sound acquisition card supports a 48000Hz sampling rate, and the microprocessor needs to meet real-time computing requirements (main frequency ≥180MHz).
[0063] Signal parameters: The vibration source output frequency is 400Hz, 600Hz, 800Hz, and 1000Hz (uniformly selected within the range of 400~2000Hz);
[0064] (II) Testing Procedures and Data Processing:
[0065] Test subjects: 10 healthy subjects (aged 20-40, arm circumference 25-35cm) were recruited. Each subject was required to make 10 different hand gestures from 1 to 10, and hold each gesture for 20 seconds while keeping their arm stable throughout.
[0066] Data acquisition: Transmitted myosal signals corresponding to each gesture were acquired through an acoustic array armband. Each gesture was recorded for 20 seconds at a sampling rate of 48,000 Hz. Each 10 ms data set contained 480 data points.
[0067] Feature calculation, N is 480. Take 400Hz, 600Hz, 800Hz, and 1000Hz respectively;
[0068] Model Training and Recognition: The processed feature data is divided into training and testing sets in a 7:3 ratio and input into the KNN training model. In practical gesture recognition applications, the transmitted myosalpingography (TMI) signals corresponding to gestures are collected through an armband. The obtained TMI feature signals are then substituted into the trained KNN model to output the corresponding gesture result.
[0069] In addition to the KNN model, the recognition model in this application can also use SVM or DNN neural networks, with the training process remaining unchanged and the recognition accuracy fluctuation ≤10%;
[0070] The vibration source uses a bone conduction speaker or an electromagnetic speaker, and the vibration sensor uses an accelerometer or a microphone. It is necessary to ensure that the sampling rate matches the signal strength.
[0071] It has strong anti-interference capabilities. Compared with EMG signals, tMMG signals are not affected by sweat or electromagnetic interference. The multi-frequency modulation design further reduces external vibration interference, making it suitable for complex environments (such as outdoor and industrial scenarios).
[0072] While this application has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of this application. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of this application as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.
Claims
1. A gesture recognition device based on instantaneous transmitted myosalpingography (MMI) signals, characterized in that, It includes N vibration sensors, at least one vibration source, a sound collector, and a recognition and processing unit; the N vibration sensors and at least one vibration source form a ring-shaped armband, which is worn on the arm during recognition. The vibration source is used to send sinusoidal signals modulated with different vibration frequencies to the arm muscles. The sinusoidal signals modulated with different vibration frequencies penetrate the muscles and generate transmitted myosal signals. Each vibration sensor is used to collect transmitted myosalpingography signals at all vibration frequencies and send them to the sound acquisition unit; The sound acquisition unit is used to amplify and convert the transmitted muscle sound signals collected by N vibration sensors to obtain N sets of data, and then send them to the recognition and processing unit. The recognition and processing unit is used to process the received N sets of data and determine the corresponding gesture.
2. The gesture recognition device based on instantaneous transmitted myosophytic signals according to claim 1, characterized in that, The vibration source is located on the side of the flexor digitorum profundus muscle, and N vibration sensors are located on the side of the flexor digitorum superficialis muscle, all of which avoid the radius and ulna.
3. The gesture recognition device based on instantaneous transmitted myosophytic signals according to claim 1, characterized in that, The device also includes mounting positions for installing vibration sensors and vibration sources. The two mounting positions are connected by a hinge, and the first and last mounting positions are connected by an elastic strap to form a ring-shaped armband.
4. The gesture recognition device based on instantaneous transmitted myosophytic signals according to claim 3, characterized in that, The mounting position, vibration sensor, vibration source, and hinge connection structure are housed in a 3D-printed shell.
5. The gesture recognition device based on instantaneous transmitted myosophytic signals according to claim 1, characterized in that, Different vibration frequencies were selected uniformly within the range of 400~2000Hz.
6. The gesture recognition device based on instantaneous transmitted myosophytic signals according to claim 1, characterized in that, The recognition processing unit processes the received N sets of data and determines the corresponding gesture using the following methods: Feature extraction is performed on each set of data to obtain the characteristic signals of transmitted myosophytic signals at different vibration frequencies. Each vibration sensor corresponds to a set of characteristic signals. N sets of feature signals are input into a trained recognition model, and the recognition model outputs the corresponding gesture. The recognition model is a neural network model.
7. The gesture recognition device based on instantaneous transmitted myosophytic signals according to claim 6, characterized in that, The characteristic signals of transmitted muscle acoustic signals are: ； in, It is a characteristic signal of transmitted muscle acoustic signals. This represents the k-th vibration frequency. This represents the number of different vibration frequencies, where N is the number of sampling points in each data set. It is the sampling frequency of the vibration sensor. Let i be the i-th data in each set of data.
8. The gesture recognition device based on instantaneous transmitted myosalpingography signal according to claim 6, characterized in that, The recognition model is the KNN model.
9. The gesture recognition device based on instantaneous transmitted myosophytic signals according to claim 1, characterized in that, The vibration source is a bone conduction loudspeaker or an electromagnetic loudspeaker.
10. The gesture recognition device based on instantaneous transmitted myosophytic signals according to claim 1, characterized in that, Vibration sensors can be either accelerometers or microphones.