Gesture recognition method fusing myoelectricity and multi-mode signals of micro-inertial measurement unit

A technology of micro-inertial measurement and gesture recognition, applied in character and pattern recognition, neural learning methods, biological neural network models, etc., to achieve the effect of improving gesture recognition rate and accuracy

Active Publication Date: 2021-08-03
ZHEJIANG UNIV
View PDF6 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

As a form of multimodal data, EMG and IMU signals have multi-source heterogeneity. Currently, there is no effective method to fuse such multi-source heterogeneous multi-modal data for pattern recognition tasks. The invention proposes a multi-view deep learning gesture recognition algorithm, which integrates multi-modal data in the feature subspace to obtain a multi-modal feature representation with more class discrimination

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Gesture recognition method fusing myoelectricity and multi-mode signals of micro-inertial measurement unit
  • Gesture recognition method fusing myoelectricity and multi-mode signals of micro-inertial measurement unit
  • Gesture recognition method fusing myoelectricity and multi-mode signals of micro-inertial measurement unit

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033] The present invention will be described in further detail below in conjunction with the accompanying drawings and specific embodiments.

[0034] Such as figure 1 As shown, the embodiment of the present invention provides a gesture recognition method that fuses myoelectricity and micro-inertial measurement unit multimodal signals, and the specific implementation steps are as follows:

[0035] Step (1) requires the subjects to make corresponding gestures in accordance with the preset gesture sequence, collect the myoelectric data and motion data of several gestures of several subjects through the myoelectric electrodes and the micro-inertial measurement unit, and several times of one gesture Repetition corresponds to a data file, and the corresponding gesture label is stored in the data file; each gesture action is repeated 3 times during the collection process, and a rest gesture needs to be maintained for a certain period of time between each two repetitions; the motion...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a gesture recognition method fusing myoelectricity and micro-inertial measurement unit multi-modal signals, which comprises the following steps: acquiring myoelectricity data and motion data by using a myoelectricity electrode and a micro-inertial measurement unit, performing synchronous processing on the myoelectricity data and the motion data, and dividing a training set and a test set; dividing each signal segment into a plurality of sub-signal segments with fixed lengths by using a sliding window, and respectively extracting time domain and frequency domain features from the myoelectricity data and the motion data of each sub-signal segment; and respectively extracting shallow and deep features of the myoelectricity features and the motion features by using a convolutional neural network, respectively fusing the shallow and deep features, inputting the fused features into a classification network, finally fusing and outputting the probability of each gesture category in a decision-making layer, training a recognition model, and testing to obtain a gesture recognition rate. According to the gesture recognition method fusing the myoelectricity and the multi-modal signals of the micro-inertial measurement unit, respective advantages of the myoelectricity and the motion data can be fully utilized, so that various different gestures of the same subject can be recognized more accurately.

Description

technical field [0001] The invention belongs to the field of combining computers with biological signals and motion signals, and in particular relates to a gesture recognition method based on deep learning and multi-view and multi-modal learning. Background technique [0002] Surface electromyography (sEMG) is a biological signal that records muscle activity through non-invasive electrodes attached to the skin surface. It has important academic value and Significance of application; Inertial measurement unit (IMU) is a device for measuring the three-axis attitude angle and acceleration of an object, and has a wide range of applications in motion control equipment, such as automobiles and robots. Gesture recognition technology that integrates EMG and MMU multimodal signals can take advantage of the respective advantages of two different modal data to improve the accuracy of gesture recognition methods. Among them, multi-view deep learning algorithms are often used for multi-...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06V40/117G06V40/113G06N3/045G06F2218/08G06F18/253
Inventor 耿卫东金文光厉向东梁秀波戴青锋朱俊威毋从周韩晨晨周洲姬源智刘帅
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products