Gesture recognition method fusing myoelectricity and multi-mode signals of micro-inertial measurement unit

Abstract

The invention discloses a gesture recognition method fusing myoelectricity and micro-inertial measurement unit multi-modal signals, which comprises the following steps: acquiring myoelectricity data and motion data by using a myoelectricity electrode and a micro-inertial measurement unit, performing synchronous processing on the myoelectricity data and the motion data, and dividing a training set and a test set; dividing each signal segment into a plurality of sub-signal segments with fixed lengths by using a sliding window, and respectively extracting time domain and frequency domain features from the myoelectricity data and the motion data of each sub-signal segment; and respectively extracting shallow and deep features of the myoelectricity features and the motion features by using a convolutional neural network, respectively fusing the shallow and deep features, inputting the fused features into a classification network, finally fusing and outputting the probability of each gesture category in a decision-making layer, training a recognition model, and testing to obtain a gesture recognition rate. According to the gesture recognition method fusing the myoelectricity and the multi-modal signals of the micro-inertial measurement unit, respective advantages of the myoelectricity and the motion data can be fully utilized, so that various different gestures of the same subject can be recognized more accurately.

Description

technical field [0001] The invention belongs to the field of combining computers with biological signals and motion signals, and in particular relates to a gesture recognition method based on deep learning and multi-view and multi-modal learning. Background technique [0002] Surface electromyography (sEMG) is a biological signal that records muscle activity through non-invasive electrodes attached to the skin surface. It has important academic value and Significance of application; Inertial measurement unit (IMU) is a device for measuring the three-axis attitude angle and acceleration of an object, and has a wide range of applications in motion control equipment, such as automobiles and robots. Gesture recognition technology that integrates EMG and MMU multimodal signals can take advantage of the respective advantages of two different modal data to improve the accuracy of gesture recognition methods. Among them, multi-view deep learning algorithms are often used for multi-...

AI Technical Summary

Problems solved by technology

As a form of multimodal data, EMG and IMU signals have multi-source heterogeneity. Currently, there is no effective method to fuse such multi-source heterogeneous multi-modal data for pattern recognition tasks. The invention proposes a multi-view deep learning gesture recognition algorithm, which integrates multi-modal data in the feature subspace to obtain a multi-modal feature representation with more class discrimination

Images