Gesture recognition method based on surface electromyogram signal continuous wavelet transform image

Abstract

The invention discloses a surface electromyogram signal continuous wavelet transform image-based gesture recognition method, which comprises the following steps of: 1, preprocessing a surface electromyogram signal, recognizing an active section and segmenting different gestures; 2, performing continuous wavelet transform processing on each channel; 3, drawing an energy distribution image, and independently storing each channel picture in a jpg format; 4, performing integration processing on each channel image; 5, windowing the image; and 6, performing pattern classification on the gestures by using the convolutional neural network model. The method solves the problem that the extracted myoelectricity information has certain redundancy and limitation in the existing feature extraction and selection method in the gesture recognition process based on the surface myoelectricity signals, and is of great significance in improving the gesture recognition capability. Meanwhile, the method can be used for processing high-dimensional data and capturing details of complex gestures, and can be applied to multiple fields such as human-computer interaction, medical rehabilitation and intelligent control.

Description

Technical Field

[0001] The present invention relates to a gesture recognition method, and in particular to a gesture recognition method based on a surface electromyography information image of a convolutional neural network. Background Art

[0002] The main idea of ​​existing gesture recognition technology based on surface electromyography is to establish a model with better accuracy and robustness by obtaining effective electromyography information. In terms of feature extraction of electromyography signals, time domain features, frequency domain features, and time-frequency domain features are often used to extract electromyography information, and then combined with data dimensionality reduction analysis methods to reduce feature dimensions and optimize data. The choice of model depends more on the specific target task, and commonly used ones include traditional machine learning, deep learning, and hybrid algorithms. The existing research level of related technologies has achieved some results in realizing gesture recognition, but there are still certain shortcomings that need to be further studied: 1) Existing feature extraction methods may extract multiple features with similar information, resulting in redundant data and increasing the computational burden. 2) Some feature selection methods have large computational complexity and long processing time. 3) The nonlinear characteristics of electromyography signals are difficult to fully capture through linear feature extraction methods, which limits the recognition ability. Summary of the invention

[0003] In view of the problem that the existing methods for feature extraction and selection in the process of gesture recognition based on surface electromyography signals still have certain redundancy and limitations in the extracted electromyography information, the purpose of the present invention is to provide a gesture recognition method based on continuous wavelet transform image of surface electromyography signals.

[0004] The objective of the present invention is achieved through the following technical solutions:

[0005] A gesture recognition method based on surface electromyography signal continuous wavelet transform image comprises the following steps:

[0006] Step 1: pre-process the surface electromyography signal to identify the activity segment and segment different gestures;

[0007] Step 2: Perform continuous wavelet transform on each channel: By performing continuous wavelet transform on each channel, the time-frequency relationship of each channel is obtained;

[0008] Step 3: Draw the energy distribution image and save each channel image separately in jpg format: Generate a scale map with time on the x-axis and scale on the y-axis, convert the scale map into an RGB image, resize the RGB image and save it as a jpg file;

[0009] Step 4: Integrate and process the images of each channel: Integrate and splice the energy distribution images of each channel of the same action to obtain a longitudinal splicing image of each channel;

[0010] Step 5: Windowing the image: Windowing each action image in the vertical mosaic image obtained in step 4 in turn, horizontally mosaicing the image according to the channel to obtain a horizontal mosaic image of each action;

[0011] Step 6: Use the horizontal splicing image generated in step 5 as the input of the convolutional neural network model, and use the convolutional neural network model to classify the gesture pattern.

[0012] Compared with the prior art, the present invention has the following advantages:

[0013] The present invention solves the problem that the existing methods for feature extraction and selection in the process of gesture recognition based on surface electromyography signals still have certain redundancy and limitations in the extracted electromyography information, which is of great significance to the improvement of gesture recognition capabilities. At the same time, the method can process high-dimensional data and capture the details of complex gestures, and can be applied to many fields such as human-computer interaction, medical rehabilitation and intelligent control. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of a gesture recognition method based on continuous wavelet transform image of surface electromyography signal;

[0015] Figure 2 It is the 12-channel energy distribution image of action one;

[0016] Figure 3 It is the 12-channel energy distribution image of action 2;

[0017] Figure 4 It is the 12-channel energy distribution image of action three;

[0018] Figure 5 It is the 12-channel energy distribution image of action four;

[0019] Figure 6 It is the 12-channel energy distribution image of action five;

[0020] Figure 7 It is the 12-channel energy distribution image of action six;

[0021] Figure 8 It is the 12-channel energy distribution image of Action 7;

[0022] Fig. 9 It is the 12-channel energy distribution image of Action Eight;

[0023] Fig.10It is the 12-channel energy distribution image of action nine;

[0024] Fig.11 It is the motion + 12-channel energy distribution image;

[0025] Fig.12 It is the 12-channel energy distribution image of Action 11;

[0026] Fig.13 It is the 12-channel energy distribution image of action twelve;

[0027] Fig.14 It is the 12-channel energy distribution image of Action 13;

[0028] Fig.15 It is the 12-channel energy distribution image of Action 14;

[0029] Fig.16 It is the 12-channel energy distribution image of Action 15;

[0030] Fig.17 It is the 12-channel energy distribution image of Action Sixteen;

[0031] Fig.18 It is the 12-channel energy distribution image of Action 17;

[0032] Fig.19 This is a picture processing diagram (taking action 1 as an example);

[0033] Fig. 20 is the recognition result graph (first time);

[0034] Fig.21 is the recognition result graph (second time);

[0035] Fig. 22 is the recognition result graph (third time);

[0036] Fig.23 This is the recognition result diagram (fourth time)

[0037] Fig.24 This is the recognition result graph (fifth time). DETAILED DESCRIPTION

[0038] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0039] The present invention provides a gesture recognition method based on the continuous wavelet transform image of surface electromyography signal. Gesture recognition is essentially a classification problem. The main idea of ​​the present invention is to transform the classification problem into an image recognition problem. By converting data into an image, the extracted electromyography information features are made more complete, and important features are automatically extracted through a model. The specific steps are as follows:

[0040] Step 1: Preprocess the surface electromyography signal to identify the activity segments and segment different gestures. The specific steps are as follows:

[0041] Step 1: Filter the original signal with a 50 Hz power frequency noise filter and a 5-450 Hz bandpass filter to improve the signal-to-noise ratio;

[0042] Step 1 and 2: Distinguish the parts with and without movement from the continuous surface electromyography signal, and filter out the parts with movement.

[0043] Step 2: Perform Morse continuous wavelet transform on each channel: For non-stationary signals such as surface electromyography, the time-frequency window needs to be adjustable, that is, it requires good time resolution characteristics in the high-frequency part and good frequency resolution characteristics in the low-frequency part. The basic function of continuous wavelet transform is to localize time, which is used to construct the time-frequency representation of the signal to provide very good time and frequency positioning. Morse wavelet is a complex mother wavelet containing polynomial and exponential functions, and the time and frequency are adjusted by the time bandwidth parameter β and the symmetry parameter γ. By performing Morse continuous wavelet transform on each channel, its time-frequency relationship is obtained.

[0044] Step 3: Draw its energy distribution image and save each channel image separately in jpg format: Generate a scale map with time on the x-axis and scale (related to frequency) on the y-axis. Convert the scale map into an RGB image. Through the change of color, the RGB image can intuitively show the energy distribution of the signal at different times and scales, which is helpful for subsequent analysis and interpretation of the characteristics and patterns of the signal. Finally, adjust the image size (60×25000) and save it as a jpg file.

[0045] Step 4: Integrate and process the images of each channel: Integrate and splice the energy distribution of each channel of the same action to obtain a longitudinal splicing image of each channel.

[0046] Step 5: Window the image: Window each action image in the spliced ​​image obtained in step 4 in turn, and splice the image horizontally according to the channel after resolution conversion processing, so as to obtain 5000 images for each action, and the image resolution is 60×160.

[0047] Step 6: Model training: Use the image generated in step 5 as the input of the convolutional neural network model, and use the convolutional neural network model to classify gesture patterns.

[0048] In this step, the first layer of the neural convolutional network model is the input layer, which accepts a 3-channel RGB image as input. The second layer is a two-dimensional convolutional layer with a convolution kernel size of 3×3 and an output channel number of 32. Then the normalization layer is used to accelerate the training and reduce the gradient vanishing problem. The nonlinear activation is performed through the ReLU activation function layer. Below the ReLU activation function layer is the fully connected layer, and the number of neurons corresponds to the number of categories. Finally, the Softmax layer is used to convert the output of the network into a probability distribution, and the final classification task is performed through the classification output layer.

[0049] The present invention verifies the method by using the data from DB2 in the Ninapro dataset. The surface electromyography signal data in the selected dataset is obtained by 12 Delsys Trigno electrodes with a sampling frequency of 2000 Hz, including 17 different gestures. Figure 2-Figure 24 As shown, after experimental verification, the average accuracy of the present invention in recognizing 17 different gestures is 94.95%, which shows that the method has high accuracy and strong recognition ability.

Claims

1. A gesture recognition method based on continuous wavelet transform image of surface electromyography signal, characterized in that The method comprises the following steps: Step 1: pre-process the surface electromyography signal to identify the activity segment and segment different gestures; Step 2: Perform continuous wavelet transform on each channel: Perform Morse continuous wavelet transform on each channel to obtain the time-frequency relationship of each channel; Step 3: Draw the energy distribution image and save each channel image separately in jpg format: Generate a scale map with time on the x-axis and scale on the y-axis, convert the scale map into an RGB image, resize the RGB image and save it as a jpg file; Step 4: Integrate and process the images of each channel: Integrate and splice the energy distribution images of each channel of the same action to obtain a longitudinal splicing image of each channel; Step 5: Windowing the image: Windowing each action image in the vertical mosaic image obtained in step 4 in turn, horizontally mosaicing the image according to the channel to obtain a horizontal mosaic image of each action; Step 6: Use the horizontal splicing image generated in step 5 as the input of the convolutional neural network model, and use the convolutional neural network model to perform pattern classification.

2. The gesture recognition method based on surface electromyography continuous wavelet transform image according to claim 1 is characterized in that The specific steps of step one are as follows: Step 1: Filter the original signal with a 50 Hz power frequency noise filter and a 5-450 Hz bandpass filter to improve the signal-to-noise ratio; Step 1 and 2: Distinguish the parts with and without movement from the continuous surface electromyography signal, and filter out the parts with movement.

Images