A gesture recognition method based on continuous wavelet transform images of surface electromyogram signals

Abstract

The present invention discloses a gesture recognition method based on continuous wavelet transform images of surface electromyography signals, the method being as follows: 1. pre-processing the surface electromyography signals to identify active segments and segment different gestures; 2. performing continuous wavelet transform processing on each channel; 3. drawing an energy distribution image and saving each channel image separately in jpg format; 4. integrating and processing the images of each channel; 5. performing windowing processing on the image; 6. using a convolutional neural network model to perform pattern classification on gestures. The present invention solves the problem that the existing methods for feature extraction and selection in the process of gesture recognition based on surface electromyography signals still have certain redundancy and limitations in the extracted electromyographic information, and is of great significance to the improvement of gesture recognition capabilities. At the same time, the method can perform high-dimensional data processing and capture the details of complex gestures, and can be applied to many fields such as human-computer interaction, medical rehabilitation, and intelligent control.

Description

Technical Field

[0001] The present invention relates to a gesture recognition method, and in particular to a gesture recognition method based on surface electromyography information images using a convolutional neural network. Background Art

[0002] The main idea behind existing gesture recognition technologies based on surface electromyography (EMG) signals is to acquire effective EMG information to establish models with better accuracy and robustness. When extracting EMG signals, methods such as time-domain features, frequency-domain features, and time-frequency-domain features are often used to extract EMG signal information, and then combined with data dimensionality reduction analysis to reduce feature dimensions and optimize the data. The choice of model depends more on the specific target task, with commonly used methods including traditional machine learning, deep learning, and hybrid algorithms. While existing research in related technologies has achieved some results in realizing gesture recognition, there are still certain deficiencies that require further investigation: 1) Existing feature extraction methods may extract multiple features with similar information, resulting in redundant data and increasing the computational burden. 2) Some feature selection methods are computationally intensive and require long processing times. 3) The nonlinear characteristics of EMG signals are difficult to fully capture using linear feature extraction methods, limiting recognition capabilities. Summary of the Invention

[0003] In view of the fact that the existing methods for feature extraction and selection in gesture recognition based on surface electromyography still have certain redundancy and limitations in the extracted electromyography information, the purpose of the present invention is to provide a gesture recognition method based on continuous wavelet transform image of surface electromyography.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A gesture recognition method based on surface electromyography signal continuous wavelet transform image includes the following steps:

[0006] Step 1: Preprocess the surface electromyography signal to identify the activity segments and segment different gestures;

[0007] Step 2: Perform continuous wavelet transform on each channel: By performing continuous wavelet transform on each channel, the time-frequency relationship of each channel is obtained;

[0008] Step 3: Draw the energy distribution image and save each channel image separately in jpg format: Generate a scalogram with time on the x-axis and scale on the y-axis, convert the scalogram into an RGB image, resize the RGB image and save it as a jpg file;

[0009] Step 4: Integrate and process the images of each channel: Integrate and splice the energy distribution images of each channel of the same action to obtain a longitudinal splicing image of each channel;

[0010] Step 5: Windowing the image: Windowing each action image in the vertical mosaic obtained in step 4 is performed in sequence, and the images are horizontally mosaicked according to the channels to obtain a horizontal mosaic of each action;

[0011] Step 6: Use the horizontal splicing image generated in step 5 as the input of the convolutional neural network model, and use the convolutional neural network model to perform pattern classification on the gesture.

[0012] Compared with the prior art, the present invention has the following advantages:

[0013] This invention addresses the redundancy and limitations of the extracted EMG information in existing methods for feature extraction and selection during gesture recognition based on surface EMG signals, significantly improving gesture recognition capabilities. Furthermore, this method can process high-dimensional data and capture the details of complex gestures, potentially finding applications in a variety of fields, including human-computer interaction, medical rehabilitation, and intelligent control. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of a gesture recognition method based on continuous wavelet transform image of surface electromyography signal;

[0015] Figure 2 It is the 12-channel energy distribution image of action one;

[0016] Figure 3 It is the 12-channel energy distribution image of action 2;

[0017] Figure 4 It is the 12-channel energy distribution image of action three;

[0018] Figure 5 It is the 12-channel energy distribution image of action four;

[0019] Figure 6 It is the 12-channel energy distribution image of action five;

[0020] Figure 7 It is the 12-channel energy distribution image of action six;

[0021] Figure 8 It is the 12-channel energy distribution image of Action 7;

[0022] Figure 9 It is the 12-channel energy distribution image of action eight;

[0023] Figure 10It is the 12-channel energy distribution image of action nine;

[0024] Figure 11 It is the motion + 12-channel energy distribution image;

[0025] Figure 12 It is the 12-channel energy distribution image of action eleven;

[0026] Figure 13 It is the 12-channel energy distribution image of action twelve;

[0027] Figure 14 It is the 12-channel energy distribution image of Action 13;

[0028] Figure 15 It is the 12-channel energy distribution image of action fourteen;

[0029] Figure 16 It is the 12-channel energy distribution image of action fifteen;

[0030] Figure 17 It is the 12-channel energy distribution image of action sixteen;

[0031] Figure 18 It is the 12-channel energy distribution image of action seventeen;

[0032] Figure 19 This is a diagram of the image processing process (taking action 1 as an example);

[0033] Figure 20 is the recognition result graph (first time);

[0034] Figure 21 is the recognition result graph (second time);

[0035] Figure 22 is the recognition result graph (third time);

[0036] Figure 23 This is the recognition result graph (fourth time)

[0037] Figure 24 This is the recognition result graph (fifth time). DETAILED DESCRIPTION

[0038] The technical solution of the present invention is further described below with reference to the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention that does not depart from the spirit and scope of the technical solution of the present invention should be included in the scope of protection of the present invention.

[0039] The present invention provides a gesture recognition method based on continuous wavelet transform images of surface electromyographic signals. Gesture recognition is essentially a classification problem. The main idea of the present invention is to transform the classification problem into an image recognition problem. By converting data into an image, the extracted electromyographic information features are made more complete, and important features are automatically extracted through a model. The specific steps are as follows:

[0040] Step 1: Preprocess the surface electromyography signal to identify the activity segments and segment different gestures. The specific steps are as follows:

[0041] Step 1: Filter the original signal with a 50 Hz power frequency noise filter and a 5-450 Hz bandpass filter to improve the signal-to-noise ratio;

[0042] Steps 1 and 2: Distinguish the parts with and without movement from the continuous surface electromyography signal, and filter out the parts with movement.

[0043] Step 2: Perform Morse continuous wavelet transform on each channel: For non-stationary signals such as surface electromyography, the time-frequency window needs to be adjustable, that is, it requires better time resolution characteristics in the high-frequency part and better frequency resolution characteristics in the low-frequency part. The basic function of the continuous wavelet transform is to localize time, which is used to construct the time-frequency representation of the signal to provide very good time and frequency positioning. The Morse wavelet is a complex mother wavelet containing polynomial and exponential functions, and the time and frequency are adjusted by the time bandwidth parameter β and the symmetry parameter γ. By performing Morse continuous wavelet transform on each channel, its time-frequency relationship is obtained.

[0044] Step 3: Plot the energy distribution image and save each channel image separately in .jpg format. Generate a scalogram with time on the x-axis and scale (related to frequency) on the y-axis. Convert the scalogram into an RGB image. Through color variations, the RGB image can intuitively display the energy distribution of the signal at different times and scales, facilitating subsequent analysis and interpretation of signal characteristics and patterns. Finally, resize the image (60 × 25,000) and save it as a .jpg file.

[0045] Step 4: Integrate and process the images of each channel: Integrate and splice the energy distribution of each channel of the same action to obtain a longitudinal splicing image of each channel.

[0046] Step 5: Window the images: Window each action image in the spliced image obtained in step 4 in turn. After resolution conversion, the images are horizontally spliced according to the channels to obtain 5,000 images for each action, with a resolution of 60×160.

[0047] Step 6: Model training: Use the image generated in step 5 as the input of the convolutional neural network model, and use the convolutional neural network model to classify gesture patterns.

[0048] In this step, the first layer of the neural convolutional network model is the input layer, which accepts a 3-channel RGB image as input. The second layer is a two-dimensional convolutional layer with a convolution kernel size of 3×3 and 32 output channels. A normalization layer is then used to accelerate training and reduce the vanishing gradient problem. A ReLU activation function layer is used for nonlinear activation. Below the ReLU activation function layer is a fully connected layer, where the number of neurons corresponds to the number of categories. Finally, a Softmax layer is used to convert the network output into a probability distribution, which is then passed through the classification output layer to perform the final classification task.

[0049] The present invention verifies the method by using the data from DB2 in the Ninapro dataset. The surface electromyography signal data in the selected dataset is obtained by 12 Delsys Trigno electrodes with a sampling frequency of 2000Hz, including 17 different gestures. Figure 2-Figure 24 As shown in FIG, after experimental verification, the average accuracy of the present invention in recognizing 17 different gestures is 94.95%, indicating that the method has high accuracy and strong recognition ability.

Claims

1. A gesture recognition method based on continuous wavelet transform image of surface electromyography signal, characterized in that The method comprises the following steps: Step 1: Preprocess the surface electromyography signal to identify the activity segments and segment different gestures; Step 2: Perform continuous wavelet transform on each channel: Perform Morse continuous wavelet transform on each channel to obtain the time-frequency relationship of each channel; Step 3: Draw the energy distribution image and save each channel image separately in jpg format: Generate a scalogram with time on the x-axis and scale on the y-axis, convert the scalogram into an RGB image, resize the RGB image and save it as a jpg file; Step 4: Integrate and process the images of each channel: Integrate and splice the energy distribution images of each channel of the same action to obtain a longitudinal splicing image of each channel; Step 5: Windowing the image: Windowing each action image in the vertical mosaic obtained in step 4 is performed in sequence, and the images are horizontally mosaicked according to the channels to obtain a horizontal mosaic of each action; Step 6: Use the horizontal splicing image generated in step 5 as the input of the convolutional neural network model and use the convolutional neural network model to perform pattern classification.

2. The gesture recognition method based on surface electromyography continuous wavelet transform image according to claim 1 is characterized in that The specific steps of step one are as follows: Step 1: Filter the original signal with a 50 Hz power frequency noise filter and a 5-450 Hz bandpass filter to improve the signal-to-noise ratio; Steps 1 and 2: Distinguish the parts with and without movement from the continuous surface electromyography signal, and filter out the parts with movement.

Images