EMG signal gesture prediction system based on dual-stream feature fusion convolutional neural network
Through the dual-stream feature fusion convolutional neural network, using technologies such as narrow convolution kernel and Batch Normalization, the problem of delayed output time of control commands in gesture recognition in the existing technology is solved, and the accuracy and timeliness of gesture recognition are improved, so as to achieve More efficient prediction of EMG signal gestures.
Patent Information
- Application Number
- CN202310088533.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-09
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-02-09
AI Technical Summary
The existing deep networks for gesture recognition do not fully consider the time series characteristics and channel independence of surface electromyography signals, resulting in delayed control command output time, affecting the accuracy and timeliness of gesture recognition.
Using an electromyography signal gesture prediction system based on dual-current feature fusion convolutional neural network, temporal features and cross-channel spatial features are extracted through narrow and long convolution kernels, combined with shallow and deep features for prediction, and using Batch Normalization and LeakyRelu activation functions to reduce parameters and enhance the robustness of the network.
It realizes faster output of control instructions, improves control efficiency and prediction accuracy, reduces the complexity of model training and deployment, and enhances the robustness and prediction accuracy of the model.
Smart Images

Figure CN116172582B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of human physiological signal decoding, and in particular to an electromyographic signal gesture prediction system based on a dual-stream feature fusion convolutional neural network. Background Art
[0002] As an important organ of human beings, hands are not only the main tool for us to work, but also an important tool for communication. Human hands can not only make fine and complex gestures, but also convey thoughts, emotions and behavioral intentions. They are an important way for humans to interact with the outside world.
[0003] Due to war, accidents, work-related injuries and other reasons, a large number of patients lose their hands or parts of their arms every year; at the same time, the number of people with limb disabilities caused by diseases such as spinal cord injuries and cerebral stroke is also increasing; according to the second national sample survey of people with disabilities, limb disabilities account for about 30% of the total number of people with disabilities; the loss or dysfunction of limbs has brought many inconveniences and troubles to the daily life, work and communication of people with disabilities;
[0004] Prosthetic limbs (also called "artificial limbs") can restore the original shape or function of the patient's missing limbs, reduce functional impairments, and improve the quality of life. Currently, there are three main types of prosthetic hands: the first type is a prosthetic limb that only serves as a decoration and cannot perform any movements; the second type is a simple mechanical cable-controlled prosthetic limb that can perform simple movements with a single degree of freedom; the third type is a myoelectric prosthetic limb;
[0005] Myoelectric prosthetic hands mainly use the surface electromyographic signals near the patient's residual limb as the control basis of the prosthesis, and use certain signal processing methods to use the processing results as the control signal of the prosthetic control system; since myoelectric prostheses are a type of prosthesis that uses the body's own physiological signals for control, accurately and efficiently identifying gestures based on surface electromyographic signals, judging the body's movement intentions and outputting control signals are the key issues in myoelectric prosthetic hand control;
[0006] Due to its excellent performance in image feature extraction, deep learning algorithms have been applied to EMG decoding for gesture recognition and achieved good results;
[0007] In the prior art, LeNet is used to perform offline training on self-collected gesture electromyographic data, achieving a high gesture recognition accuracy;
[0008] You can also use a large amount of gesture EMG data from different sources for pre-training, and then use a small amount of collected personal EMG data to calibrate the model and use it for online real-time evaluation;
[0009] The HDC-BiGRU-Attention hybrid model is used to enhance the feature extraction capability of the algorithm, which improves the classification accuracy while reducing the amount of calculation;
[0010] However, the deep networks currently used for gesture recognition do not fully consider the time series characteristics and channel independence of surface electromyographic signals. The use of traditional square convolution kernels destroys channel independence, and the time correlation features within independent channels are not fully mined. In addition, model training and gesture recognition are more focused on the data during the gesture holding period, and cannot operate effectively during the action execution period, which delays the output time of control commands. The above problems have affected the accuracy and timeliness of gesture recognition to a certain extent. Summary of the invention
[0011] The purpose of the present invention is to provide an electromyographic signal gesture prediction system based on a dual-stream feature fusion convolutional neural network to solve the problems raised in the above background technology.
[0012] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:
[0013] The present invention is an electromyographic signal gesture prediction system based on a dual-stream feature fusion convolutional neural network, comprising a gesture action acquisition module, an electromyographic signal preprocessing module, a model data preparation module, a model construction module, a training and testing module, a model prediction module, and an electrical stimulation auxiliary module;
[0014] The gesture action acquisition module is used to collect electromyographic signals during the execution of gesture actions;
[0015] The electromyographic signal preprocessing module is used to filter, trap and reduce noise of electromyographic signals, and label the training data;
[0016] The model data preparation module is used to intercept data, normalize data and construct input matrix;
[0017] The model building module is mainly used to build a dual-stream feature fusion convolutional neural network;
[0018] The training and testing module trains and tests the model through the collected gesture action electromyography data;
[0019] The model prediction module collects the electromyographic data of the gesture action in real time, analyzes and predicts the gesture action according to the electromyographic data of the execution period, and outputs the prediction result;
[0020] The electrical stimulation auxiliary module drives the electrical stimulator to output stimulation current according to the output prediction result, so as to assist in completing the execution of the gesture action.
[0021] Furthermore, the gesture action acquisition module acquires the surface electromyographic signals of the upper limbs of each subject when performing different gestures through patch electrodes;
[0022] The upper limb surface electromyography signal includes execution period data and holding period data;
[0023] The execution period data comes from the dynamic process of the gesture action from the beginning of execution to the formation of the gesture;
[0024] The holding period data comes from the holding stage after the gesture is formed.
[0025] Furthermore, in the gesture action acquisition module, the locations of the patch electrodes are respectively the flexor digitorum superficialis, the brachioradialis, the palmaris longus, the proper extensor of the index finger, the extensor digitorum, and the proper extensor of the thumb.
[0026] Further, the filtering of the electromyographic signal is used to perform Fourier transform on the signal and use a bandpass filter;
[0027] The notch of the electromyographic signal is used to pass through a 50 Hz notch filter to eliminate the power frequency interference of the power line;
[0028] The noise reduction of the electromyographic signal is used to decompose various frequency components in the signal into non-overlapping frequency bands by using wavelet decomposition based on the sym mother wave, detect the components of the mutation signal in the normal signal, and remove the stubborn noise.
[0029] Further, the data interception is used to intercept the data in each data segment;
[0030] The data normalization is used to normalize the training set and the test set in the data interception;
[0031] The input matrix construction refers to converting the electromyographic signal into a three-dimensional electromyographic matrix as the input of the model.
[0032] Furthermore, the dual-stream feature fusion convolutional neural network is used to perform feature extraction and classification prediction on the input three-dimensional electromyographic matrix;
[0033] The dual-stream feature fusion convolutional neural network includes an input module, a convolutional neural network pathway, a feature fusion convolution module, a fully connected module and a classification decision module;
[0034] The input module is used to divide the input three-dimensional electromyographic matrix into two parts of the same size according to the channel dimension;
[0035] There are two convolutional neural network pathways, which are used to extract features from two parts of the same size divided by the input module respectively;
[0036] There are two feature fusion convolution modules, one of which is used to perform feature fusion on shallow feature maps x1 and x2, and the other is used to perform feature fusion on deep feature maps y1 and y2;
[0037] The two fully connected modules are used to further extract features from the feature graphs x and y to obtain two one-dimensional feature vectors;
[0038] The classification decision module concatenates shallow features and deep features into a one-dimensional feature vector, which is then sent to the am-softmax layer, and finally outputs a one-dimensional vector, which is the classification result.
[0039] Furthermore, the batch normalization layers in the convolutional neural network pathway and the feature fusion convolution module all use Batch Normalization to accelerate network convergence; the nonlinear activation layers all use LeakyRelu activation function to enhance the nonlinear expression ability of the network.
[0040] Furthermore, the pooling layers all use average pooling, which is more consistent with the characteristics of electromyographic signals than maximum pooling.
[0041] Furthermore, in the model prediction module, the electromyographic signals of the subjects are collected in real time, the electromyographic signals are analyzed, and the hand gestures are predicted using the trained model, and the hand gestures are converted into driving signals of the electric stimulator.
[0042] Furthermore, in the electrical stimulation auxiliary module, the stimulation current is provided by a four-way electrical stimulator, and the stimulation electrodes are respectively located at the flexor digitorum superficialis, brachioradialis, palmaris longus, intrinsic extensor of the index finger, extensor digitorum, and intrinsic extensor of the thumb.
[0043] The present invention has the following beneficial effects:
[0044] The present invention trains the model by using the data of the execution period of the gesture action, so that the model can predict the gesture action based on the data of the execution period of the gesture action, and then output the control instruction faster, thereby improving the control efficiency.
[0045] When designing the convolution structure, the present invention introduces a narrow convolution kernel based on the characteristics of the surface electromyographic signal time series, and uses a multi-layer convolution kernel with a size of k*1 for electromyographic signal feature extraction, effectively extracting the time characteristics inside each channel and fully mining the internal information of the channel. At the same time, compared with the traditional convolution kernel with the same length and width, the use of a narrow convolution kernel saves the number of parameters, reduces the algorithm complexity, reduces the possibility of overfitting, and reduces the burden of model training and deployment.
[0046] When designing the convolution structure, the present invention takes into account the fact that there is a certain correlation between the channels of surface electromyographic signals. After multiple layers of convolution layers with a convolution kernel size of k*1, the present invention adds a convolution layer with a convolution kernel size of 1*3 to effectively capture the spatial features across channels and further explore the correlation between channels.
[0047] When designing the convolution structure, the present invention introduces multi-stream fusion, which has better fitting performance for training data, stronger model robustness and higher prediction accuracy compared with the traditional single-stream network.
[0048] When designing the convolution structure, the present invention uses the shallow features and deep features extracted by the network together for the prediction and classification of gesture actions, so that the electromyographic feature information obtained by the network is richer, and the robustness of the model and the accuracy of the prediction are enhanced. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0050] Figure 1 This is a flow chart of the electromyographic signal gesture prediction system based on dual-stream feature fusion convolutional neural network of the present invention;
[0051] Figure 2 This is a schematic diagram of surface electromyography signal collection in the present invention;
[0052] Figure 3 Schematic diagram of the dual-stream feature fusion convolutional neural network structure of the present invention. Implementation
[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] See also Figure 1-Figure 3 As shown, the present invention is an electromyographic signal gesture prediction system based on a dual-stream feature fusion convolutional neural network, including a gesture action acquisition module, an electromyographic signal preprocessing module, a model data preparation module, a model construction module, a training and testing module, a model prediction module and an electrical stimulation auxiliary module;
[0055] The gesture action acquisition module mainly realizes the acquisition of electromyographic signals during the execution of gesture actions, such as Figure 2 As shown;
[0056] Specifically, the gesture collection module uses patch electrodes to collect the upper limb surface electromyographic signals of each subject when making different gestures, including data from the execution period and the holding period. The execution period data comes from the dynamic process of the gesture from the beginning to the formation of the gesture, and the holding period data comes from the holding stage after the gesture is formed. The gesture execution process is controlled to be completed within 2 seconds, and the action needs to be maintained for 2 seconds after it is formed. The purpose of holding the action is to simulate the real scene.
[0057] For example, in the gesture collection module, the number of patch electrodes is 6, and the electrode positions are respectively the superficial flexor of the finger, the brachioradialis, the palmaris longus, the intrinsic extensor of the index finger, the extensor of the finger, and the intrinsic extensor of the thumb. The collected gestures include 30 actions such as clenching a fist, opening, and pinching fingers. A total of 20 subjects were recruited, and each subject performed each action 20 times to ensure sufficient data volume. The sampling frequency of the electromyographic signal is 500Hz, and each action is performed for about 4 seconds, resting for 3 seconds, and resting for 50 seconds after every 20 executions to eliminate the adverse effects of muscle fatigue, and 12,000 data segments are obtained. At the same time, before each subject is collected, standardized steps such as collection training, interference elimination, and corrective actions are required to ensure the quality of the collected data.
[0058] The EMG signal preprocessing module includes filtering, notching, and noise reduction of the EMG signal, and labeling the training data.
[0059] Specifically, in the EMG signal preprocessing module, EMG signal filtering refers to Fourier transforming the signal and using a bandpass filter with a frequency range of 10-500Hz (the frequency of the main components of the surface EMG signal); notching refers to using a 50Hz notch filter to eliminate power line interference; noise reduction refers to using a wavelet decomposition based on the sym mother wave to decompose the various frequency components in the signal into non-overlapping frequency bands, detecting the components of the mutation signal in the normal signal to remove stubborn noise. At the same time, 12,000 data segments were manually labeled, with label numbers from 0 to 29, representing 30 different gestures.
[0060] The model data preparation module mainly includes data capture, data normalization and input matrix construction.
[0061] Specifically, in the model data preparation module, the training data is intercepted, and the data from 0.5 seconds to 1.5 seconds in each data segment are intercepted to obtain 12,000 data with a length of 1 second, and the size of the two-dimensional data matrix is 500*6; then, for a total of 20 repeated data of each subject, the data of the 3rd, 5th, 7th, 13th, 15th, and 17th repetitions are selected as test sets, and the rest are training sets, which are used to train and test the model respectively; then, the training set is normalized using the standard deviation, and the test set is normalized in the same way using the mean and variance of the training set, so that the test set meets the data probability distribution of the training set, eliminates the dimensional differences of each channel of the electromyographic data, and makes the network convergence more stable during training; finally, the multi-channel electromyographic signal with a size of 500*6 is increased by one dimension and converted into a three-dimensional electromyographic matrix of 1*500*6 to meet the needs of two-dimensional convolution.
[0062] The model building module mainly constructs a dual-stream feature fusion convolutional neural network.
[0063] Specifically, the dual-stream feature fusion convolutional neural network is used to extract features and classify the input three-dimensional electromyographic matrix. It includes an input module, a convolutional neural network pathway (2), a feature fusion convolutional module (2), a fully connected module (2), and a classification decision module. Its structure is as follows: Figure 3 shown.
[0064] In this embodiment, fully connected refers to a fully connected network, which is equivalent to a convolutional neural network, and is the name of a network model (algorithm).
[0065] Exemplarily, the function of the input module is to divide the input three-dimensional electromyographic matrix of 1*500*6 into two parts of the same size according to the channel dimension, that is, two input three-dimensional electromyographic matrices of size 1*500*3.
[0066] Exemplarily, the structures of the two convolutional neural network pathways are exactly the same, and they are used to extract features from the two input three-dimensional electromyographic matrices. Each convolutional neural network pathway is composed of a shallow feature extraction module and a deep feature extraction module in sequence. The shallow feature extraction module is composed of a convolution layer c1, a batch normalization layer b1, a nonlinear activation layer f1, a convolution layer c2, a batch normalization layer b2, a nonlinear activation layer f2, and a pooling layer p1 in sequence. The number of convolution kernels in the convolution layer c1 is 8, the size is set to 5*1, the step length is 1, and the padding is 0. The number of convolution kernels in the convolution layer c2 is 16, the size is set to 5*1, the step length is 1, and the padding is 0. Finally, the extracted shallow feature maps x1 and x are output through the pooling layer p1 with a size of 2*1 and a step length of 2*1. 2; The deep feature extraction module is composed of convolution layer c3, batch normalization layer b3, non-linear activation layer f3, convolution layer c4, batch normalization layer b4, non-linear activation layer f4, convolution layer c5, batch normalization layer b5, and non-linear activation layer f5. The number of convolution kernels in convolution layer c3 is 24, the size is set to 3*1, the step size is 1, and the padding is 0. The number of convolution kernels in convolution layer c4 is 32, the size is set to 3*1, the step size is 1, and the padding is 0. The number of convolution kernels in convolution layer c5 is 32, the size is set to 1*3, the step size is 1, and the padding is (0,1). Finally, the deep feature maps y1 and y2 are obtained.
[0067] Exemplarily, the structures of the two feature fusion convolution modules are exactly the same, one of which is used to perform feature fusion on the shallow feature maps x1 and x2, and the other is used to perform feature fusion on the deep feature maps y1 and y2. Each feature fusion convolution module first adds the two input feature maps as the input feature map, and then is sequentially composed of convolution layer c6, batch normalization layer b6, non-linear activation layer f6, pooling layer p2, convolution layer c7, batch normalization layer b7, and non-linear activation layer f7. The number of convolution kernels in convolution layer c6 is 32, the size is set to 5*1, the step size is 1, and the padding is 0. The size of pooling layer p2 is 2*1, the step size is 2*1, and the number of convolution kernels in convolution layer c7 is 32, the size is set to 1*3, the step size is 1, and the padding is (0,1). The shallow feature map and the deep feature map are fused respectively through two feature fusion convolution modules to obtain the fused feature maps x and y. By introducing multi-stream fusion, this module has better fitting performance for training data, stronger model robustness and higher prediction accuracy compared with the traditional single-stream network.
[0068] For example, batch normalization layers in the convolutional neural network pathway and feature fusion convolution module all use Batch Normalization to speed up network convergence; nonlinear activation layers all use LeakyRelu activation function to enhance the nonlinear expression ability of the network; pooling layers all use average pooling, which is more in line with the characteristics of electromyographic signals than maximum pooling. At the same time, both modules use convolution kernels of size k*1 to effectively extract the time features within each channel in the time direction and fully mine the internal information of the channel, which saves parameters compared to using convolution kernels with the same length and width, and reduces the burden of model training and deployment. At the same time, convolution kernels of size 1*3 are also used to effectively capture the spatial features across channels and further mine the correlation between channels.
[0069] In this embodiment, Batch Normalization and LeakyRelu are both existing mature and widely used technologies.
[0070] Exemplarily, the structures of the two fully connected modules are exactly the same. First, the input feature maps x and y are flattened into one-dimensional feature vectors as input. The flattening here only means straightening the three-dimensional data into one dimension and using it as the input data of the fully connected module. Then, it is composed of a linear layer l1, a nonlinear activation layer f8, and a linear layer l2 in sequence, where the linear layer l1 contains 64 neurons and the linear layer l2 contains 32 neurons. Finally, two one-dimensional feature vectors with a length of 32 are output, representing the shallow features and deep features finally obtained, respectively.
[0071] Exemplarily, the classification decision module concatenates shallow features and deep features into a one-dimensional feature vector of length 64, and then feeds it into the am-softmax layer, and finally outputs a one-dimensional vector of length 30, the number of gesture types, which is the classification result.
[0072] The training and testing module mainly uses the collected gesture electromyography data to train and test the model, optimize the model, and evaluate the gesture prediction performance of the model.
[0073] Specifically, in the test and training module, the data of 20 subjects were trained and tested separately, and each subject was trained for 50 iterations. In each iteration, the training set was divided into 3 batches, and each batch was input into the network in turn to obtain the classification result, and compared with the label, and then back-propagation was performed to update the trainable parameters in the network, and the classification accuracy of the test set was calculated in each iteration. The highest test set accuracy obtained at the end is the model gesture prediction accuracy of the subject.
[0074] The model prediction module collects the electromyographic data of the gesture action in real time after the model is tuned, analyzes and predicts the gesture action based on the electromyographic data of the execution period, and outputs the prediction results.
[0075] Specifically, in the model prediction module, the electromyographic signals of the subjects are collected in real time, analyzed, and the gestures are predicted using the trained model, and the gestures are converted into driving signals for the electric stimulator.
[0076] The electrical stimulation auxiliary module mainly drives the electrical stimulator to output stimulation current based on the output prediction results to assist in completing the execution of gesture actions.
[0077] Specifically, in the electrical stimulation auxiliary module, gesture actions 00-29 correspond to electrical stimulation modes 00'-29' respectively. The stimulation modes are set in advance. The stimulation current is given by a four-way electrical stimulator. The stimulation electrodes are located at the superficial flexor digitorum, brachioradialis, palmaris longus, intrinsic extensor of the index finger, extensor digitorum, and intrinsic extensor of the thumb.
[0078] In practical applications, the gesture prediction system is used in conjunction with the existing gesture recognition method. First, the system is used to predict the gesture in advance based on the surface electromyography signals during the gesture execution period, and then the existing gesture recognition method is used to perform normalized gesture recognition during the gesture holding period. This outputs control instructions faster than using the existing gesture recognition method alone, thereby improving control efficiency.
[0079] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.
[0080] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the present invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.
Claims
1. The electromyographic signal gesture prediction system based on dual-stream feature fusion convolutional neural network is characterized by: It includes gesture action acquisition module, electromyographic signal preprocessing module, model data preparation module, model construction module, training and testing module, model prediction module, and electrical stimulation auxiliary module; The gesture action acquisition module is used to collect electromyographic signals during the execution of gesture actions; The electromyographic signal preprocessing module is used to filter, trap and reduce noise of the electromyographic signal, and label the training data; The model data preparation module is used to intercept data, normalize data and construct input matrix; The model building module is used to build a dual-stream feature fusion convolutional neural network; The training and testing module trains and tests the model through the collected gesture action electromyography data; The model prediction module collects the electromyographic data of the gesture action in real time, analyzes and predicts the gesture action according to the electromyographic data of the execution period, and outputs the prediction result; The electrical stimulation auxiliary module drives the electrical stimulator to output stimulation current according to the output prediction result, so as to assist in completing the execution of the gesture action; The data interception is used to intercept the data in each data segment; The data normalization is used to normalize the training set and the test set in the data interception; The input matrix construction refers to converting the electromyographic signal into a three-dimensional electromyographic matrix as the input of the model; The dual-stream feature fusion convolutional neural network is used to extract features and perform classification prediction on the input three-dimensional electromyographic matrix; The dual-stream feature fusion convolutional neural network includes an input module, a convolutional neural network pathway, a feature fusion convolution module, a fully connected module and a classification decision module; The input module is used to divide the input three-dimensional electromyographic matrix into two parts of the same size according to the channel dimension; There are two convolutional neural network pathways, which are used to extract features from two parts of the same size divided by the input module respectively; There are two feature fusion convolution modules, one of which is used to perform feature fusion on shallow feature maps x1 and x2, and the other is used to perform feature fusion on deep feature maps y1 and y2; The two fully connected modules are used to further extract features from the feature graphs x and y to obtain two one-dimensional feature vectors; The classification decision module concatenates shallow features and deep features into a one-dimensional feature vector, which is then sent to the am-softmax layer, and finally outputs a one-dimensional vector, which is the classification result.
2. The electromyographic signal gesture prediction system based on dual-stream feature fusion convolutional neural network according to claim 1 is characterized in that: The gesture action acquisition module collects the upper limb surface electromyographic signals of each subject when performing different gestures through patch electrodes; The upper limb surface electromyography signal includes execution period data and holding period data; The execution period data comes from the dynamic process of the gesture action from the beginning of execution to the formation of the gesture; The holding period data comes from the holding stage after the gesture is formed.
3. The electromyographic signal gesture prediction system based on dual-stream feature fusion convolutional neural network according to claim 1 is characterized in that: In the gesture action acquisition module, the patch electrodes are located at the superficial flexor digitorum, the brachioradialis, the palmaris longus, the intrinsic extensor muscle of the index finger, the extensor digitorum, and the intrinsic extensor muscle of the thumb.
4. The electromyographic signal gesture prediction system based on dual-stream feature fusion convolutional neural network according to claim 1 is characterized in that: The filtering of the electromyographic signal is used to perform Fourier transform on the signal, and a bandpass filter is used; The notch of the electromyographic signal is used to pass through a 50 Hz notch filter to eliminate the power frequency interference of the power line; The noise reduction of the electromyographic signal is used to decompose various frequency components in the signal into non-overlapping frequency bands by using wavelet decomposition based on the sym mother wave, detect the components of the mutation signal in the normal signal, and remove the stubborn noise.
5. The electromyographic signal gesture prediction system based on dual-stream feature fusion convolutional neural network according to claim 1 is characterized in that: The batch normalization layers in the convolutional neural network pathway and the feature fusion convolution module all use BatchNormalization to accelerate network convergence; the nonlinear activation layers all use LeakyRelu activation function to enhance the nonlinear expression ability of the network.
6. The electromyographic signal gesture prediction system based on dual-stream feature fusion convolutional neural network according to claim 5 is characterized in that: The pooling layers all use average pooling, which is more consistent with the characteristics of electromyographic signals than maximum pooling.
7. The electromyographic signal gesture prediction system based on dual-stream feature fusion convolutional neural network according to claim 1 is characterized in that: In the model prediction module, the electromyographic signals of the subject are collected in real time, the electromyographic signals are analyzed, and the hand gestures are predicted using the trained model, and the hand gestures are converted into driving signals for the electric stimulator.
8. The electromyographic signal gesture prediction system based on dual-stream feature fusion convolutional neural network according to claim 1 is characterized in that: In the electrical stimulation auxiliary module, the stimulation current is provided by a four-way electrical stimulator, and the stimulation electrodes are respectively located at the flexor digitorum superficialis, brachioradialis, palmaris longus, intrinsic extensor of the index finger, extensor digitorum, and intrinsic extensor of the thumb.
Citation Information
Patent Citations
Multi-stream divide-and-conquer convolutional neural network-based surface electromyogram (EMG) signal gesture recognizing method
CN108491077A
Electromyographic signal gesture recognition method based on double-flow network
CN110658915A
Crowd distribution prediction method and system based on double-flow convolutional neural network
CN112767451A
Gesture recognition method based on double-flow neural network
CN113191408A
Exoskeleton performance test method, device and system
CN113442174A
