A surface electromyography signal classification method based on double-branch network
Through the dual-branch network combined with CNN and GRU network parallel training, the problem of ignoring correlation and timing characteristics during feature extraction in electromyography signal recognition is solved, which improves the accuracy and stability of electromyography signal recognition, and enhances the robustness of the model sex.
Patent Information
- Application Number
- CN202210540067.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-17
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-05-17
AI Technical Summary
The prior art has the problem of poor classification accuracy when extracting the feature in electromyography signal recognition.
The dual-branch network method is adopted, combined with parallel training of CNN and GRU networks, features are extracted through window analysis, and special two-dimensional processing is performed, and spatial and temporal features are fused using the full connection layer, and finally classified through the softmax layer.
It improves the accuracy and stability of electromyography signal recognition, enhances the robustness of the model, reduces training time, and can better extract the spatiotemporal characteristics of electromyography signals.
Smart Images

Figure CN115137375B_ABST
Abstract
Description
Technical Field
[0001] The invention is a method for identifying and classifying electromyographic signals on the surface of a human body. Background Art
[0002] With the continuous progress of society and the continuous development of modern science and technology, people with hand disabilities have a great desire for normal hand grasping, and bionic prostheses driven by artificial intelligence, machine perception, intelligent control and rehabilitation medicine technology have emerged. The control of the signal source is required for the disabled to control the prosthesis. The human body is full of various signals, which control and represent the various activities and intentions of the human body. The human body has a variety of bioelectric signals, which contain rich information reflecting people's behavioral intentions. The bioelectric signals that are currently widely concerned include electromyographic signals (EMG), electrocardiographic signals (ECG), electroencephalographic signals (EEG) and electrooculographic signals (EOG).
[0003] The electromyographic signal represents the sum of subcutaneous motor action potentials generated by muscle contraction. Among many muscle fibers, it represents the superposition of motor unit action potentials (MUAP) in space and time. At present, the main control signal of bionic prostheses comes from surface electromyographic signals (sEMG). Surface electromyographic signals are the comprehensive effect of electrical activity on superficial muscles and neurons on the surface of human skin. They can represent the hand movements and muscle movements performed by the human body at the moment, and to a certain extent reflect the intention of human movements. Surface electromyographic signals contain rich biological information, and the signal collection is usually non-invasive and does not cause any adverse effects on the human body. They are often used in clinical medicine to diagnose neurological functional diseases. In the fields of rehabilitation medicine and clinical medicine, they are often used as control signal sources for bionic prostheses and rehabilitation equipment. Therefore, they have great medical application value.
[0004] However, electromyographic signals are a relatively weak bioelectric signal. Effectively extracting electromyographic signal features and realizing accurate classification and recognition of sEMG is a hot topic in the current electromyographic research field. Traditional feature extraction methods vary in form, but most of them use one or more statistical values to represent the information implied by sEMG, which may often lose deeper feature information. Secondly, due to the influence of many environmental factors such as individual differences among subjects and electrode offset, traditional classification methods such as Bayesian classification, linear discriminant analysis, random forest, etc., will also be greatly affected in their classification accuracy. In recent years, with the continuous development of artificial intelligence, electromyographic recognition research based on deep learning has begun to rise, and some scholars have put electromyographic signals into deep learning models for training and classification, such as convolutional neural networks. However, the existing EMG signal recognition models based on deep learning still have the following problems: (1) Most of the original EMG signals are extracted to obtain one-dimensional discrete features. However, there are often certain connections between each discrete feature. If the one-dimensional discrete features are directly convolved, the correlation between different discrete features is often easily ignored, and the relative relationship between the features cannot be represented; (2) The EMG signal is a continuous time series signal. A simple convolutional neural network cannot fully consider its correlation in the time series and has weak feature processing capabilities for time series changes; (3) Discrete features are extracted from the original signal, which represents the various characteristics of the EMG signal, but cannot completely restore all the information of the original signal. Some key information of the original signal may be ignored, resulting in poor model classification results. Summary of the invention
[0005] In view of the above problems existing in the prior art, the present invention provides a surface electromyography signal classification method based on a dual-branch network, which adopts CNN network and GRU network for parallel training, and combines the obtained results into a classifier to obtain the final result. This method combines the advantages of both models, overcomes the shortcomings of the existing methods, and further improves the accuracy, stability and robustness of electromyography recognition.
[0006] A surface electromyography signal classification method based on a double-branch network comprises the following steps:
[0007] S1: Use window analysis method to preprocess the original signal data. For the original surface electromyography signal, use a window of length w to intercept it. Where w is the window length and t is the sliding step. After every time t, the data of 16 channels are extracted at the same time.
[0008] S2: Use machine learning methods to extract features from each channel of the electromyographic signal to obtain a one-dimensional discrete feature x. Then use a special two-dimensional method to square the m discrete feature information in the original feature vector x to obtain x. 2 , and then the eigenvector x and x 2Cartesian product operation is performed to obtain a two-dimensional composite matrix F.
[0009] S3: Put the obtained two-dimensional feature matrix into the CNN network, perform multiple convolutions and pooling, extract the spatial feature information, and obtain the spatial feature map Map space .
[0010] S4: Normalize the original electromyographic signal so that the original data is scaled to the same data range after the feature is two-dimensionalized, reducing the impact of data distribution differences on the model.
[0011] S5: Put the processed information into the GRU network for training, extract the temporal feature information, and obtain the temporal feature map Map time .
[0012] S6: Use a fully connected layer to fuse the parameter maps of the two network trainings space and Map time , and finally classified through the softmax layer.
[0013] Preferably, in step S1, w is 300 and t is 50.
[0014] The network model proposed by the method of the present invention uses a convolutional neural network and a GRU network to simultaneously extract the spatial features and temporal features of the surface electromyographic signal. The original surface electromyographic signal recorded by the electrode is processed by a window analysis method, and a machine learning method is used to extract discrete features, and a two-dimensional synthetic feature is generated by a special two-dimensional method, and then the two-dimensional feature map is put into a convolutional neural network for training; the original one-dimensional electromyographic signal is normalized and preprocessed, and put into a recurrent neural network for training; a fully connected layer is used to fuse the features obtained by the two network trainings, and finally classified by a softmax layer. The present invention reduces the training time of the model, can better extract the spatial and temporal features of the electromyographic signal, improves the accuracy and stability of electromyographic signal recognition, and is of great significance to improving the accuracy of prosthetic movement completion.
[0015] Compared with the prior art, the present invention has the following advantages:
[0016] (1) Solve the problem of sparse feature information and excessive parameter redundancy after discrete features are converted to two-dimensional features. This method is based on the theory of feature combination. It converts one-dimensional signals into two-dimensional or even higher-dimensional regular data for deep learning through feature combination. At the same time, it also improves the problem of repeated parameter redundancy that may be caused by the original two-dimensional conversion method. The objects represented by the obtained high-dimensional features have stronger correlation.
[0017] (2) Improved recognition accuracy. The dual-branch network proposed in this invention, CNN, has local connections and shared weights, and focuses more on extracting the spatial features of the input data. GRU can learn the temporal features of the feature sequence, and then fuse the outputs of the two through the fully connected layer. The network can learn the spatiotemporal features more effectively, which is conducive to improving the recognition accuracy of gestures.
[0018] (3) The model is more robust and the training time is shorter. The GRU network uses the original signal as input and retains the correlation and time dependence of the original sEMG time series, so that the original model can highlight certain local information while mining the key details in the original signal that are ignored by the convolution operation. At the same time, the GRU structure is simpler, using only update gates and reset gates, and the number of training parameters is reduced, which can effectively reduce the training time and improve the calculation rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 The figure is an overall flow chart of the method of the present invention.
[0020] Figure 2 It is a schematic diagram of the window analysis method of the present invention.
[0021] Figure 3 This is a network model diagram of the present invention.
[0022] Figure 4 This is the forward propagation diagram of the GRU network. DETAILED DESCRIPTION
[0023] The present invention is described in detail below with reference to examples and formulas.
[0024] Example: The equipment used for data collection is the ELONXI electromyograph developed by the team of the University of Portsmouth in the UK. The electrode set is equipped with 18 electrodes, 16 of which are used to collect surface electromyographic signals. This experiment uses 16 channels for sampling, the sampling frequency is 1k Hz, and the filtering operation is performed at 50 Hz. 8 healthy volunteers (6 males, 2 females, age: 25±5 years old, height: 175±10cm, weight: 65±10kg) participated in the 3-day data collection. Each person demonstrated 5 gestures in each time period, and the middle 10 seconds of each gesture were recorded. The experiment used the electromyographic data of 7 people as the training set, and the signal collected by one person as the test set.
[0025] S1: Use window analysis method to preprocess the original signal data. Figure 2 As shown in FIG. 1 , the collected original surface electromyographic signal is intercepted using a window of 300 milliseconds in length, and the data of different channels are intercepted at the same time every 50 milliseconds.
[0026] S2: Extract features from each channel of the electromyographic signal according to S21 to obtain one-dimensional discrete features, and perform special two-dimensionalization on them according to S22 to S23 to obtain a two-dimensional synthetic matrix F.
[0027] S21: Use machine learning methods to extract features from each channel of the electromyographic signal to obtain a one-dimensional discrete feature x. In this example, m features are collected, and the value of m is 15, where x i represents the i-th feature.
[0028] x=[x1,x2,...,x i ,...,x m ] (1)
[0029] S22: Square the m discrete feature information in the original feature vector x to avoid excessive redundant information and repeated input of two-dimensional matrix data.
[0030]
[0031] S23: Combine the feature vector x and x 2 Cartesian product operation is performed to obtain a two-dimensional composite matrix F.
[0032] F=D(x,x 2 ,α,β)=G(α·(x T × 2 ) β ) (3)
[0033]
[0034]
[0035] Among them, α and β are conversion parameters, each with a value of 0.5; × is the Cartesian product operator symbol; G is the nonlinear activation function sigmoid, which maps the variable to the interval [0,1] and introduces nonlinearity.
[0036] S3: Put the obtained two-dimensional feature matrix into the CNN network, perform multiple convolutions and pooling according to S31 to S34 operations, extract spatial feature information, and obtain the spatial feature map Map space .
[0037] S31: Put the two-dimensional feature matrix F into the CNN network, and perform convolution with a convolution kernel of size 5×5×64 and a step size of 1 to obtain a feature map Map1 of size 15×15×64.
[0038] Map1=Conv 5×5×64 (F) (6)
[0039] S32: Use a convolution kernel of size 2×2 and stride 2 to perform a maximum pooling operation on the feature map Map1 to obtain a feature map Map2 of size 8×8×64.
[0040] Map2=Maxpool 2×2 (Map1) (7)
[0041] S33: Convolve the feature map Map2 with a convolution kernel of 3×3×128 size and a step size of 1 to obtain the feature map Map3, and perform maximum pooling with a convolution kernel of 2×2 size and a step size of 2 to obtain the feature map Map4 of size 4×4×128.
[0042] Map3=Conv 3×3×128 (Map2) (8)
[0043] Map4=Maxpool 2×2 (Map3) (9)
[0044] S34: Finally, in order to extract the most effective spatial features, two fully connected layers containing 512 units and 128 units are performed to obtain the spatial feature map Map space .
[0045] Map space =Soft max(Map4) (10)
[0046] S4: Normalize the original electromyographic signal to obtain the sequence y. The specific formula is as follows:
[0047]
[0048]
[0049]
[0050] Where E(x) represents the mean of the EMG signal sequence, Var(x) represents the variance of the EMG signal sequence, and ε is the bias value.
[0051] S5: Put the processed 16×300 one-dimensional sequence y into the GRU network for training, extract the temporal feature information, and obtain the feature map Map5.
[0052] Map5=GRU(y) (14)
[0053] The specific parameter settings of the GRU network are: the number of input features is 300, the number of hidden layer nodes is 128, and the number of network layers is 2. The training of the GRU network mainly includes two gate units: update gate and reset gate. The update gate controls how much useful information the current state needs to retain from the historical state and how much new information needs to be accepted from the candidate state. The larger the value of the update gate, the more state information from the previous moment is brought in. The information from the previous moment and the current moment is right-multiplied by the weight matrix W hz and W xz , and then the added data is put into the update gate and then into the sigmoid function.
[0054] z t =σ(x t W xz +h t-1 W hz +b z ) (15)
[0055] The reset gate is used to control the candidate state h t Whether it depends on the state h at the previous moment t-1 The larger the reset gate is, the more previous state information is written. The information of the previous moment and the current moment are right-multiplied by the weight matrix W hr and W xr , and then the added data is put into the reset gate and then into the sigmoid function.
[0056] r t =σ(x t W xr +h t-1 W hr +b r ) (16)
[0057] When the gate signal is received, the state value of the candidate hidden layer unit is updated. The specific operation is as follows:
[0058]
[0059] The final hidden state update method of the network node is:
[0060]
[0061] Where σ is the sigmoid function, which transforms the data into a value in the range of 0-1 and acts as a gating signal. t-1 is the previous information, r t is the reset gate, tanh is the activation function, b z , b r , b h , W hz , W xz, W hr , W xr , W xh are the gate neuron parameters, learned during training, is element-wise multiplication.
[0062] The input data is processed by the GRU network to obtain a 16×128 output, and then two fully connected layers containing 512 units and 128 units are performed to obtain the target's temporal feature map Map time .
[0063] Map time =Soft max(Map5) (19)
[0064] S6: Use Concatenate to fuse the parameters of the two network trainings, and finally classify through the Softmax layer to obtain the final classification result.
[0065] Class=Soft max(Concatenate(Map space ,Map time )) (20)
[0066] The present invention proposes a surface electromyography signal classification method based on a dual-branch network to solve the surface electromyography signal classification problem. The problem that low-dimensional features are difficult to convolve and dig deeper correlations between features is solved through a special two-dimensional method. At the same time, the convolutional neural network and the GRU network are trained in parallel to better extract the spatial and temporal features of the electromyography signal, so that the accuracy of electromyography recognition has been further improved in terms of accuracy, stability and robustness.
[0067] Although the specific implementation process of the present invention is described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific implementation schemes. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A surface electromyography signal classification method based on a double-branch network, comprising the following steps: S1: Preprocess the original signal data using the window analysis method; intercept the collected original surface electromyography signal using a window of preset length; and intercept the data of different channels at the same time at every interval; S2: Extract features from each channel of the electromyographic signal to obtain a one-dimensional discrete feature, and then perform a special two-dimensional transformation to obtain a two-dimensional synthetic matrix F; specifically, it includes: S21: Use machine learning methods to extract features from each channel of the electromyographic signal to obtain a one-dimensional discrete feature x. In this example, m features are collected, and the value of m is 15, where x i represents the i-th feature; x=[x1,x2,…,x i ,…,x m ] (1) S22: square the m discrete feature information in the original feature vector x to avoid excessive redundant information and repeated input of two-dimensional matrix data; S23: Combine the feature vector x and x 2 Cartesian product operation is performed to obtain a two-dimensional composite matrix F; F=D(x,x 2 ,α,β)=G(α·(x T ×x 2 ) β ) (3) Among them, α and β are conversion parameters, each with a value of 0.5; × is the Cartesian product operator symbol; G is the nonlinear activation function sigmoid, which maps the variable to the interval [0,1] and introduces nonlinearity; S3: Put the obtained two-dimensional feature matrix into the CNN network, perform multiple convolutions and pooling, extract spatial feature information, and obtain the spatial feature map Map space ; Specifically include: S31: Put the two-dimensional feature matrix F into the CNN network, and perform convolution with a convolution kernel of size 5×5×64 and a step size of 1 to obtain a feature map Map1 of size 15×15×64; Map1=Conv 5×5×64 (F) (6) S32: Use a convolution kernel of size 2×2 and stride 2 to perform a maximum pooling operation on the feature map Map1 to obtain a feature map Map2 of size 8×8×64; Map2=Maxpool 2×2 (Map1) (7) S33: Convolve the feature map Map2 with a convolution kernel of 3×3×128 size and a step size of 1 to obtain the feature map Map3, and perform maximum pooling with a convolution kernel of 2×2 size and a step size of 2 to obtain the feature map Map4 of 4×4×128 size; Map3=Conv 3×3×128 (Map2) (8) Map4=Maxpool 2×2 (Map3) (9) S34: Finally, in order to extract the most effective spatial features, two fully connected layers containing 512 units and 128 units are performed to obtain the spatial feature map Map space ; Map space =Softmax(Map4) (10) S4: Normalize the original electromyographic signal to obtain the sequence y. The specific formula is as follows: Where E(x) represents the mean of the EMG signal sequence, Var(x) represents the variance of the EMG signal sequence, and ε is the bias value; S5: put the processed 16×300 one-dimensional sequence y into the GRU network for training, extract the temporal feature information, and obtain the feature map Map5; Map5=GRU(y) (14) The specific parameter settings of the GRU network are: the number of input features is 300, the number of hidden layer nodes is 128, and the number of network layers is 2; the training of the GRU network mainly includes two gate units: update gate and reset gate; the update gate controls how much useful information the current state needs to retain from the historical state and how much new information needs to be accepted from the candidate state. The larger the value of the update gate, the more state information of the previous moment is brought in. The information of the previous moment and the current moment are right-multiplied by the weight matrix W hz and W xz , then the added data is put into the update gate and then into the sigmoid function; z t =σ(x t W xz +h t-1 W hz +b z ) (15) The reset gate is used to control the candidate state h t Whether it depends on the state h at the previous moment t-1 The larger the reset gate is, the more previous state information is written. The information of the previous moment and the current moment are right-multiplied by the weight matrix W hr and W xr , then the added data is put into the reset gate and then into the sigmoid function; r t =σ(x t W xr +h t-1 W hr +b r ) (16) When the gate signal is received, the state value of the candidate hidden layer unit is updated. The specific operation is as follows: The final hidden state update method of the network node is: Where σ is the sigmoid function, which transforms the data into a value in the range of 0-1 and acts as a gating signal. t-1 is the previous information, r t is the reset gate, tanh is the activation function, b z 、b r 、b h , W hz , W xz , W hr , W xr , W xh are the gate neuron parameters, learned during training, is element-wise multiplication; The input data is processed by the GRU network to obtain a 16×128 output, and then two fully connected layers containing 512 units and 128 units are performed to obtain the target's temporal feature map Map time ; Map time =Softmax(Map5) (19) S6: Use Concatenate to fuse the parameters of the two network trainings, and finally perform classification through the Softmax layer to obtain the final classification result; Class=Softmax(Concatenate(Map space ,Map time )) (20)。 2. A surface electromyography signal classification method based on a double-branch network as claimed in claim 1, characterized in that: The preset length of the window in step S1 is 300 milliseconds, and the interval length is 50 milliseconds.
