A sEMG data enhancement method based on BiLSTM and WGAN-GP network

Abstract

A sEMG data enhancement method based on BiLSTM and WGAN‑GP network, the specific steps are as follows: step S1, collecting surface electromyographic signals and performing preprocessing; step S2, standardizing the real electromyographic data after preprocessing, and dividing it into a training data set and a test data set; step S3, establishing a WGAN‑GP network model based on the BiLSTM network and inputting the training data set for training, and outputting a generated sample set; step S4, comparing the generated sample set with the test data set for model error analysis, and judging the stability of the model. The present invention generates surface electromyographic signals (sEMG) based on the neural network framework of WGAN‑GP in combination with BiLSTM, which improves the stability of the model training process on the one hand, and improves the quality of the generated samples on the other hand.

Description

Technical Field

[0001] The present invention belongs to the field of computer and information science and technology, and specifically relates to a sEMG data enhancement method based on BiLSTM and WGAN-GP networks. Background Art

[0002] Surface electromyography is a technology that measures the electrical activity of muscles stimulated by nerves. Due to its non-invasiveness and low cost, it has good prospects in the fields of motion classification and prosthetic control. In recent years, the classification accuracy of biosignal classification using deep learning models has increased dramatically, but the development of deep learning is limited by the amount of training data. Surface electromyography itself is a relatively weak electrical signal, and the acquisition process is very time-consuming and susceptible to noise interference. Therefore, it is impossible to collect enough sEMG data to build a large database to fully train gesture recognition models. In addition, real data is strictly regulated, and sharing data will cause the privacy of volunteers to be leaked. Therefore, data expansion based on a small amount of sEMG and applied to deep learning research is of great significance for the recognition and analysis of biosignals.

[0003] Generative adversarial networks have been applied to various fields (images, audio, etc.) as a mainstream neural network framework for small sample processing. The original GAN ​​network has problems such as gradient dissipation and mode collapse. The WGAN-GP network introduces the Wasserstein distance to measure the similarity of two probability distributions, and proposes a truncation pruning strategy for the gradient norm of the discriminator, adding a regularization term, namely the gradient penalty.

[0004] Since traditional GANs are mainly built with fully connected layers (FC), and FC cannot capture the complex correlation between the characteristics of time series signals and related attributes, the model is prone to lose time correlation information when processing SEMG. Summary of the invention

[0005] In view of the problems existing in the above background technology introduction, the purpose of the present invention is to provide a sEMG data enhancement method based on BiLSTM and WGAN-GP network, which can improve the stability of the model training process and the quality of generated samples.

[0006] The technical solution adopted by the present invention is:

[0007] A sEMG data enhancement method based on BiLSTM and WGAN-GP network, the specific steps are as follows:

[0008] Step S1, collecting surface electromyography signals and performing preprocessing;

[0009] Step S2, standardizing the preprocessed real electromyographic data and dividing it into a training data set and a test data set;

[0010] Step S3, establishing a WGAN-GP network model based on the BiLSTM network and inputting a training data set for training, and outputting a generated sample set;

[0011] Step S4, compare the generated sample set with the test data set to perform model error analysis and determine the stability of the model.

[0012] Further, the preprocessing of the surface electromyography signal in step S1 includes:

[0013] Step S11, performing denoising processing;

[0014] Step S12, sliding the window to segment the data and perform feature extraction.

[0015] Further, the feature extraction in step S12 includes average absolute value, slope change times, waveform length, and zero crossing times, where

[0016] (1) Mean absolute value MAV:

[0017]

[0018] Where N represents the length of the sEMG signal, k = 1, 2, 3, ..., N, x k is the amplitude of the kth sEMG;

[0019] (2) Wavelength WL:

[0020]

[0021] (3) Zero crossing times ZC:

[0022] To introduce a threshold a, for two consecutive sEMG signals x k and x k+1 , the number of zero crossings is:

[0023]

[0024] (4) Slope change times SSC:

[0025] Given three continuous sEMG signals x k-1 , x k , x k+1 , if formula (4) holds true, the number of slope changes SSC increases by one;

[0026] (x k -x k-1 )×(x k -xk+1 )≥a (4).

[0027] Further, the specific steps of the standardization in step S2 are as follows:

[0028] The real EMG data after feature extraction is standardized, and the MinMaxScale function in the sklearn library is used to standardize the real EMG data samples to [0,1].

[0029]

[0030] X scaled represents the normalized result, X represents the sample value at that point, and X max represents the maximum sample value, X min Indicates the minimum sample value, (max-min) indicates the scaling range, and min indicates the minimum scaling value.

[0031] Furthermore, the WGAN-GP network model in step S3 includes a generator network and a discriminator network, wherein the generator network is composed of 1 Bidirectional LSTM layer, 5 conv1d layers, and 2 up_sampling layers, and each conv1d layer is activated by adding an activation function LeackyReLu; the discriminator network is composed of 1 conv1d layer and 2 max_pooling layers, and each conv1d layer is activated by adding an activation function LeackyReLu.

[0032] Further, the model training steps in step S3 are as follows:

[0033] The generated sample data output by the generator network and the standardized training data set are input into the discriminator network of the WGAN-GP network model, and the probability of each sample being judged as a real sample is output; the objective function of WGAN-GP is set to:

[0034]

[0035] Where E(·) represents the expectation, and the real data sample x r The distribution of p r (x), fake data sample x output by the generator network g The distribution of p g (x), ||·||2 is the 2-norm, ▽ represents the gradient; λ represents the coefficient of the gradient constraint, through Calculate the real data sample x r and fake data sample x g The sampling value of the random difference on the connection line ε is a random number between (0, 1);

[0036] Calculate the loss value of the generator network in the WGAN-GP network model at the current iteration, calculate the loss value of the discriminator network in the WGAN-GP network model at the current iteration, and use the stochastic gradient descent method to update the parameters of the discriminator network and the generator network in the WGAN-GP network model in sequence with the loss value of the discriminator network and the loss value of the generator network in the WGAN-GP network at the current iteration;

[0037] Change the training parameters of WGAN-GP to determine whether the loss values ​​of the discriminator network and the generator network in the WGAN-GP network model tend to be stable and low. In this case, the generated samples are of high quality.

[0038] Furthermore, the step of generating a sample set in step S3 includes:

[0039] M noise samples are randomly generated from a normal distribution to form a noise sample set. The M noise samples are input into the WGAN-GP network model, and the generated sample data are output, where M and P are equal; the generated sample data are inversely standardized and output to obtain the generated sample set.

[0040] Further, the specific steps of the model error analysis in step S4 are as follows:

[0041] Compare and analyze the generated sample set of the WGAN-GP network model with the test data set to measure the quality of the model;

[0042] Root mean square error RMSE:

[0043]

[0044] Symmetric mean absolute percentage error SMAPE:

[0045]

[0046] Where y is the true value, that is, the generated sample set, For the predicted value, that is, the test data set, when RMSE < 0.2 and SMAPE < 100%, the model is stable and the synthesized sEMG has a high quality.

[0047] Furthermore, the method further includes step S5, which verifies the effectiveness and diversity of the model in enhancing electromyographic data by qualitative or quantitative methods.

[0048] Furthermore, in step S5, the qualitative verification is performed by principal component analysis (PCA) to measure the distribution of the generated sample set and the distribution of the real data sample;

[0049] Quantitative verification is to introduce different classification algorithms to compare classifiers, and obtain the accuracy under different training sets: real data, synthetic data, real data and synthetic data, and the same test set conditions. The training set is real data and the test set is real data. The obtained classifier accuracy rate of 1 is used as the experimental baseline, and compared with the classifier accuracy rates of 2 and 3 obtained when the training set is synthetic data and mixed data, and the test set is real data, respectively.

[0050] Compared with the prior art, the present invention has the following significant advantages: improving the stability of the model training process and improving the quality of generated samples; adding the synthesized sEMG signal to the training database to verify the effectiveness and diversity of the present invention in enhancing the sEMG signal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the process structure of the present invention.

[0052] Figure 2 It is a schematic diagram of the structure of the BiLSTM network of the present invention;

[0053] Figure 3 It is a schematic diagram of the structure of the generator network and the discriminator network of the present invention;

[0054] Figure 4 It is a schematic diagram of the framework structure of the WGAN-GP network model of the present invention;

[0055] Figure 5 It is a comparative schematic diagram of verifying the effectiveness of sEMG data expansion using different classification algorithms of the present invention. DETAILED DESCRIPTION

[0056] The present invention is further described below in conjunction with specific embodiments, but the present invention is not limited to these specific embodiments. Those skilled in the art should recognize that the present invention covers all possible alternatives, improvements and equivalents within the scope of the claims.

[0057] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, the meaning of "multiple" is two or more, unless otherwise clearly defined.

[0058] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0059] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0060] See also Figures 1 to 5 , this embodiment provides a sEMG data enhancement method based on BiLSTM and WGAN-GP network, and its specific steps are as follows:

[0061] Step S1, collecting surface electromyography signals and performing preprocessing;

[0062] The surface electromyography signals are collected to collect electromyography signals of multiple gestures, for example, gesture 1 is fist clenching, gesture 2 is open, and gesture 3 is resting.

[0063] The preprocessing of surface electromyography signals includes:

[0064] Step S11, denoising: Generally speaking, the collected electromyographic signals inevitably contain noise due to the influence of the equipment and the human body. These noises mainly include power frequency interference and electromyographic interference. Low-pass filtering is used for denoising;

[0065] Step S12, the sliding window is used to segment the data and perform feature extraction, including the average absolute value (MAV), the number of slope changes (SSC), the waveform length (WL), and the number of zero crossings (ZC). The extracted feature vectors are used as input samples of the GAN network, for example, 100 MAV feature vectors with a dimension of 128.

[0066] in

[0067] (1) Mean absolute value MAV:

[0068]

[0069] Where N represents the length of the sEMG signal, k = 1, 2, 3, ..., N, x k is the amplitude of the kth sEMG.

[0070] (2) Wavelength WL: The waveform length is the simple accumulation of the signal lengths at N points, which can reflect the complexity of the EMG signal waveform.

[0071]

[0072] (3) Zero-crossing times ZC: The zero-crossing times are used to estimate the frequency domain characteristics by counting the number of times the signal waveform passes through the zero point. In order to reduce the impact of noise on the zero-crossing times, a threshold a needs to be introduced. For two consecutive sEMG signals x k and x k+1 , the number of zero crossings is:

[0073]

[0074] (4) Slope change number SSC: This parameter counts the number of times the slope sign of the signal changes. Again, it requires a threshold a to reduce the interference of the slope sign change number caused by noise. Given three consecutive sEMG signals x k-1 , x k , x k+1 , if formula (4) holds true, the number of slope changes SSC increases by one.

[0075] (xk -x k-1 )×(x k -x k+1 )≥a (4).

[0076] Step S2, standardizing the preprocessed real electromyographic data and dividing it into a training data set and a test data set;

[0077] The specific steps of standardization are as follows:

[0078] The real EMG data after feature extraction is standardized, and the MinMaxScale function in the sklearn library is used to standardize the real EMG data samples to [0,1].

[0079]

[0080] X scaled represents the normalized result, X represents the sample value at that point, and X max represents the maximum sample value, X min Indicates the minimum sample value, (max-min) indicates the scaling range, and min indicates the minimum scaling value.

[0081] Step S3, establishing a WGAN-GP network model based on the BiLSTM network and inputting a training data set for training, and outputting a generated sample set;

[0082] The WGAN-GP network model includes a generator network and a discriminator network. The generator network consists of 1 layer of Bidirectional LSTM, 5 layers of conv1d, and 2 layers of up_sampling. Each conv1d layer is activated by adding the activation function LeackyReLu. The network structure and output dimension are as shown in the attached figure. Figure 3 As shown in the figure, the discriminator network consists of 4 layers of conv1d and 2 layers of max_pooling. Each conv1d layer is activated by adding the activation function LeackyReLu. The network structure and output dimension are shown in the attached figure. Figure 3 shown.

[0083] The generative adversarial network uses WGAN-GP as the basic framework, and the generator network is developed based on BiLSTM to adapt to sEMG sequences. The BiLSTM network structure is as follows Figure 2 shown.

[0084] BiLSTM network: LSTM units handle time series prediction problems, while the BiLSTM network adds another backward LSTM layer on the basis of the LSTM layer to further process backward information and improve the prediction performance of the network.

[0085] BiLSTM network processing: The BiLSTM network uses forward LSTM to read the time series data in normal order, and uses backward LSTM to read the time series data in reverse order;

[0086] BiLSTM network output: The outputs of the forward and backward LSTM layers are concatenated and input into the convolutional layer.

[0087] A sEMG data enhancement method based on the WGAN-GP neural network framework combined with BiLSTM is to construct a gradient-penalized Wasserstein generative adversarial network WGAN-GP, divide the real data samples into a training data set and a test data set, use the random noise sample set as the input of the WGAN-GP network generator, and output the generated sample set, i.e., fake data. The training data set and the generated sample set are used as the input of the discriminator in the WGAN-GP network model.

[0088] The details are as follows:

[0089] Real data samples: The WGAN-GP network is trained for one category of gestures at a time. Each type of gesture is a small sample set consisting of P samples, where P = 100. Each sample in the small sample set consisting of real electromyographic data is standardized, and all processed samples are divided into a training data set and a test data set at a ratio of 8:2.

[0090] The generated sample data output by the generator network and the standardized training data set are input into the discriminator network of the WGAN-GP network model, and the probability of each sample being judged as a real sample is output; the objective function of WGAN-GP is set to:

[0091]

[0092] Where E(·) represents the expectation, and the real data sample x r The distribution of p r (x), fake data sample x output by the generator network g The distribution of p g (x), ||·||2 is the 2-norm, ▽ represents the gradient; λ represents the coefficient of the gradient constraint, preferably λ=10. Calculate the real data sample x r and fake data sample x g The sampling value of the random difference on the connection line ε is a random number between (0, 1);

[0093] Calculate the loss value of the generator network in the WGAN-GP network model at the current iteration, calculate the loss value of the discriminator network in the WGAN-GP network model at the current iteration, and use the stochastic gradient descent method to update the parameters of the discriminator network and the generator network in the WGAN-GP network model in sequence with the loss value of the discriminator network and the loss value of the generator network in the WGAN-GP network at the current iteration;

[0094] Change the training parameters of WGAN-GP, including but not limited to the learning rate, batch size BATCHSIZE, and number of iterations, to determine whether the loss values ​​of the discriminator network and the generator network in the WGAN-GP network model tend to be stable and low. In this case, the generated samples are of high quality.

[0095] The steps of generating a sample set include:

[0096] M noise samples are randomly generated from a normal distribution to form a noise sample set. The M noise samples are input into the WGAN-GP network model, and the generated sample data are output, where M and P are equal; the generated sample data are inversely standardized and output to obtain the generated sample set.

[0097] Step S4, compare the generated sample set with the test data set to perform model error analysis and determine the stability of the model.

[0098] The specific steps of model error analysis are as follows:

[0099] Compare and analyze the generated sample set of the WGAN-GP network model with the test data set to measure the quality of the model;

[0100] Root mean square error RMSE:

[0101]

[0102] Symmetric mean absolute percentage error SMAPE:

[0103]

[0104] Where y is the true value, that is, the generated sample set, For the predicted value, that is, the test data set, when RMSE < 0.2 and SMAPE < 100%, the model is stable and the synthesized sEMG has a high quality.

[0105] Step S5, verifying the effectiveness and diversity of the model in enhancing the electromyographic data by qualitative or quantitative means.

[0106] The qualitative verification is principal component analysis (PCA) to measure the distribution of the generated sample set and the distribution of the real data sample;

[0107] Quantitative verification is to introduce different classification algorithms (classification algorithms include support vector machine (SVM), decision tree (DT), random forest (RF), multilayer perceptron (MLP), and K nearest neighbor algorithm (KNN)) to compare classifiers, and obtain the accuracy under different training sets: real data, synthetic data, real data and synthetic data, and the same test set conditions. The training set is real data and the test set is real data. The obtained classifier accuracy rate of 1 is used as the experimental baseline, and compared with the classifier accuracy rates of 2 and 3 obtained when the training set is synthetic data and mixed data, and the test set is real data, respectively.

[0108] The present invention combines BiLSTM with the neural network framework of WGAN-GP to generate surface electromyography (sEMG), which improves the stability of the model training process on the one hand and the quality of the generated samples on the other hand. The synthesized sEMG signal is added to the training database to verify the effectiveness of the present invention in enhancing the sEMG signal data.

Claims

1. A sEMG data enhancement method based on BiLSTM and WGAN-GP network, the specific steps are as follows: Step S1, collecting surface electromyography signals and performing preprocessing; Step S2, standardizing the preprocessed real electromyographic data and dividing it into a training data set and a test data set; Step S3, establishing a WGAN-GP network model based on a BiLSTM network and inputting a training data set for training, and outputting a generated sample set; wherein the WGAN-GP network model includes a generator network and a discriminator network, wherein the generator network is composed of 1 layer of Bidirectional LSTM, 5 layers of conv1d, and 2 layers of up_sampling, and each conv1d layer is activated by adding an activation function LeackyReLu; the discriminator network is composed of 1 layer of conv1d and 2 layers of max_pooling, and each conv1d layer is activated by adding an activation function LeackyReLu; The model training steps are as follows: The generated sample data output by the generator network and the standardized training data set are input into the discriminator network of the WGAN-GP network model, and the probability of each sample being judged as a real sample is output; the objective function of WGAN-GP is set to: Where E(·) represents the expectation, and the real data sample x r The distribution of p r (x), fake data sample x output by the generator network g The distribution of p g (x), ||·||2 is the 2-norm, represents the gradient; λ represents the coefficient of the gradient constraint, through Calculate the real data sample x r and fake data sample x g The sampling value of the random difference on the connection line ε is a random number between (0, 1); Calculate the loss value of the generator network in the WGAN-GP network model at the current iteration, calculate the loss value of the discriminator network in the WGAN-GP network model at the current iteration, and use the stochastic gradient descent method to update the parameters of the discriminator network and the generator network in the WGAN-GP network model in sequence with the loss value of the discriminator network and the loss value of the generator network in the WGAN-GP network at the current iteration; Change the training parameters of WGAN-GP to determine whether the loss values ​​of the discriminator network and the generator network in the WGAN-GP network model tend to be stable and low. In this case, the generated samples are of high quality. Step S4, compare the generated sample set with the test data set to perform model error analysis and determine the stability of the model.

2. The sEMG data enhancement method based on BiLSTM and WGAN-GP network according to claim 1, characterized in that: The preprocessing of the surface electromyography signal in step S1 includes: Step S11, performing denoising processing; Step S12, sliding the window to segment the data and perform feature extraction.

3. The sEMG data enhancement method based on BiLSTM and WGAN-GP network according to claim 2, characterized in that: The feature extraction in step S12 includes average absolute value, number of slope changes, waveform length, and number of zero crossings. (1) Mean absolute value MAV: Where N represents the length of the sEMG signal, k = 1, 2, 3, ..., N, x k is the amplitude of the kth sEMG; (2) Wavelength WL: (3) Zero crossing times ZC: Introducing a threshold a, for two consecutive sEMG signals x k and x k+1 , the number of zero crossings is: (4) Slope change times SSC: Given three continuous sEMG signals x k-1 , x k , x k+1 , if formula (4) holds true, the number of slope changes SSC increases by one; (x k -x k-1 )×(x k -x k+1 )≥a(4)。 4. The sEMG data enhancement method based on BiLSTM and WGAN-GP network according to claim 1, characterized in that: The specific steps of the standardization in step S2 are as follows: The real EMG data after feature extraction is standardized, and the MinMaxScale function in the sklearn library is used to standardize the real EMG data samples to [0,1]. X scaled represents the normalized result, X represents the sample value at that point, and X max represents the maximum sample value, X min Indicates the minimum sample value, (max-min) indicates the scaling range, and min indicates the minimum scaling value.

5. The sEMG data enhancement method based on BiLSTM and WGAN-GP network according to claim 1, characterized in that: The steps of generating a sample set in step S3 include: M noise samples are randomly generated from a normal distribution to form a noise sample set. The M noise samples are input into the WGAN-GP network model, and the generated sample data are output, where M and P are equal; the generated sample data are inversely standardized and output to obtain the generated sample set.

6. The sEMG data enhancement method based on BiLSTM and WGAN-GP network according to claim 1, characterized in that: The specific steps of the model error analysis in step S4 are as follows: Compare and analyze the generated sample set of the WGAN-GP network model with the test data set to measure the quality of the model; Root mean square error RMSE: Symmetric mean absolute percentage error SMAPE: Where y is the true value, that is, the generated sample set, For the predicted value, that is, the test data set, when RMSE < 0.2 and SMAPE < 100%, the model is stable and the synthesized sEMG has a high quality.

7. The sEMG data enhancement method based on BiLSTM and WGAN-GP network according to claim 1, characterized in that: The method further includes step S5, which verifies the effectiveness and diversity of the model in enhancing electromyographic data by qualitative or quantitative methods.

8. The sEMG data enhancement method based on BiLSTM and WGAN-GP network according to claim 1, characterized in that: In step S5, the qualitative verification is performed by principal component analysis (PCA) to measure the distribution of the generated sample set and the distribution of the real data sample; Quantitative verification is to introduce different classification algorithms to compare classifiers, and obtain the accuracy under different training sets: real data, synthetic data, real data and synthetic data, and the same test set conditions. The training set is real data and the test set is real data. The obtained classifier accuracy rate of 1 is used as the experimental baseline, and compared with the classifier accuracy rates of 2 and 3 obtained when the training set is synthetic data and mixed data, and the test set is real data, respectively.

Images