A deep learning-based electromyographic signal data generation method

Abstract

The application discloses a kind of based on deep learning's electromyographic signal data generation method, belong to deep learning field;First, build including deep angle sequence encoder, context encoder, electromyographic signal generator and discriminator's confrontation model framework, and place on user hand joint, generate electromyographic signal data;Among them, deep angle sequence encoder is used to encode hand joint angle sequence control signal, and input context encoder generates the latent information of electromyographic signal, generates corresponding electromyographic signal using electromyographic signal generator;Electromyographic signal discriminator discriminates whether each time step conforms to real electromyographic signal mode;Then, electromyographic signal data is used to train electromyographic signal classification network based on convolutional neural network, and is deployed to actual embedded device or mobile terminal, carries out real-time gesture recognition.The application effectively expands existing electromyographic signal dataset, and simultaneously uses sequence information as control signal, effectively improves the quality of data generation.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of deep learning, and particularly relates to a deep learning-based electromyography signal data generation method. BACKGROUND

[0002] Gestures play an important role in non-verbal communication. Traditional methods use cameras and optical motion capture devices to depict the motion trajectory of the human body, and then recognize the gesture action; surface electromyography (sEMG) signals provide another solution for recognizing motion intention including gestures.

[0003] sEMG is a bioelectric signal obtained by placing electromyography electrodes on the surface of the human body, which is directly related to the behavior intention of the human being. Even for patients with decreased limb muscle function and unable to control muscle contraction, the gesture action intention can still be mapped through the collection of sEMG of specific muscles.

[0004] In recent years, sEMG has been widely used in the fields of prosthesis control, medical devices and human-computer interaction. By using artificial intelligence algorithms to analyze the collected sEMG, it can better help disabled people to independently complete some daily interactions. Due to the rich information, non-invasive to the human body and easy to obtain the collection device, the recognition of gesture action based on sEMG has gradually become a research hotspot, and the core is to accurately distinguish the sEMG signals of different gestures.

[0005] Using appropriate electromyography equipment and preprocessing methods can effectively improve the gesture recognition rate, but more importantly is the selection of the classifier. Traditional machine learning models have been maturely applied in this field, taking the extracted signal features as the input of the model, but too complex feature set will lead to the loss of sEMG information.

[0006] Common traditional machine learning models include linear discriminant analysis (LDA), artificial neural network (ANN), support vector machine (SVM), k-nearest neighbor (KNN) and naive Bayes (NB).

[0007] Deep learning model is a special branch of machine learning, which is different from traditional machine learning relying on manual feature extraction. It can take the original sEMG signal as the input of the model and allow feature extraction and model construction at the same time. Commonly used deep learning models include convolutional neural network (CNN), long short-term memory (LSTM), temporal convolutional network (TCN), gate recurrent unit (GRU), and deep belief network (DBN), etc. Compared with traditional machine learning, deep learning has high recognition accuracy, but it requires a large amount of data and a long training time.

[0008] In the field of gesture recognition, although sEMG-based gesture recognition can achieve good results under certain experimental conditions, it still faces some challenges in practical applications, especially in terms of model generalization ability and transfer ability. The relative scarcity of sEMG dataset and factors such as electrode patch position and user individual differences make the performance of sEMG-based gesture recognition system greatly affected in different users and new gesture postures. Specifically as follows:

[0009] 1) Data set scarcity and model generalization ability;

[0010] At present, the sEMG dataset is still relatively small, especially in terms of diversity and scale. Most existing sEMG datasets often cover only a limited number of gestures and a small sample size of subjects, resulting in insufficient diversity of the dataset when training the model, making it difficult to cope with variable gestures and user characteristics in practical applications. With the existence of physiological differences between users (such as muscle size, skin characteristics, etc.), the generalization ability is poor when facing different users, and the recognition accuracy is also prone to decline.

[0011] In addition, sEMG has strong individual differences. Even the same person, when performing gestures at different times and in different environments, the generated electromyogram may have large fluctuations.

[0012] Lack of sufficient and diverse training data makes it difficult for the model to accurately adapt to these changes, affecting its performance in different users or new postures.

[0013] 2) Poor transfer ability in new gesture postures

[0014] Another challenge of sEMG signals is the impact of gesture variations on recognition performance. Gesture recognition not only requires accurate recognition of specific gestures, but also the ability to adapt to variations in gestures. Traditional sEMG-based recognition methods often have weak transferability when facing new gesture postures (e.g., different gesture angles, muscle tension levels, etc.). Because new postures can cause significant changes in sEMG signals, even completely different from the signals in the training dataset.

[0015] Due to the limited variety of postures in the dataset, the model is not fully trained in diverse postures, so the transferability of the model is obviously insufficient when facing unseen postures, resulting in a significant decrease in recognition accuracy. In order to improve the performance of the model in new postures, a large amount of additional training data is usually required, which is often not feasible in practical applications.

[0016] 3) Adaptation problem caused by user differences

[0017] User differences (such as muscle structure, skin conductivity, exercise habits, etc.) directly affect the characteristics of sEMG signals. Therefore, the model often performs well within the user group of the training dataset, but when faced with new users, the performance will decrease significantly due to individual differences. Traditional methods often rely on a large amount of labeled individual data for training, and the collection of labeled data is very expensive and time-consuming, which makes it difficult for the model to handle user differences.

[0018] Existing transfer learning-based methods attempt to transfer a model trained on one user or dataset to another user. However, they still face the problems of insufficient training samples and excessive individual differences, and the transfer effect is often not ideal, especially in the absence of sufficient diverse data support.

[0019] In order to improve the performance of sEMG signals in practical applications, improvements need to be made from multiple directions:

[0020] First, increase the diversity and size of the dataset, collect more sEMG signals from different users and different postures, which can effectively enhance the generalization and transferability of the model. In addition, enhance the adaptability of the model, especially develop training strategies that can adapt to individual differences of users, such as personalized transfer learning methods, which can also significantly improve the performance in practical scenarios.

[0021] Although existing methods attempt to solve the above problems, the scarcity of sEMG signals, individual differences, and posture variations remain the main challenges faced by current technology. Therefore, in the application of gesture recognition based on sEMG signals, more research and technological innovation are needed to improve its stability and accuracy in various complex environments and user differences. SUMMARY

[0022] In view of the problems of data scarcity and poor model generalization ability of the existing gesture recognition method based on surface electromyography (sEMG), the application provides an electromyography data generation method based on deep learning, which can effectively expand the existing electromyography data set and contribute to solving data scarcity and improving model generalization ability. Meanwhile, the application uses sequence information as a control signal to effectively improve the quality of data generation and reduce optimization difficulty.

[0023] The electromyography data generation method based on deep learning has the following specific steps:

[0024] Step one, build a generative adversarial model framework and place it on a user's hand joint to generate electromyography data;

[0025] The generative adversarial model framework includes a deep angle sequence encoder, a context encoder, an electromyography signal generator and an electromyography signal discriminator;

[0026] Step 101, the deep angle sequence encoder is used to encode the hand joint angle sequence control signal and input the context encoder;

[0027] Step 102, at each time step t, the lower layer of the context encoder uses a GRU unit to receive the splicing of the electromyography signal s t and Gaussian noise ∈ t , and outputs a feature vector i t and a gating signal g t :

[0028] i t ,g t =GRU(s t ,∈ t ,g t-1 ),

[0029] Step 103, the upper layer of the context encoder uses an Ang2Gist unit to combine the output vector i t and the story context h t to generate an output Gist vector o t , which encodes the latent information of the generated electromyography signal;

[0030] The output Gist vector o t is:

[0031] o t ,h t =Ang2Gist(i t ,h t-1 ),

[0032] Step 104, the story context ht Updated by Ang2Gist unit to reflect the change of potential context information;

[0033] The specific update formula is as follows:

[0034]

[0035] r t = σ r (W r i t + U r h t-1 + b r ),

[0036] h t = (1-z t )⊙h t-1 + z t ⊙σ h (W h i t + U h h t-1 + b h ),

[0037] o t = Filter(i t )*h t

[0038] U z , is the first layer of learnable network parameters in Ang2Gist, used to encode the feature vector i t and h t-1 input, and output the intermediate layer feature vector At the same time, Ang2Gist also contains another set of learnable network parameters σ r , W r , U r , b r to encode the encoded feature vector i t and h t-1 input, and then output the intermediate layer feature vector r t ; σ h , W h , U h , b h are used to encode the intermediate feature vector and r t , output h t of the next moment; i t after convolution and h t multiplication, the Gist vector o t is obtained.

[0039] Step 105, output the potential information to the electromyography signal generator to generate corresponding electromyography signals;

[0040] The electromyography signal generator uses a deep convolutional network to generate electromyography signals according to the Gist vector o t Step-by-step up-sampling generates electromyography signals

[0041] Step 106, the electromyography signal discriminator measures the generated electromyography signals With the real electromyography signals s t , the matching degree m o of the initial hand joint electromyography signal information h t at a given time step is used to determine whether each time step

[0042]

[0043] Step 107, the electromyography encoder encodes the hand joint angle sequence into a feature vector, concatenates the matching degrees of all time steps, and calculates the global consistency score through a fully connected layer and a sigmoid function;

[0044] P = FC (sigmoid ([m0, m1, …, m t ])

[0045] FC is a fully connected layer, and P is the classification probability obtained;

[0046] The MSE error calculation formula is:

[0047] The GAN adversarial loss calculation formula is:

[0048] When the MSE error is lower than 0.01, the signal feature distribution matching degree is high, and the GAN generator discriminator , it means that the signal is similar enough.

[0049] Step two, use the generated electromyography signal data to train the electromyography signal classification network;

[0050] First, a 10Hz high-pass filter is used to remove low-frequency noise components in the electromyography signal data and extract high-frequency noise components of the electromyography signal;

[0051] Then, the square root algorithm is used to further optimize the electromyography signal data of the high-frequency noise components.

[0052] Finally, a convolutional neural network (CNN) based electromyography signal classification network is constructed, the optimized electromyography signal data is used for training, the spatio-temporal features in the signal are extracted, and the corresponding gesture category sequence is predicted; through continuous training, the electromyography signal classification network gradually learns the mapping relationship between the electromyography signal and the gesture, and high-accuracy gesture recognition is realized.

[0053] Step three, the trained convolutional neural network (CNN) based electromyography signal classification network is deployed to an actual embedded device or mobile terminal for real-time gesture recognition.

[0054] The advantages of the present application are:

[0055] 1) The electromyography signal data generation method based on deep learning of the present application enhances the diversity of the data set: the electromyography signal data of a new gesture can be controlled to be generated, and this controllable generation method can not only effectively expand the existing electromyography signal data set, but also further improve the generalization ability of the model by generating signal data with diversity. An effective solution is provided to solve the problem of data scarcity and improve the robustness of the model.

[0056] 2) The electromyography signal data generation method based on deep learning of the present application improves the data generation quality by using sequence information: unlike traditional electromyography signal generation methods, the present application can more accurately generate electromyography signals that meet actual conditions by using hand joint angle sequences as control signals. The introduction of sequence information can not only ensure the quality of data generation, but also help to reduce the optimization difficulty, thereby improving the training efficiency.

[0057] 3) The electromyography signal data generation method based on deep learning of the present application adopts a generative adversarial model (GAN) framework, encodes the hand joint angle sequence through a deep angle sequence encoder, generates electromyography signals through a generator, and uses a discriminator to improve the quality and consistency of the generated signals, ensuring that the generated electromyography signals are closer to the real signals and enhancing the reliability of the model.

[0058] 4) The electromyography signal data generation method based on deep learning of the present application can effectively solve the overfitting problem caused by insufficient samples in traditional methods by generating a large amount of high-quality electromyography signal data, further improving the generalization ability of the gesture recognition model. This enables the present application to achieve good recognition performance in various complex scenarios.

[0059] 5) The electromyography signal data generation method based on deep learning of the present application is not only suitable for electromyography signal recognition, but also provides a new idea for other signal generation-based applications. Especially in machine learning tasks that require a large amount of data for training, high-quality samples can be synthesized through a generative adversarial network, expanding the universality of its application. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a flowchart of a deep learning-based electromyographic signal data generation method of the present application;

[0061] Figure 2 is a schematic diagram of a generative adversarial model framework for generating electromyographic signal data according to the present application. DETAILED DESCRIPTION

[0062] The deep learning-based electromyographic signal generation system according to the present application will be described in further detail below with reference to the accompanying drawings and examples.

[0063] The deep learning-based electromyographic signal data generation method according to the present application converts noise into synthetic EMG data through a generative adversarial network; at the same time, it uses a hand joint angle sequence as a generation control signal to control the generation of EMG data for a new gesture. The EMG data for the new gesture can enrich the data set, so as to more fully train the classifier and improve the generalization ability.

[0064] The deep learning-based electromyographic signal data generation method according to the present application, as shown in Figure 1 The specific steps are as follows:

[0065] Step 1: Build a generative adversarial model framework and place it on the user's hand joint to generate electromyographic signal data.

[0066] As shown in Figure 2 The generative adversarial model framework is based on a sequence condition, and the unique feature is that it contains an angle sequence encoder (Angle Encoder) for dynamically tracking the hand joint angle sequence signal to control data generation, as well as an electromyographic signal generator and a discriminator to enhance the quality and consistency of the generated sequence.

[0067] Unlike previous electromyographic signal generation methods, the present application uses a hand joint angle sequence as a control signal to generate corresponding electromyographic signal data for a new gesture.

[0068] First, the deep angle sequence encoder is used to encode the hand joint angle sequence control signal and input the context encoder to dynamically generate the latent information of the electromyographic signal. The output is input to the electromyographic signal generator to generate corresponding electromyographic signal data according to the control signal output by the context encoder. Finally, the electromyographic signal discriminator is used to measure the quality and consistency of the generated electromyographic signal data to ensure the similarity between the generated signal and the real electromyographic signal.

[0069] The specific steps are as follows:

[0070] Step 101, the depth angle sequence encoder is used to encode the hand joint angle sequence control signal, and the potential information of the electromyographic signal is input into the context encoder;

[0071] Angle Sequence Encoder (Angle Encoder): Encode the entire hand joint sequence into a low-dimensional vector h0 as the initial hidden state of the context encoder; use two neural networks to realize the mean μ(S) and variance Σ(S), and sample h0 from the normal distribution N(μ(S), Σ(S)).

[0072] Encode the hand joint angle sequence into a low-dimensional vector through the reparameterization trick, and use it as the initial hidden state of the control signal. Here the reparameterization trick can be expressed as the following formula, the goal is to make the random variable derivable:

[0073] Original sampling (non-differentiable):

[0074]

[0075] Reparameterized sampling: Introduce a random noise independent of the parameters

[0076]

[0077] Among the above parameters: the randomness of ε is independent of the parameters μ and σ. The generation process of becomes a deterministic calculation (μ and σ are network output parameters)

[0078] The encoded hand joint angle h0 can be written as h0 = μ(S) + σ 2 (S) 1 / 2 ⊙∈ s , where ∈ s ~ N(0, I).

[0079] The context encoder (Context Encoder) consists of two layers of recurrent neural networks (RNN), the lower layer uses the standard GRU unit, and the upper layer uses the newly proposed Ang2Gist unit.

[0080] Step 102, at each time step t, the GRU unit used by the lower layer of the context encoder receives the electromyographic signal s t and the concatenation of the Gaussian noise ∈ t , outputs the feature vector i t and the gating signal g t :

[0081] i t ,g t = GRU(s t , ∈ t , g t-1 ),

[0082] Step 103, the upper layer of the context encoder uses Ang2Gist unit to combine the output vector i t and story context h t to generate output Gist vector o t ;

[0083] output Gist vector o t is:

[0084] o t = Ang2Gist(i t , h t ) t-1

[0085] Step 104, story context h t is updated by Ang2Gist unit to reflect the changes of underlying context information;

[0086] The specific update formula is as follows:

[0087]

[0088] r t = σ r (W r i t + U r h t-1 + b r )

[0089] h h = (1 - z t ) ⊙ h t-1 + z t ⊙ σ h (W h i t + U h h t-1 + b h )

[0090] o t = Filter(i t ) * h t

[0091] U z , is the first layer of learnable network parameters in Ang2Gist, used to encode feature vector i t and h t-1 input, and output intermediate layer feature vector z t ; meanwhile Ang2Gist also contains another set of learnable network parameters σ​r W r U r ,b r De-encoding is used to encode feature vector i t and h t-1 Input, then output the intermediate layer feature vector r t ;σ h W h U h ,b h Used to encode intermediate feature vectors and r t Output h at the next time step. t i t After convolution and h t Multiplying them together yields the Gist vector o. t .

[0092] Potential information includes:

[0093] 1. Kinematic characteristics: acceleration and shaking patterns of joint movements.

[0094] 2. Muscle synergistic activation mode: the activation sequence and intensity ratio of different muscle groups.

[0095] 3. Individual adaptive parameters: the effects of muscle stiffness and subcutaneous fat layer thickness on signal attenuation.

[0096] 4. Task semantic encoding: Implicit identifier of action type (clenched fist / extended fist).

[0097] Step 105: Output the potential information to the electromyography signal generator to generate the corresponding electromyography signal;

[0098] The electromyography (EMG) generator uses a deep convolutional network to generate signals based on the Gist vector o at each time step t. t Generate electromyographic signals This is achieved using a deep convolutional network, which progressively upsamples to generate high-density electromyographic signals.

[0099] o t =G(o t )

[0100] It includes the following physical characteristics:

[0101] 1. Time-domain characteristics: including waveform amplitude (μV level), pulse width (50-200ms), and the superposition of motion unit action potentials (MUAP);

[0102] 2. Frequency domain characteristics: including energy concentration in the range of 20-500Hz, peak frequency dependence on muscle type, and high-frequency component attenuation characteristics (reflecting the signal propagation path);

[0103] 3. Spatial features: spatial correlation of multi-channel signals (e.g. topological consistency of electrode array) and temporal coordination patterns across muscle groups.

[0104] Step 106, the EMG discriminators measure the generated EMG signals against the true EMG signals s t , given the initial hand joint EMG signal information h o at time t, to determine the matching degree m t at each time step t.

[0105] The EMG discriminators (EMGs Discriminators) ensure that the generated signals have similar features to the true EMG signals by evaluating the matching degree of the generated EMG signals against the control signals and the global consistency of the signals.

[0106] The discriminators evaluate both global consistency (global signal) and local consistency (frame-level signal):

[0107]

[0108] D(x) = D glocal (f(x)) + D local (x t )

[0109] In the EMG task:

[0110] Global discriminator D global : judges whether the entire generated EMG signal sequence matches the movement intention.

[0111] Local discriminator D local : discriminates whether each time step t matches the true EMG signal pattern.

[0112] Step 107, the EMG encoder encodes the hand joint angle sequence into a feature vector, concatenates the matching degrees corresponding to all time steps, and calculates the global consistency score through the fully connected layer and the sigmoid function;

[0113] P = FC (sigmoid ([m0, m1, …, m t ])

[0114] FC is the fully connected layer, and P is the classification probability obtained.

[0115] The MSE error calculation formula is:

[0116] The GAN adversarial loss calculation formula is:​

[0117] When the MSE error is below 0.01, the signal feature distribution matching degree is high, and the GAN generator discriminator is correct, it means that the signals are similar enough.

[0118] Step two, use the generated electromyographic signal data to train the electromyographic signal classification network;

[0119] In order to reduce noise data and enhance effective electromyographic signal, the generated electromyographic signal data is preprocessed, including: 1) high-pass filter: using 10Hz high-pass filter to remove low-frequency noise component in the generated signal, so as to extract the high-frequency component of electromyographic signal and improve the effectiveness of the signal; 2) square root algorithm optimization: using square root algorithm to further optimize the preprocessed high-frequency noise component of electromyographic signal, improving the quality and classification effect of the signal;

[0120] Then, based on the convolutional neural network (CNN), the electromyographic signal classification network is constructed, and the optimized electromyographic signal data is used for training. This network can effectively extract the spatial and temporal features of the signal and predict the corresponding gesture category sequence. Through continuous training, the electromyographic signal classification network gradually learns the mapping relationship between electromyographic signal and gesture, realizing high-accuracy gesture recognition.

[0121] Training process: use Adam optimizer to update the parameters of deep angle sequence encoder, context encoder, electromyographic signal generator and electromyographic signal discriminator. The parameters of electromyographic signal discriminator are updated in two for loops respectively, while the parameters of electromyographic generator are updated in both loops. Different mini-batch size and different time step are used to update the generator and discriminator to accelerate the training convergence.

[0122] Step three, deploy the trained convolutional neural network (CNN) based electromyographic signal classification network to actual embedded devices or mobile terminals for real-time gesture recognition.

[0123] Deploy the trained electromyographic signal classification network to the actual application environment, such as embedded devices or mobile terminals, for real-time gesture recognition. This system can be widely used in intelligent prosthesis control, brain-computer interface, rehabilitation medicine and other fields, helping users to achieve more natural and accurate gesture control.

[0124] The myoelectric signal data generation system of the present application can expand the existing myoelectric signal data set by synthesizing high-quality myoelectric signal data, thereby improving the generalization ability and robustness of the gesture recognition model. It is suitable for gesture recognition tasks and can improve the recognition accuracy of the model under different users and new gesture postures by generating myoelectric signals consistent with real data. The generative adversarial network framework can generate diversified myoelectric signal data, support adaptation to individual differences of users and changes in gestures, and thus improve the migration ability and recognition accuracy of the model.

Claims

1.A method for deep learning-based electromyography signal data generation, characterized by, The specific steps are as follows: Step one, build a generative adversarial model framework and place it on the user's hand joint to generate electromyographic signal data; The generative adversarial model framework includes a deep angle sequence encoder, a context encoder, an electromyographic signal generator, and an electromyographic signal discriminator; Step 101, the deep angle sequence encoder is used to encode the hand joint angle sequence control signal and input the context encoder; Step 102, at each time step t, the GRU unit used by the lower layer of the context encoder receives the electromyographic signal and the concatenation of the Gaussian noise outputs a feature vector : Step 103, the upper layer of the context encoder uses the Ang2Gist unit, combining the output vector and story context Generating the output Gist vector , encoding the latent information generating the myoelectric signal; Output Gist vector is: Step 104, story context Updated by the Ang2Gist unit to reflect changes in underlying contextual information; The specific update formula is as follows: is the first layer of learnable network parameters in Ang2Gist, used to encode the feature vector and output the intermediate layer feature vector ; meanwhile, Ang2Gist also contains another set of learnable network parameters to decode the encoded feature vector and then output the intermediate layer feature vector to encode the intermediate feature vector and , output the next moment ; after convolution and multiplication, the Gist vector is obtained; Step 105, outputting the potential information to the myoelectric signal generator to generate corresponding myoelectric signals ; Step 106, the electromyography signal discriminator measures the generated electromyography signal with the real electromyography signal , the matching degree when the initial hand joint electromyography signal information is given , whether each time step meets the real electromyography signal mode: Step 107, the electromyographic encoder encodes the hand joint angle sequence into a feature vector, concatenates the matching degrees corresponding to all time steps, and calculates the global consistency score through a fully connected layer and a sigmoid function; is a fully connected layer, P is the resulting classification probability; Step two, use the generated electromyographic signal data to train the electromyographic signal classification network; First, a 10 Hz high-pass filter is used to remove low-frequency noise components in the electromyographic signal data and extract high-frequency noise components of the electromyographic signal; Then, the square root algorithm is used to further optimize the electromyographic signal data of the high-frequency noise component; Finally, based on the convolutional neural network, the electromyographic signal classification network is constructed, the optimized electromyographic signal data is used for training, the temporal and spatial features in the signal are extracted, and the corresponding gesture category sequence is predicted; through continuous training, the electromyographic signal classification network gradually learns the mapping relationship between the electromyographic signal and the gesture, realizing high-accuracy gesture recognition; Step three, deploy the trained convolutional neural network (CNN) based electromyographic signal classification network to actual embedded devices or mobile terminals for real-time gesture recognition. 2.The deep learning-based electromyography signal data generation method of claim 1, wherein, In the step 105, the electromyography signal generator uses a deep convolutional network to generate the Gist vector for each time step t based on the Gist vector Stepwise up-sampling to generate electromyography signals . 3.The deep learning-based electromyography signal data generation method of claim 1, wherein, In step 107, the MSE error calculation formula is: The GAN adversarial loss calculation formula is: - E When MSE error is lower than 0.01, and GAN generator discriminator , it means that the generated myoelectric signal matches the real myoelectric signal .

Images