Gait recognition method based on surface electromyogram signals and transfer learning

Abstract

A gait recognition method based on surface electromyographic signals and transfer learning comprises the steps that a surface electromyographic signal collecting system is designed and manufactured, and electromyographic signals of eight muscles of the legs of a subject are collected through an electromyographic sensor and data collecting equipment; the method comprises the following steps: recording gaits of a subject through motion capture equipment, carrying out timestamp alignment on electromyographic signals and gait information, carrying out label signal labeling on the electromyographic signals, and carrying out filtering and feature processing to obtain a training data set; an LDA-LSTM gait phase recognition algorithm is constructed; constructing an improved MCD transfer learning framework; and carrying out training optimization on the constructed transfer learning framework parameters to enable the recognition and classification effect of the transfer learning framework parameters in the target domain to be optimal, and storing the trained model as a whole model containing the training weight of the target domain again for recognizing the gait phase of the target domain. According to the method, the cross-individual identification accuracy of the model is improved, and the problems of difficulty in sample label labeling, high cost and the like when different individuals are trained are solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a gait recognition method based on surface electromyography signals and transfer learning. Background Art

[0002] As an important research direction in the fields of sports medicine, rehabilitation engineering and biomechanics, gait analysis aims to evaluate the movement status and health level of individuals by studying gait patterns. Traditional gait analysis techniques mainly rely on optical motion capture systems, which capture and analyze gait through multiple cameras and markers. However, although optical motion capture systems provide high-precision data, they have some significant disadvantages in practical applications: bulky, high space requirements, complex installation and the need for specific environmental support.

[0003] To overcome these shortcomings, the application of surface electromyography (sEMG) technology in gait analysis has gradually attracted attention in recent years. sEMG technology records the electrical activity of muscles by placing electrodes on the surface of the skin. Compared with optical motion capture systems, sEMG systems have the advantages of small size, easy to wear, and safe to use. This makes the application of sEMG technology in gait analysis a hot research direction, especially in the fields of gait rehabilitation, motion assessment, and instability prediction.

[0004] However, due to the individual differences in electromyographic signals, the electromyographic recognition model trained with surface electromyographic data obtained from a specific user is generally only used for the user himself and cannot be applied across individuals. Summary of the invention

[0005] In order to address the defects and shortcomings of the prior art, the present invention provides a gait recognition method based on surface electromyographic signals and transfer learning, and designs an electromyographic signal gait recognition model, which is used as an initialization model for transfer learning to realize the classification of 7-phase gaits. Through the domain adaptive learning method of maximizing classifier differences (MCD), the initialization model is trained again using sample data of different individuals to improve the cross-individual recognition accuracy of the model.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A gait recognition method based on surface electromyography signal and transfer learning comprises the following steps:

[0008] Step 1, design and make a surface electromyography signal acquisition system, which includes an electromyography signal acquisition subsystem and a gait phase capture subsystem. The electromyography signal acquisition subsystem collects electromyography signals from 8 muscles of the subject's legs through electromyography sensors and data acquisition equipment; the gait phase capture subsystem records the subject's gait through a motion capture device; then the electromyography signal is timestamped and aligned with the gait information, the electromyography signal is labeled, and after filtering and feature processing, a training data set is obtained;

[0009] Step 2: Construct an LDA-LSTM gait phase recognition algorithm. The particle swarm algorithm PSO is used to optimize the parameters of the LDA-LSTM model. PSO is used to adjust the parameters of the LDA-LSTM model. The model performance is evaluated by classification accuracy. In the LDA-LSTM model, PSO explores the optimal value of each parameter in LSTM based on the fitness function. After training, the entire model including the training weights is saved to obtain the final LDA-LSTM model as the initialization model for transfer learning.

[0010] Step 3: Build an improved MCD transfer learning framework: Split the model obtained in step 2 into two parts: feature extractor and classifier. The classifiers are constructed identically and imported into the framework for training. During the training process, the training set of the initialized model is called the source domain, and the training set that needs to be trained through transfer is called the target domain. The target domain and the source domain are aligned by maximizing and minimizing the difference between the classifiers.

[0011] Step 4: Train and optimize the parameters of the transfer learning framework constructed in step 3 to achieve the best recognition and classification effect in the target domain. The trained model is saved again as the entire model containing the target domain training weights for identifying the gait phase of the target domain.

[0012] Furthermore, in step 2, the LSTM model includes two LSTM layers, the first layer includes 32 units, the second layer includes 128 units, the dropout is 0.2, each LSTM layer is followed by a batch normalization layer, a fully connected layer including 32 neurons, and a softmax activation function as the output classification part.

[0013] Furthermore, in step 3, before MCD, LDA dimensionality reduction is replaced by PCA dimensionality reduction for dimensionality reduction of unlabeled target domain data.

[0014] Preferably, in step 3, the model obtained in step 2 is split into two parts: a feature extractor G and classifiers F1 and F2. The feature extractor uses an input layer, 2 layers of LSTM, 2 layers of batch normalization, 2 layers of dropout, and 1 layer of linear rectification activation function; the classifiers are exactly the same, including 1 fully connected layer with 32 neurons, 1 layer of linear rectification activation function, 1 layer of dropout, and 1 layer of softmax activation function.

[0015] In step 3, the training process of MCD is to fix the feature extractor G in the first stage and update the classifiers F1 and F2. In the second stage, the classifiers F1 and F2 are fixed and the feature extractor G is updated. In the first stage, the objective function is improved. for: That is, a balance factor is added to control the weight of adversarial loss in the total loss, prevent gradient explosion, and stabilize training.

[0016] In step 3, during the transfer learning training, the model uses Focal Loss as the loss function. FocalLoss is a loss function designed to solve the problem of class imbalance. It reduces the impact of easy-to-classify samples on the total loss and enhances the influence of difficult-to-classify samples by introducing an adjustable weight coefficient. Compared with the traditional cross entropy loss function, Focal Loss pays more attention to those samples that are misclassified. By dynamically adjusting the category weights, FocalLoss can effectively improve the model's classification performance for minority categories in the case of unbalanced samples, avoid the model's excessive bias towards the majority class, and thus improve the overall classification accuracy.

[0017] The beneficial effects of the present invention are as follows: by adopting an unsupervised domain adaptation method, cross-individual gait phase recognition based on surface electromyographic signals is effectively realized; this method does not rely on manual labeling of target domain data, which greatly reduces the workload of sample labeling, especially when facing large-scale data, significantly reduces labor costs and time consumption; at the same time, since unsupervised domain adaptation can automatically align data distribution differences between different individuals, the generalization ability of the model is improved, so that it can accurately identify gait phases between different individuals; this method effectively solves the influence of electromyographic signal differences between individuals on the model recognition accuracy, thereby improving the accuracy and robustness of cross-individual gait phase recognition, and has broad promotion value in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a gait recognition method based on surface electromyography signals and transfer learning.

[0019] Figure 2 It is the structural diagram of the gait recognition model. DETAILED DESCRIPTION

[0020] The present invention will be further described below in conjunction with the accompanying drawings.

[0021] Reference Figure 1 and Figure 2 A gait recognition method based on surface electromyography signals and transfer learning is proposed. By using a customized initialization model and combining it with an improved transfer learning framework, the generalization of the initialization model is improved and the cost of obtaining training data is reduced.

[0022] like Figure 1 As shown in the figure, it is a flow chart of the method of the present invention, and the gait recognition method based on surface electromyography signal and transfer learning comprises the following steps:

[0023] Step 1, design and make a surface electromyography signal acquisition system, which includes an electromyography signal acquisition subsystem and a gait phase capture subsystem. The electromyography signal acquisition subsystem collects electromyography signals from 8 muscles of the subject's legs through electromyography sensors and data acquisition equipment; the gait phase capture subsystem records the subject's gait through a motion capture device; then the electromyography signal is timestamped and aligned with the gait information, the electromyography signal is labeled, and after filtering and feature processing, a training data set is obtained;

[0024] This embodiment uses electromyographic sensors and data acquisition equipment to collect gait data of eight leg muscles when a subject walks on a treadmill at a speed of 2.5 km / h, wherein the eight muscles include: rectus femoris RF, semitendinosus ST, medial gastrocnemius GM, and lateral gastrocnemius GA. While collecting electromyographic data, a motion capture device is used to collect gait data when the subject is walking. The motion capture data and electromyographic signal data are aligned according to the timestamp to obtain the label data of the gait, and the gait is divided into 7 phases, the electromyographic data is labeled, and the data is pre-processed by filtering, feature processing, etc. to obtain a data set for model training.

[0025] Step 2: Construct an LDA-LSTM gait phase recognition algorithm. The particle swarm algorithm PSO is used to optimize the parameters of the LDA-LSTM model. PSO is used to adjust the parameters of the LDA-LSTM model. The model performance is evaluated by classification accuracy. In the LDA-LSTM model, PSO explores the optimal value of each parameter in LSTM based on the fitness function. After training, the entire model including the training weights is saved to obtain the final LDA-LSTM model as the initialization model for transfer learning.

[0026] Use the obtained data set for training. After a good result is obtained, use it as the initialization model. Figure 2 Shown is the structural diagram of the constructed gait recognition model.

[0027] This model is mainly a sequence model with LSTM (Long Short-Term Memory Network), combined with BatchNormalization, Dropout (discard layer) and a fully connected layer (Dense Layer).

[0028] Specifically, the model has two LSTM layers. The first layer contains 32 units to capture the temporal dependencies in the input data, followed by an Activation function to introduce nonlinearity. The second layer contains 128 units to further process the sequence data, and also comes with an activation function. The dropout is 0.2, and there is a batch normalization layer after each LSTM layer to accelerate network training and improve the stability of the model. It prevents the gradient from disappearing or exploding by normalizing the hidden layer output. A fully connected layer containing 32 neurons and a softmax activation function are used as the output classification part.

[0029] Step 3: Build an improved MCD transfer learning framework: Split the model obtained in step 2 into two parts: feature extractor and classifier. The classifiers are constructed identically and imported into the framework for training. During the training process, the training set of the initialized model is called the source domain, and the training set that needs to be trained through transfer is called the target domain. The target domain and the source domain are aligned by maximizing and minimizing the difference between the classifiers.

[0030] Step 4: Train and optimize the parameters of the transfer learning framework constructed in step 3 to achieve the best recognition and classification effect in the target domain. The trained model is saved again as the entire model containing the target domain training weights for identifying the gait phase of the target domain.

[0031] This embodiment adopts the framework of transfer learning to build a transfer learning model. After training, improving and optimizing the transfer learning model, a model with a higher accuracy rate for the target domain is obtained.

[0032] The model in step 2 above is used as the initialization model, where the first few layers are used as feature extractors in the transfer learning framework, and the latter few layers are used as classifiers in the transfer learning framework, and two such classifiers are constructed. The MCD transfer learning method is used to construct the transfer learning framework. The MCD process is to fix the feature extractor G in the first stage and update the classifiers F1 and F2. In the second stage, the classifiers F1 and F2 are fixed and the feature extractor G is updated. In the first stage, the objective function is:

[0033]

[0034] Increase the balance factor:

[0035]

[0036] It is used to control the weight of adversarial loss in the total loss, prevent gradient explosion, and stabilize training. s represents the source domain data, Y s represents the source domain label. adv (X t ) is the adversarial loss of the target domain, which is calculated as:

[0037]

[0038] Among them, X t represents the target domain data, Represents the target domain data X t Medium t The expected value of d(p1(y|x t ),p2(y|x t )) are two probability distributions p1(y|x t ) and p2(y|x t ) between two distance metrics. t ) and p2(y|x t ) represent the classifier in a given target domain sample x t , the conditional probability distribution of output category y.

[0039] The uncertainty of the model is measured by comparing the outputs of two classifiers p1 and p2 on the target domain data. The purpose of the loss is to improve the robustness of the classifier by increasing the difference between the classifier outputs.

[0040] d(p1,p2) is the distance between the outputs of two classifiers and is calculated as:

[0041]

[0042] Among them, K represents the number of categories, p 1k represents the probability of the first probability distribution p1 in the kth category, p 2k Represents the probability of the second probability distribution p2 in the kth category.

[0043] Get the prediction difference between the two classifiers for each category k on the target domain data.

[0044] In particular, due to the imbalance of gait, Focal_loss is used as the loss function of training, and the loss function is:

[0045]

[0046] Among them, y true represents the true label, y predrepresents the model prediction label, N represents the number of samples, represents the true label of the i-th sample, where k means that the sample belongs to category k (one-hot encoding), and K represents the number of categories. Represents the probability of the kth category predicted by the model in the i-th sample.

[0047] It is used to reduce the loss weight of easy-to-classify samples and amplify the loss of difficult-to-classify samples. α represents the weight of the balanced category. γ represents the hyperparameter of the adjustment factor, which is used to adjust the decay rate of the loss and increase the attention to difficult-to-classify samples. The training process of the model is optimized by adjusting the size of these two values.

[0048] The neural network in this embodiment uses the Tensorflow framework, and the remaining functions are all written in Python.

[0049] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A gait recognition method based on surface electromyography signals and transfer learning, characterized in that: The method comprises the following steps: Step 1, design and make a surface electromyography signal acquisition system, which includes an electromyography signal acquisition subsystem and a gait phase capture subsystem. The electromyography signal acquisition subsystem collects electromyography signals from 8 muscles of the subject's legs through electromyography sensors and data acquisition equipment; the gait phase capture subsystem records the subject's gait through a motion capture device; then the electromyography signal is timestamped and aligned with the gait information, the electromyography signal is labeled, and after filtering and feature processing, a training data set is obtained; Step 2: Construct an LDA-LSTM gait phase recognition algorithm. The particle swarm algorithm PSO is used to optimize the parameters of the LDA-LSTM model. PSO is used to adjust the parameters of the LDA-LSTM model. The model performance is evaluated by classification accuracy. In the LDA-LSTM model, PSO explores the optimal value of each parameter in LSTM based on the fitness function. After training, the entire model including the training weights is saved to obtain the final LDA-LSTM model as the initialization model for transfer learning. Step 3: Build an improved MCD transfer learning framework: Split the model obtained in step 2 into two parts: feature extractor and classifier. Build two identical classifiers and import them into the framework for training. During the training process, the training set of the initialized model is called the source domain, and the training set that needs to be trained through transfer is called the target domain. The target domain and the source domain are aligned by maximizing and minimizing the difference between the classifiers. Step 4: Train and optimize the parameters of the transfer learning framework constructed in step 3 to achieve the best recognition and classification effect in the target domain. The trained model is saved again as the entire model containing the target domain training weights for identifying the gait phase of the target domain.

2. A gait recognition method based on surface electromyography signals and transfer learning as claimed in claim 1, characterized in that: In step 2, the LSTM model includes two LSTM layers, the first layer includes 32 units, the second layer includes 128 units, the dropout is 0.2, each LSTM layer is followed by a batch normalization layer, a fully connected layer including 32 neurons, and a softmax activation function as the output classification part.

3. A gait recognition method based on surface electromyography signals and transfer learning as described in claim 1 or 2, characterized in that: In step 3, before MCD, LDA dimensionality reduction is replaced by PCA dimensionality reduction for dimensionality reduction of unlabeled target domain data.

4. A gait recognition method based on surface electromyography signal and transfer learning as claimed in claim 3, characterized in that: In the step 3, the model obtained in step 2 is split into two parts: a feature extractor G and classifiers F1 and F2. The feature extractor uses an input layer, 2 layers of LSTM, 2 layers of batch normalization, 2 layers of dropout, and 1 layer of linear rectification activation function; the classifiers are exactly the same, including 1 fully connected layer with 32 neurons, 1 layer of linear rectification activation function, 1 layer of dropout, and 1 layer of softmax activation function.

5. A gait recognition method based on surface electromyography signal and transfer learning as claimed in claim 3, characterized in that: In step 3, the training process of MCD is to fix the feature extractor G in the first stage and update the classifiers F1 and F2, and fix the classifiers F1 and F2 in the second stage to update the feature extractor G; In the first stage, the objective function is improved for: That is, a balancing factor λ is added to control the weight of adversarial loss in the total loss, prevent gradient explosion, and stabilize training; X s represents the source domain data, Y s represents the source domain label, L adv (X t ) is the adversarial loss of the target domain, which is calculated as: Among them, X t represents the target domain data, Represents the target domain data X t Medium t The expected value of d(p1(y|x t ),p2(y|x t )) are two probability distributions p1(y|x t ) and p2(y|x t ), p1(y|x t ) and p2(y|x t ) represent the classifier in a given target domain sample x t When , the conditional probability distribution of output category y is; The uncertainty of the model is measured by comparing the outputs of two classifiers p1 and p2 on the target domain data; d(p1, p2) is the distance between the outputs of the two classifiers, which is calculated as: Among them, K represents the number of categories, p 1k represents the probability of the first probability distribution p1 in the kth category, p 2k Represents the probability of the second probability distribution p2 in the kth category; Get the prediction difference between the two classifiers for each category k on the target domain data.

6. A gait recognition method based on surface electromyography signal and transfer learning as claimed in claim 3, characterized in that: In step 3, during the transfer learning training, the model uses Focal Loss as the loss function. Among them, y true represents the true label, y pred represents the model prediction label, N represents the number of samples, represents the true label of the i-th sample, k represents the category to which the sample belongs, and K represents the number of categories. represents the probability of the kth category predicted by the model in the i-th sample; It represents the weight used to reduce the loss of easy-to-classify samples and amplify the loss of difficult-to-classify samples. α represents the weight of the balanced category. γ represents the hyperparameter of the adjustment factor, which is used to adjust the decay rate of the loss and increase the attention to difficult-to-classify samples. The training process of the model is optimized by adjusting the size of these two values.

Images