A gait recognition method based on surface electromyogram signals and transfer learning

Abstract

A gait recognition method based on surface electromyogram signal and transfer learning, a surface electromyogram signal acquisition system is designed and made, and electromyogram signals of 8 muscles of a leg of a subject are acquired through electromyogram sensors and a data acquisition device; a motion capture device is used to record the gait of the subject, the electromyogram signals are time-stamped aligned with the gait information, the electromyogram signals are labeled with label signals, and after filtering and feature processing, a training data set is obtained; an LDA-LSTM gait phase recognition algorithm is constructed; an improved MCD transfer learning framework is constructed; the parameters of the constructed transfer learning framework are trained and optimized, so that the recognition and classification effect in the target domain is optimal, and the trained model is saved again as a whole model containing the target domain training weight, which is used for recognizing the gait phase of the target domain. The present application improves the cross-individual recognition accuracy of the model, and solves the problems of difficult sample label annotation and high cost during training of different individuals.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a gait recognition method based on surface electromyography signals and transfer learning. Background Art

[0002] Gait analysis, an important research area in sports medicine, rehabilitation engineering, and biomechanics, aims to assess an individual's movement status and health level by studying gait patterns. Traditional gait analysis techniques rely primarily on optical motion capture systems, which capture and analyze gait using multiple cameras and markers. However, while optical motion capture systems provide highly accurate data, they suffer from significant drawbacks in practical applications: bulk, high space requirements, complex installation, and the need for specialized environmental support.

[0003] To overcome these shortcomings, the application of surface electromyography (sEMG) technology in gait analysis has gained increasing attention in recent years. sEMG technology records muscle electrical activity by placing electrodes on the skin's surface. Compared to optical motion capture systems, sEMG systems offer advantages such as compact size, ease of wear, and safety. This has made the application of sEMG technology in gait analysis a hot research topic, particularly in areas such as gait rehabilitation, motor assessment, and instability prediction.

[0004] However, due to the individual differences in electromyographic signals, the electromyographic recognition model trained using surface electromyographic data obtained from a specific user is generally only used for the user himself and cannot be applied across individuals. Summary of the Invention

[0005] In order to address the defects and shortcomings of the existing technology, the present invention provides a gait recognition method based on surface electromyographic signals and transfer learning. A myoelectric signal gait recognition model is designed, which serves as the initialization model for transfer learning and is used to classify the seven-phase gait. Through the domain adaptive learning method of maximizing classifier differences (MCD), the initialized model is retrained using sample data from different individuals to improve the cross-individual recognition accuracy of the model.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A gait recognition method based on surface electromyography signals and transfer learning comprises the following steps:

[0008] Step 1. Design and produce a surface electromyography signal acquisition system, which includes an electromyography signal acquisition subsystem and a gait phase capture subsystem. The electromyography signal acquisition subsystem uses electromyography sensors and data acquisition equipment to collect electromyography signals from eight muscles in the subject's legs; the gait phase capture subsystem uses a motion capture device to record the subject's gait; then, the electromyography signals are timestamped and labeled, and after filtering and feature processing, a training data set is obtained;

[0009] Step 2: Construct an LDA-LSTM gait phase recognition algorithm and optimize the LDA-LSTM model parameters using the particle swarm optimization (PSO). PSO is used to adjust the parameters of the LDA-LSTM model, and the model performance is evaluated by classification accuracy. In the LDA-LSTM model, PSO explores the optimal value of each parameter in the LSTM based on the fitness function. After training, the entire model including the training weights is saved to obtain the final LDA-LSTM model, which serves as the initialization model for transfer learning.

[0010] Step 3: Build an improved MCD transfer learning framework: Split the model obtained in Step 2 into a feature extractor and a classifier. Both classifiers are constructed identically and imported into the framework for training. During training, the training set used to initialize the model is called the source domain, and the training set to be trained through transfer learning is called the target domain. Aligning the target domain with the source domain is achieved by maximizing and minimizing the difference between the classifiers.

[0011] Step 4: Train and optimize the parameters of the transfer learning framework constructed in step 3 to achieve the best recognition and classification effect in the target domain. The trained model is saved again as a complete model containing the target domain training weights for identifying the gait phase of the target domain.

[0012] Furthermore, in step 2, the LSTM model includes two LSTM layers, the first layer includes 32 units, the second layer includes 128 units, the dropout is 0.2, each LSTM layer is followed by a batch normalization layer, a fully connected layer containing 32 neurons, and a softmax activation function as the output classification part.

[0013] Furthermore, in step 3, before MCD, LDA dimensionality reduction is replaced with PCA dimensionality reduction for dimensionality reduction of unlabeled target domain data.

[0014] Preferably, in step 3, the model obtained in step 2 is split into two parts: a feature extractor G and classifiers F1 and F2. The feature extractor uses an input layer, 2 layers of LSTM, 2 layers of batch normalization, 2 layers of dropout, and 1 layer of linear rectification activation function; the classifiers are exactly the same, including 1 fully connected layer with 32 neurons, 1 layer of linear rectification activation function, 1 layer of dropout, and 1 layer of softmax activation function.

[0015] In step 3, the training process of MCD is to fix the feature extractor G in the first stage and update the classifiers F1 and F2. In the second stage, the classifiers F1 and F2 are fixed and updated. In the first stage, the objective function is improved. for: That is, a balance factor is added to control the weight of the adversarial loss in the total loss, prevent gradient explosion, and stabilize training.

[0016] In step 3, during transfer learning training, the model uses Focal Loss as the loss function. Focal Loss is a loss function designed to address the problem of class imbalance. By introducing an adjustable weight coefficient, it reduces the impact of easy-to-classify samples on the total loss and enhances the influence of hard-to-classify samples. Compared to the traditional cross-entropy loss function, Focal Loss pays more attention to those misclassified samples. By dynamically adjusting the class weights, Focal Loss can effectively improve the model's classification performance for minority classes in the case of imbalanced samples, preventing the model from being overly biased towards the majority class, thereby improving the overall classification accuracy.

[0017] The beneficial effects of the present invention are as follows: by adopting an unsupervised domain adaptation method, cross-individual gait phase recognition based on surface electromyography signals is effectively realized; this method does not rely on manual labeling of target domain data, which greatly reduces the workload of sample labeling, especially when facing large-scale data, significantly reducing labor costs and time consumption; at the same time, since unsupervised domain adaptation can automatically align data distribution differences between different individuals, the generalization ability of the model is improved, enabling it to accurately identify gait phases between different individuals; this method effectively solves the influence of inter-individual electromyography signal differences on model recognition accuracy, thereby improving the accuracy and robustness of cross-individual gait phase recognition, and has broad promotion value in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a gait recognition method based on surface electromyography signals and transfer learning.

[0019] Figure 2 It is the structural diagram of the gait recognition model. DETAILED DESCRIPTION

[0020] The present invention will be further described below with reference to the accompanying drawings.

[0021] Reference Figure 1 and Figure 2 A gait recognition method based on surface electromyography and transfer learning is proposed. By using a customized initialization model and combining it with an improved transfer learning framework, the generalization of the initialization model is improved and the cost of obtaining training data is reduced.

[0022] like Figure 1 FIG. 1 is a flow chart of the method of the present invention, wherein the gait recognition method based on surface electromyography signals and transfer learning comprises the following steps:

[0023] Step 1. Design and produce a surface electromyography signal acquisition system, which includes an electromyography signal acquisition subsystem and a gait phase capture subsystem. The electromyography signal acquisition subsystem uses electromyography sensors and data acquisition equipment to collect electromyography signals from eight muscles in the subject's legs; the gait phase capture subsystem uses a motion capture device to record the subject's gait; then, the electromyography signals are timestamped and labeled, and after filtering and feature processing, a training data set is obtained;

[0024] This example uses electromyographic sensors and data acquisition equipment to collect gait data from eight leg muscles while a subject walks on a treadmill at 2.5 km / h. These eight muscles include the rectus femoris (RF), semitendinosus (ST), gastrocnemius medialis (GM), and gastrocnemius lateralis (GA). Simultaneously with the electromyographic data, a motion capture device is used to collect gait data while the subject walks. The motion capture data and electromyographic signal data are aligned according to timestamps to obtain gait label data. The gait is then divided into seven phases, and the electromyographic data is labeled. After preprocessing the data through filtering and feature processing, a dataset for model training is obtained.

[0025] Step 2: Construct an LDA-LSTM gait phase recognition algorithm and optimize the LDA-LSTM model parameters using the particle swarm optimization (PSO). PSO is used to adjust the parameters of the LDA-LSTM model, and the model performance is evaluated by classification accuracy. In the LDA-LSTM model, PSO explores the optimal value of each parameter in the LSTM based on the fitness function. After training, the entire model including the training weights is saved to obtain the final LDA-LSTM model, which serves as the initialization model for transfer learning.

[0026] Use the obtained data set for training, and after obtaining a good result, use it as the initialization model. Figure 2 Shown is the structural diagram of the constructed gait recognition model.

[0027] This model is mainly a sequence model with LSTM (Long Short-Term Memory Network), combined with BatchNormalization, Dropout (discard layer) and fully connected layer (Dense Layer).

[0028] Specifically, the model has two LSTM layers. The first layer contains 32 units to capture the temporal dependencies in the input data, followed by an activation function to introduce nonlinearity. The second layer contains 128 units to further process the sequence data, and also comes with an activation function. The dropout is 0.2, and there is a batch normalization layer after each LSTM layer to accelerate network training and improve the stability of the model. It prevents the gradient from disappearing or exploding by normalizing the hidden layer output. A fully connected layer containing 32 neurons and a softmax activation function are used as the output classification part.

[0029] Step 3: Build an improved MCD transfer learning framework: Split the model obtained in Step 2 into a feature extractor and a classifier. Both classifiers are constructed identically and imported into the framework for training. During training, the training set used to initialize the model is called the source domain, and the training set to be trained through transfer learning is called the target domain. Aligning the target domain with the source domain is achieved by maximizing and minimizing the difference between the classifiers.

[0030] Step 4: Train and optimize the parameters of the transfer learning framework constructed in step 3 to achieve the best recognition and classification effect in the target domain. The trained model is saved again as a complete model containing the target domain training weights for identifying the gait phase of the target domain.

[0031] This embodiment uses a transfer learning framework to build a transfer learning model. After training, improving and optimizing the transfer learning model, a model with high accuracy for the target domain is obtained.

[0032] The model from step 2 above is used as the initialization model, with the first few layers serving as feature extractors in the transfer learning framework and the last few layers serving as classifiers in the transfer learning framework. Two such classifiers are constructed. The MCD transfer learning method is used to construct the transfer learning framework. The MCD process is to fix the feature extractor G in the first stage and update the classifiers F1 and F2. In the second stage, the classifiers F1 and F2 are fixed and the feature extractor G is updated. In the first stage, the objective function is:

[0033]

[0034] Increase the balance factor:

[0035]

[0036] It is used to control the weight of adversarial loss in the total loss, prevent gradient explosion, and stabilize training. s represents the source domain data, Y s represents the source domain label. adv (X t ) is the adversarial loss of the target domain, which is calculated as:

[0037]

[0038] Among them, X t represents the target domain data, Represents the target domain data X t Medium x t The expected value of d(p1(y|x t ),p2(y|x t )) are two probability distributions p1(y|x t ) and p2(y|x t ) between the distance measurement function. p1(y|x t ) and p2(y|x t ) represent the classifier in a given target domain sample x t When , the conditional probability distribution of the output category y is obtained.

[0039] The uncertainty of the model is measured by comparing the outputs of two classifiers p1 and p2 on the target domain data. The purpose of the loss is to improve the robustness of the classifier by increasing the difference between the classifier outputs.

[0040] d(p1,p2) is the distance between the outputs of two classifiers and is calculated as:

[0041]

[0042] Among them, K represents the number of categories, p 1k represents the probability of the first probability distribution p1 in the kth category, p 2k Represents the probability of the second probability distribution p2 in the kth category.

[0043] Get the prediction difference between the two classifiers for each category k on the target domain data.

[0044] In particular, due to the imbalance of gait, Focal_loss is used as the loss function for training, and the loss function is:

[0045]

[0046] Among them, y true represents the true label, y predRepresents the model prediction label, N represents the number of samples, represents the true label of the i-th sample, where k indicates that the sample belongs to category k (one-hot encoding), and K represents the number of categories. represents the probability of the kth category predicted by the model in the i-th sample.

[0047] It represents the weight used to reduce the loss of easy-to-classify samples while amplifying the loss of difficult-to-classify samples. α represents the weight of the balanced category. γ represents the hyperparameter of the adjustment factor, which is used to adjust the decay rate of the loss and increase the attention to difficult-to-classify samples. By adjusting the size of these two values, the training process of the model is optimized.

[0048] The neural network in this embodiment uses the Tensorflow framework, and the remaining functions are all written in Python.

[0049] The embodiments of this specification are merely examples of implementations of the invention and are provided for illustrative purposes only. The scope of protection of the present invention should not be considered limited to the specific embodiments described in these embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by a person of ordinary skill in the art based on the invention.

Claims

1. A gait recognition method based on surface electromyography signals and transfer learning, characterized in that: The method comprises the following steps: Step 1. Design and produce a surface electromyography signal acquisition system, which includes an electromyography signal acquisition subsystem and a gait phase capture subsystem. The electromyography signal acquisition subsystem uses electromyography sensors and data acquisition equipment to collect electromyography signals from eight muscles in the subject's legs; the gait phase capture subsystem uses a motion capture device to record the subject's gait; then, the electromyography signals are timestamped and labeled, and after filtering and feature processing, a training data set is obtained; Step 2: Construct an LDA-LSTM gait phase recognition algorithm and optimize the LDA-LSTM model parameters using the particle swarm optimization (PSO). PSO is used to adjust the parameters of the LDA-LSTM model, and the model performance is evaluated by classification accuracy. In the LDA-LSTM model, PSO explores the optimal value of each parameter in the LSTM based on the fitness function. After training, the entire model including the training weights is saved to obtain the final LDA-LSTM model, which serves as the initialization model for transfer learning. Step 3: Build an improved MCD transfer learning framework: Split the model obtained in step 2 into two parts: a feature extractor and a classifier. Build two identical classifiers and import them into the framework for training. During training, the training set used to initialize the model is called the source domain, and the training set to be trained through transfer is called the target domain. The target domain and the source domain are aligned by maximizing and minimizing the difference between the classifiers. Step 4: Train and optimize the parameters of the transfer learning framework constructed in step 3 to achieve the best recognition and classification effect in the target domain. The trained model is saved again as a complete model containing the target domain training weights for identifying the gait phase of the target domain.

2. A gait recognition method based on surface electromyography and transfer learning as claimed in claim 1, characterized in that: In step 2, the LSTM model contains two LSTM layers, the first layer contains 32 units, the second layer contains 128 units, the dropout is 0.2, each LSTM layer is followed by a batch normalization layer, a fully connected layer containing 32 neurons, and a softmax activation function as the output classification part.

3. A gait recognition method based on surface electromyography and transfer learning as described in claim 1 or 2, characterized in that: In step 3, before MCD, LDA dimensionality reduction is replaced with PCA dimensionality reduction for dimensionality reduction of unlabeled target domain data.

4. A gait recognition method based on surface electromyography and transfer learning as claimed in claim 3, characterized in that: In step 3, the model obtained in step 2 is split into two parts: a feature extractor G and classifiers F1 and F2. The feature extractor uses an input layer, 2 layers of LSTM, 2 layers of batch normalization, 2 layers of dropout, and 1 layer of linear rectifier activation function; the classifiers are exactly the same, including 1 fully connected layer with 32 neurons, 1 layer of linear rectifier activation function, 1 layer of dropout, and 1 layer of softmax activation function.

5. The gait recognition method based on surface electromyography and transfer learning as claimed in claim 3, characterized in that: In step 3, the training process of MCD is to fix the feature extractor G in the first stage and update the classifiers F1 and F2. In the second stage, the classifiers F1 and F2 are fixed and the feature extractor G is updated. In the first stage, the objective function is improved for: That is, a balance factor λ is added to control the weight of the adversarial loss in the total loss, prevent gradient explosion, and stabilize training; X s represents the source domain data, Y s represents the source domain label, L adv (X t ) is the adversarial loss of the target domain, which is calculated as: Among them, X t represents the target domain data, Represents the target domain data X t Medium x t The expected value of d(p1(y|x t ),p2(y|x t )) are two probability distributions p1(y|x t ) and p2(y|x t ), p1(y|x t ) and p2(y|x t ) represent the classifier in a given target domain sample x t When , the conditional probability distribution of the output category y; The uncertainty of the model is measured by comparing the outputs of two classifiers p1 and p2 on the target domain data; d(p1, p2) is the distance between the outputs of the two classifiers, which is calculated as: Among them, K represents the number of categories, p 1k represents the probability of the first probability distribution p1 in the kth category, p 2k Represents the probability of the second probability distribution p2 in the kth category; Get the prediction difference between the two classifiers for each category k on the target domain data.

6. A gait recognition method based on surface electromyography and transfer learning as claimed in claim 3, characterized in that: In step 3, during the transfer learning training, the model uses Focal Loss as the loss function. Among them, y true represents the true label, y pred Represents the model prediction label, N represents the number of samples, represents the true label of the i-th sample, k represents the category to which the sample belongs, and K represents the number of categories. represents the probability of the kth category predicted by the model in the i-th sample; It represents the weight used to reduce the loss of easy-to-classify samples while amplifying the loss of difficult-to-classify samples. α represents the weight of the balanced category. γ represents the hyperparameter of the adjustment factor, which is used to adjust the decay rate of the loss and increase the attention to difficult-to-classify samples. The training process of the model is optimized by adjusting the size of these two values.

Images