A Lower Limb Movement Recognition Method Based on CNN-KELM

Abstract

The present invention relates to a lower limb movement recognition method based on CNN-KELM, which includes the following steps: Step S1, collect electromyography signals; Step S2, preprocess the collected electromyography signals; Step S3, construct a CNN-KELM recognition and classification model, and the CNN-KELM recognition and classification model includes an input layer, a feature extraction layer based on a CNN network, and a classification output layer based on KELM; input the preprocessed electromyography signals into the CNN-KELM recognition and classification model, the input layer forms an input matrix from the preprocessed electromyography signals and inputs it into the feature extraction layer for feature extraction, and then inputs the extracted feature signals into the classification output layer for classification of lower limb movements; Step S4, output the classification result. This method is beneficial to improving the recognition accuracy of lower limb movements.

Description

Technical Field

[0001] The present invention belongs to the technical field of motion pattern classification, and particularly relates to a lower limb motion recognition method based on CNN-KELM. Background Art

[0002] With the development of artificial intelligence algorithms, applying pattern recognition algorithms to motion pattern classification has become a trend. For example, the Support Vector Machine (SVM), which is a linear classifier on the feature plane and a supervised learning method that stretches low-dimensional data into high-dimensional data for classification. Ding Yu et al. used SVM to classify four types of actions: wrist flexion, wrist extension, finger extension, and grasping, achieving good results. However, SVM has the disadvantages of long training time and being prone to falling into local optima. The Back Propagation Neural Network (BP-NN) propagates experimental data forward and uses errors for backpropagation (the most commonly used is to use the gradient descent method for error backpropagation). However, when it is used for motion pattern classification, it has the disadvantages of cumbersome setting of hyperparameters and spending a lot of time on error backpropagation.

[0003] The Convolutional Neural Network (CNN) is composed of neurons with learnable weights and biases. Each neuron receives some inputs, performs a dot product, and selectively follows it non-linearly. A typical convolutional neural network consists of an input layer, a convolutional layer, a downsampling layer (pooling layer), a fully connected layer, and an output layer. Ronao et al. proposed using CNN to recognize patterns such as walking and going up and down stairs, and the effect is better than that of SVM. However, CNN has the disadvantage that it has strong feature extraction ability but poor classification ability.

[0004] Extreme Learning Machines (ELM) is a training method for Single-hidden Layer Feedforward Neural Network (SLFN), proposed by Huang Guangbin. Since it directly uses the Moore-Penrose generalized inverse matrix method to solve the pseudo-inverse matrix, the weight parameters can be directly solved. Compared with BP-NN, it has a faster training speed. However, the initial value selection of the weight parameters and the setting of the number of hidden layer neurons will have a greater impact on the classification results. To solve the problem that during the training process, due to the random acquisition of input weights and neuron thresholds, the output weights of some hidden layer neurons are too small. Huang Guangbin proposed the Incremental Extreme Learning Machine (I-ELM). This method modifies the number of hidden layer neurons by restricting the number of hidden layer neurons and the expected error (the square root of the mean square of the residual error) as the discriminant condition. However, it does not solve the problem of inconsistent classification results caused by randomly initializing the weight parameters. The Kernel Based Extreme Learning Machine (KELM) introduces the regularization coefficient and the kernel function into ELM. This method can simultaneously solve problems such as poor classification result stability caused by randomly selecting weight parameters and the number of hidden layer neurons in ELM. However, the KELM has many network parameters, and the selection of parameters affects the learning speed, generalization ability, accuracy, recall rate, etc. of the network. Simple and single selection cannot balance the fitting performance and generalization performance of the algorithm, easily resulting in a low accuracy of the generated classification model and unstable lower limb movement recognition results.

[0005] In the process of motion classification, methods of time domain, frequency domain, and nonlinear analysis are generally used to extract the time domain, frequency domain, time-frequency domain, and nonlinear features of sEMG signals. However, these methods have poor feature extraction capabilities, and the extracted myoelectric information contains insufficient features, resulting in a decline in classification performance. Summary of the Invention

[0006] The purpose of the present invention is to provide a lower limb movement recognition method based on CNN-KELM, which is beneficial to improving the recognition accuracy of lower limb movements.

[0007] To achieve the above purpose, the technical solution adopted by the present invention is: A lower limb movement recognition method based on CNN-KELM, including the following steps:

[0008] Step S1, collect myoelectric signals;

[0009] Step S2: Preprocess the collected electromyography (EMG) signals.

[0010] Step S3: Construct a CNN-KELM recognition and classification model, where the CNN-KELM recognition and classification model includes an input layer, a feature extraction layer based on a CNN network, and a classification output layer based on KELM. Input the preprocessed EMG signals into the CNN-KELM recognition and classification model. The input layer forms an input matrix from the preprocessed EMG signals and inputs it into the feature extraction layer for feature extraction, and then inputs the extracted feature signals into the classification output layer for classification of lower limb movements.

[0011] Step S4: Output the classification result.

[0012] Further, in step S1, collect the surface EMG signals of the rectus femoris, vastus medialis, vastus lateralis, semitendinosus, biceps femoris, and gastrocnemius. Record the sEMG signals of the six muscles under five movement modes: walking on flat ground, sitting knee extension, standing knee flexion, ascending stairs, and descending stairs, and record the corresponding labels.

[0013] Further, in step S2, use a normalization method to preprocess the collected EMG signals column by column.

[0014] The normalization formula for the d-th column is:

[0015]

[0016] In the formula, d represents the d-th column, q represents the n-th row, q = 1, 2, …, n, n represents the maximum number of rows, y dq is the sEMG signal data of the q-th row in the d-th column after normalization processing, x dq is the sEMG signal data of the q-th row in the d-th column before normalization processing, represents the minimum value in the d-th column of the sEMG signal matrix, represents the maximum value in the d-th column of the signal matrix.

[0017] Further, in step S3, the CNN network, namely the convolutional neural network, includes multiple convolutional layers and pooling layers arranged alternately. The convolutional layer is the core part of the CNN network, and uses the regular movement of the convolutional kernel to extract features from the sEMG signal matrix. The calculation formula of the convolutional layer is:

[0018]

[0019] In the formula, M j represents the set of sEMG signal matrices input into the convolutional layer, f is a non-linear function; k ij mdenotes the convolution kernel connecting the $i$-th sEMG signal matrix in the $m - 1$ layer and the $j$-th sEMG signal matrix in the $m$ layer; $b$ j m is the bias;

[0020] A pooling layer is set between consecutive convolutional layers for data and parameter compression to reduce the data dimension; to reduce the impact of overfitting, the average pooling is adopted in the pooling layer to maintain the matrix data information effectively to the greatest extent and suppress the main edge data;

[0021] Let the $m$-th layer be the pooling layer, then the output formula of the pooling layer is:

[0022]

[0023] where $pool(·)$ is the pooling function; $\omega$ j m is the weight; $b$ j m is the bias;

[0024] Let the $i$-th layer be the fully connected output layer, and the $j$-th sEMG signal feature matrix is:

[0025] $x$ m $= f(\beta$ m $\cdot v + b$ m )

[0026] where $v$ is the sEMG signal matrix output by the pooling layer in the $m - 1$ layer; $\beta$ m is the weight; $b$ m is the bias;

[0027] In the said CNN network, the forward propagation uses the above formula to obtain the network output value, and the Adam optimization backpropagation is adopted for the backpropagation.

[0028] Furthermore, the feature extraction method of the CNN network is:

[0029] First, the training set is fed into the CNN network for training, then the trained CNN network is saved, and then the training set and the test set are input into the trained CNN network to extract features, forming the training set features and the test set features for the classification output layer processing.

[0030] Further, when the signal matrix is input with 6×600, it first passes through a convolutional layer of 2×101 with a stride of 1×1, no padding, 12 convolutional layers, and an output layer size of 5×500; then through an average pooling layer of 1×20 with no padding and a stride of 1×20, and after calculation, the output layer size is 5×25; then through a convolutional layer of 5×6 with a stride of 1×1, no padding, 6 convolutional layers, and an output layer size of 1×20; further average pooling processing, with a pooling layer size of 1×2, no padding, and a stride of 1×2, and after calculation, the output layer size is 1×10; finally, the output layer is converted into a feature vector of 60×1 and input into the classification output layer for recognition; where the activation function is set to the Relu function, optimized using Adam, and at the same time, Dropout with a hyperparameter of 0.5 is added to delete some neurons to prevent overfitting during training, the learning rate is set to 0.01, and the number of iterations is set to 200.

[0031] Further, in step S3, the KELM, that is, the kernel extreme learning machine, introduces a kernel function on the basis of ELM to solve the problem of poor classification result stability caused by randomly selecting weight parameters and the number of hidden layer neurons in ELM; the construction method of the KELM initial model is as follows:

[0032] Initialize the connection weights between the input layer and the hidden layer and the thresholds of the hidden layer neurons, set the number of neurons in the hidden layer, select the radial basis function RBF as the kernel function of KELM, and build the KELM initial model;

[0033] β = H + t i

[0034] In the formula, H + is the Moore-Penrose generalized inverse matrix of matrix H, and t i is the type of the sample;

[0035] The KELM initial model uses the RBF radial basis function:

[0036]

[0037] Therefore, the output function of the KELM initial model is:

[0038]

[0039] In the formula: K(c, c1) is the kernel function, and (c1, c2…c m ) is each column data of the sEMG feature matrix output by the feature extraction layer based on the CNN network, m is the number of rows (dimension) of the sEMG data matrix, C is the penalty parameter, I is the identity matrix, and Ω KELMis the kernel matrix, and T is the matrix composed of the class label vectors of the training samples.

[0040] Furthermore, the penalty parameter of the KELM is set to 20, and the kernel parameter is set to 1.

[0041] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a lower limb movement recognition method based on CNN-KELM, which combines the advantages of CNN's autonomous feature extraction and KELM's fast training speed and good generalization performance, and makes full use of the advantages of these two networks for lower limb classification. Compared with the traditional lower limb classification learning algorithm, the present invention can extract deeper features and greatly improve the recognition accuracy, which has important application significance in the field of myoelectric signal recognition. Brief Description of the Drawings

[0042] Figure 1 is the flowchart of the method implementation of the embodiment of the present invention.

[0043] Figure 2 is the architecture diagram of the CNN-KELM recognition and classification model in the embodiment of the present invention.

[0044] Figure 3 is the instance diagram of the CNN-KELM recognition and classification model in the embodiment of the present invention. Detailed Embodiments

[0045] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are all exemplary and are intended to provide further descriptions of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0048] As Figure 1 shown, this embodiment provides a lower limb movement recognition method based on CNN-KELM, including the following steps:

[0049] Step S1, collect myoelectric signals.

[0050] In this embodiment, the surface electromyography signals of the rectus femoris, vastus medialis, vastus lateralis, semitendinosus, biceps femoris, and gastrocnemius are collected; the sEMG signals of the six muscles under five motion modes of walking on flat ground, knee extension while sitting, knee flexion while standing, ascending stairs, and descending stairs are respectively recorded, and the corresponding labels are recorded.

[0051] Step S2: Preprocess the collected electromyography signals.

[0052] In this embodiment, the collected electromyography signals are preprocessed column by column using the normalization method;

[0053] The normalization formula for the d-th column is:

[0054]

[0055] where d represents the d-th column, q represents the n-th row, q = 1, 2,... n, n represents the maximum number of rows, y dq is the sEMG signal data of the q-th row in the d-th column after normalization processing, x dq is the sEMG signal data of the q-th row in the d-th column before normalization processing, represents the minimum value in the d-th column of the sEMG signal matrix, the maximum value in the d-th column of the signal matrix.

[0056] Step S3: Construct a CNN-KELM recognition and classification model as Figure 2 shown. The CNN-KELM recognition and classification model includes an input layer, a feature extraction layer based on a CNN network, and a classification output layer based on KELM; the preprocessed electromyography signals are input into the CNN-KELM recognition and classification model. The input layer forms an input matrix from the preprocessed electromyography signals and inputs it into the feature extraction layer for feature extraction, and then the extracted feature signals are input into the classification output layer for classification of lower limb movements.

[0057] In this embodiment, the input layer forms a 6×600 input matrix from the preprocessed electromyography signals and inputs it into the feature extraction layer based on the CNN network for feature extraction.

[0058] A convolutional neural network is an artificial neural network that can handle high-dimensional data well. It is a good feature extractor but not the best classifier. It is usually applied in image recognition, video recognition, and text classification.

[0059] As Figure 2As shown, the CNN network (i.e., convolutional neural network) takes the sEMG signal matrix as input to further extract deeper features of the electromyography signal. The CNN network includes multiple convolutional layers and pooling layers arranged alternately. The convolutional layer is the core part of the CNN network, and it extracts features from the sEMG signal matrix by the regular movement of the convolutional kernel.

[0060] The calculation formula of the convolutional layer is:

[0061]

[0062] In the formula, M j represents the set of sEMG signal matrices input to the convolutional layer, f is a non-linear function (in this embodiment, the Relu function is used); k ij m represents the convolutional kernel connecting the i-th sEMG signal matrix in the m-1 layer and the j-th sEMG signal matrix in the m layer; b j m is the bias.

[0063] A pooling layer is set between consecutive convolutional layers for data and parameter compression to reduce the data dimension; to reduce the impact of overfitting, the pooling layer uses average pooling (calculating the average value of the image area as the value of the area after pooling) to maintain the effective matrix data information to the greatest extent and suppress the main edge data.

[0064] Suppose the m-th layer is the pooling layer (subsampling layer), then the output formula of the pooling layer is:

[0065]

[0066] In the formula, pool(·) is the pooling function (average pooling is used in the present invention); ω j m is the weight; b j m is the bias.

[0067] Suppose the i-th layer is the fully connected output layer, and the j-th sEMG signal feature matrix is:

[0068] x m = f(β m ·v + b m )

[0069] In the formula, v is the sEMG signal matrix output by the pooling layer of the m-1 layer; β m is the weight; b m is the bias.

[0070] In the CNN network, there are forward propagation and backward propagation. Forward propagation is used to obtain the network output value using the above formula, and backward propagation uses Adam optimization for backward propagation, with Dropout set to 0.5.

[0071] Figure 3 An example of the CNN-KELM recognition and classification model in this embodiment is shown. As Figure 3 shown, when the signal matrix is input with 6×600, it first passes through a convolutional layer of 2×101, with a stride set to 1×1, no padding, 12 convolutional layers, and an output layer size of 5×500; then it passes through an average pooling layer of 1×20, no padding, a stride set to 1×20, and after calculation, the output layer size is 5×25; then it passes through a convolutional layer of 5×6, with a stride set to 1×1, no padding, 6 convolutional layers, and an output layer size of 1×20; further average pooling processing is performed, with a pooling layer size of 1×2, no padding, a stride set to 1×2, and after calculation, the output layer size is 1×10; finally, the output layer is converted into a feature vector of 60×1 and input into the classification output layer for recognition; among them, the activation function is set to the Relu function, Adam is used for optimization, and at the same time, Dropout with a hyperparameter of 0.5 is added to delete some neurons to prevent overfitting during the training process, the learning rate is set to 0.01, and the number of iterations is set to 200.

[0072] The KELM (i.e., kernel extreme learning machine) solves the problem of poor stability of classification results caused by randomly selecting weight parameters and the number of hidden layer neurons in ELM by introducing a kernel function on the basis of ELM, and has good generalization ability and fast training speed. The construction method of the KELM initial model is as follows:

[0073] Initialize the connection weights between the input layer and the hidden layer and the thresholds of the hidden layer neurons, set the number of neurons in the hidden layer, select the radial basis function RBF as the kernel function of KELM, and build the KELM initial model;

[0074] β = H + t i

[0075] In the formula, H + is the Moore-Penrose generalized inverse matrix of matrix H, and t i is the type of the sample;

[0076] The KELM initial model uses the RBF radial basis function:

[0077]

[0078] Therefore, the output function of the KELM initial model is:

[0079]

[0080] Where: K(c,c1) is the kernel function, (c1,c2…c m ) is each column of the sEMG feature matrix output by the feature extraction layer based on the CNN network, m is the number of rows (dimension) of the sEMG data matrix, C is the penalty parameter, I is the unit matrix, Ω KELM is the kernel matrix, and T is the matrix composed of the class mark vectors of the training samples.

[0081] In this embodiment, the penalty parameter of the KELM is set to 20, and the kernel parameter is set to 1.

[0082] The KLEM model has very high requirements for input features and is a very good classifier. CNN convolution calculation can enhance the depth of sEMG signal features, allowing KLEM to complete tasks faster and with higher recognition accuracy.

[0083] Step S4: output the classification result.

[0084] The classification accuracy (Accuracy), precision (Precision), recall rate (Recall) and F1-score of the KELM network, CNN network and the method of the present invention based on traditional features have been verified.

[0085] The method of the present invention has the highest precision, recall, F1-score and accuracy. The average values of the precision, recall and F1-score of the method of the present invention are 16.44%, 18.88% and 17.34% higher than those of the KELM network, and 9.98%, 9.95% and 10.03% higher than those of the CNN network. The accuracy of the method of the present invention is 18.29% higher than that of the KELM network based on traditional features, and 11.86% higher than that of the CNN network. This shows that the method of the present invention can make full use of the feature extraction capability of the CNN network, and the extracted features can reveal more information about the sEMG signal than traditional features, and can combine the generalization performance and classification accuracy of the KELM classifier.

[0086] Evaluation criteria for KELM network testing and classification based on traditional features

[0087]

[0088] Evaluation criteria for CNN network testing and classification

[0089]

[0090] Evaluation criteria for network testing and classification of the method of the present invention

[0091]

[0092] In summary, the present invention collects surface electromyogram (sEMG) signals through a non-invasive method. By combining the advantages of the adaptive feature extraction ability of the convolutional neural network (CNN) and the good generalization performance of the kernel extreme learning machine (KELM), deep features are extracted in the CNN and then brought into the KELM for solution. This can avoid the problems of long time consumption and inability to fully reveal the intrinsic properties of sEMG signals caused by manual feature extraction. The CNN-KELM model used in the present invention has the characteristics of high accuracy and strong stability in the process of classifying the lower limbs based on sEMG signals, and can be widely applied to fields such as medical devices and medical rehabilitation.

[0093] The above is only a preferred embodiment of the present invention, and it is not intended to limit the present invention in other forms. Any person skilled in the art may use the technical content disclosed above to make changes or modifications into equivalent embodiments with equivalent changes. However, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution content of the present invention still belong to the protection scope of the technical solution of the present invention.

Claims

1. A lower limb movement recognition method based on CNN-KELM, characterized in that, Including the following steps: Step S1, collecting electromyography signals; Step S2, preprocessing the collected electromyography signals; Step S3, constructing a CNN-KELM recognition and classification model, where the CNN-KELM recognition and classification model includes an input layer, a feature extraction layer based on a CNN network, and a classification output layer based on KELM; inputting the preprocessed electromyography signals into the CNN-KELM recognition and classification model, the input layer forms an input matrix from the preprocessed electromyography signals and inputs it into the feature extraction layer for feature extraction, and then inputs the extracted feature signals into the classification output layer for classifying lower limb movements; Step S4, outputting the classification result; In step S3, the CNN network, that is, the convolutional neural network, includes multiple convolutional layers and pooling layers arranged alternately; the convolutional layer is the core part of the CNN network, and uses the regular movement of the convolutional kernel to extract features from the sEMG signal matrix; the calculation formula of the convolutional layer is: Where, M j represents the set of sEMG signal matrices input to the convolutional layer, f is a non-linear function; k ij m represents the convolution kernel connecting the i-th sEMG signal matrix in the m-1 layer and the j-th sEMG signal matrix in the m-th layer; b j m is the bias; A pooling layer is set between consecutive convolutional layers for compressing data and parameters to reduce the data dimension; to reduce the influence of overfitting, the pooling layer uses average pooling to maintain the effective matrix data information to the greatest extent and suppress the main edge data; Let the mth layer be the pooling layer, then the output formula of the pooling layer is: where pool(·) is the pooling function; is the weight; is the bias; Let the ith layer be the fully connected output layer, and the jth sEMG signal feature matrix is: x m = f(β m ·v + b m ) Where, v is the sEMG signal matrix output by the pooling layer of the m-1 layer; β m is the weight; b m is the bias; In the CNN network, forward propagation uses the above formula to obtain the network output value, and backpropagation uses Adam optimized backpropagation; In step S3, the KELM, that is, the kernel extreme learning machine, introduces a kernel function on the basis of ELM to solve the problem of poor stability of the classification result caused by randomly selecting weight parameters and the number of hidden layer neurons in ELM; the construction method of the initial KELM model is: Initializing the connection weights between the input layer and the hidden layer and the thresholds of the hidden layer neurons, setting the number of neurons in the hidden layer, and selecting the radial basis function RBF as the kernel function of KELM to build the initial KELM model; β = H + t i Where, H + is the Moore-Penrose generalized inverse matrix of matrix H, and t i is the type of the sample; The initial KELM model uses the RBF radial basis function: So the output function of the initial KELM model is: Where: K(c, c1) is the kernel function, and (c1, c2…c m ) is each column data of the sEMG feature matrix output by the feature extraction layer based on the CNN network. m is the number of rows of the sEMG data matrix, that is, the dimension. C is the penalty parameter, I is the identity matrix, Ω KELM is the kernel matrix, and T is the matrix composed of the class label vectors of the training samples.

2. The lower limb motion recognition method based on CNN-KELM according to claim 1, wherein In step S1, collecting the surface electromyography signals of the rectus femoris, vastus medialis, vastus lateralis, semitendinosus, biceps femoris, and gastrocnemius; respectively recording the sEMG signals of the six muscles under five movement modes of walking on flat ground, sitting and extending the knee, standing and flexing the knee, going up stairs, and going down stairs, and recording the corresponding labels.

3. A lower limb movement recognition method based on CNN-KELM according to claim 1, characterized in that, In step S2, using a normalization method to preprocess the collected electromyography signals column by column; The normalization formula for the dth column is: where d represents the d-th column, q represents the n-th row, q = 1, 2, … n, n represents the maximum number of rows, and y dq is the sEMG signal data of the d-th column and q-th row after normalization, and x dq is the sEMG signal data of the d-th column and q-th row before normalization, represents the minimum value in the d-th column of the sEMG signal matrix, and the maximum value in the d-th column of the sEMG signal matrix.

4. A lower limb movement recognition method based on CNN-KELM according to claim 1, characterized in that In step S3, the input layer forms a 6×600 input matrix from the preprocessed electromyography signals and inputs it into the feature extraction layer based on the CNN network for feature extraction.

5. A lower limb movement recognition method based on CNN-KELM according to claim 1, characterized in that After the signal matrix is input with 6×600, it first passes through a convolutional layer of 2×101 with a stride set to 1×1, no padding, 12 convolutional layers, and an output layer size of 5×500; then through an average pooling layer of 1×20 with no padding and a stride set to 1×20, and the size of the output layer after calculation is 5×25; then through a convolutional layer of 5×6 with a stride set to 1×1, no padding, 6 convolutional layers, and an output layer size of 1×20; further average pooling processing, with a pooling layer size of 1×2, no padding, and a stride set to 1×2, and the size of the output layer after calculation is 1×10; finally, the output layer is converted into a feature vector of 60×1 and input into the classification output layer for recognition; the activation function is set to the Relu function, Adam is used for optimization, and at the same time, Dropout with a hyperparameter of 0.5 is added to delete some neurons to prevent overfitting during the training process, the learning rate is set to 0.01, and the number of iterations is set to 200.

6. The lower limb motion recognition method based on CNN-KELM according to claim 1, characterized in that The penalty parameter of the KELM is set to 20, and the kernel parameter is set to 1.

Images