An elbow joint motion pattern recognition method and device, readable storage medium and electronic device

Abstract

This invention discloses a method, device, and readable storage medium for elbow joint motion pattern recognition. The invention acquires high-density surface electromyography (EMG) signal data and performs data augmentation on the original signals. It then extracts the time-domain, frequency-domain, and time-frequency-domain features of the signals. A recognition model containing a two-layer LSTM, Dropout regularization, adaptive learning rate, Adam optimizer, and sparse classification cross-entropy loss function is constructed and trained to classify the signals to be recognized and output the motion pattern results. This invention relies on the deep modeling capability of a two-layer LSTM for time-series data, accurately mining the dynamic features of EMG signals, effectively distinguishing easily confused movements such as pronation and supination, achieving a recognition accuracy of 99.98% and an F1 score of 1.0. The model exhibits strong generalization ability and robustness, low overfitting risk, and adaptability to complex real-world acquisition environments. Its recognition accuracy and stability are significantly superior to traditional CNN methods.

Description

Technical Field

[0001] This invention relates to the fields of rehabilitation engineering and pattern recognition technology, and in particular to a method, device, readable storage medium, and electronic device for recognizing elbow joint movement patterns. Background Technology

[0002] Surface electromyography (sEMG) is a non-invasive bioelectrical signal that is typically generated before limb movement. Therefore, it is widely used to decode human movement intentions to achieve natural control of assistive devices such as rehabilitation robots and exoskeletons. Currently, there is considerable research on discrete movement pattern recognition for hands and other joints, but there is still room for improvement in high-precision discrete movement recognition for the elbow joint, a key joint in the field.

[0003] The elbow joint, with its four degrees of freedom (flexion, extension, pronation, and supination), is central to achieving fine motor skills in the upper limb. Existing research often analyzes the upper limb as a whole, or its accuracy in recognizing elbow joint movements needs improvement. Traditional machine learning methods, such as the K-nearest neighbor algorithm, have limited recognition rates and robustness when processing high-dimensional, nonlinear electromyographic signal temporal features.

[0004] Therefore, it is necessary to develop a method, device, readable storage medium, and electronic device for elbow joint motion pattern recognition to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to design an elbow joint motion pattern recognition method, device, readable storage medium, and electronic device to solve the above-mentioned problems.

[0006] The present invention achieves the above objectives through the following technical solutions: A method for recognizing elbow joint movement patterns includes the following steps: S1. Acquire high-density surface electromyography (EMG) signal data. The high-density surface EMG signal data is acquired by an electrode array placed on the corresponding muscle groups of elbow flexion, extension, pronation and supination when the subject performs various discrete movements including elbow flexion, extension, pronation and supination. S2. Perform data augmentation processing on the acquired raw high-density surface electromyography signal data to generate an enhanced signal that simulates the real acquisition environment; S3. Extract features from the enhanced signal. These features include time-domain features, frequency-domain features, and time-frequency-domain features. Time-domain features include mean, standard deviation, maximum value, minimum value, and root mean square. Frequency-domain features include frequency mean, frequency standard deviation, and frequency median. Time-frequency-domain features are extracted by performing wavelet transform on the enhanced signal. S4. Based on the features extracted in the previous step, construct and train a two-layer long short-term memory network model to recognize various discrete actions; S5. Input the high-density surface electromyography signal data of the elbow joint to be identified into the trained two-layer long short-term memory network model, and output the corresponding motion pattern recognition results. The motion pattern recognition results include elbow joint flexion, extension, pronation and supination.

[0007] Specifically, in step S2, the data augmentation process includes at least one of adding Gaussian noise to the original high-density surface electromyography signal data, performing amplitude scaling, and introducing baseline drift.

[0008] Furthermore, before extracting features from the enhanced signal, the process also includes segmenting the enhanced signal using a sliding window.

[0009] Furthermore, the sliding window has a window size of 256 sampling points and a step size of 128 sampling points.

[0010] Specifically, the two-layer long short-term memory network model includes: The first long short-term memory layer has 64 units and is used to output complete sequences; The first Dropout layer connects to the first long short-term memory layer; The second long short-term memory layer has 32 units and is connected to the first dropout layer to output the final state; The second Dropout layer connects to the second long short-term memory layer; Fully connected layer; Number of neurons: 4, corresponding to the 4 types of elbow joint movements, serving as the output layer; The Softmax activation function, which connects fully connected layers, is used to output the probability of the corresponding elbow joint flexion, extension, pronation, and supination movement categories.

[0011] Furthermore, the two-layer long short-term memory network model uses the Adam optimizer with an initial learning rate of 0.001, a loss function of sparse classification cross-entropy, and a batch size of 32; the dataset is divided into training, validation, and test sets in a 6:2:2 ratio.

[0012] Furthermore, an adaptive learning rate adjustment strategy is introduced during training: if the validation set loss does not decrease within 5 consecutive rounds, the learning rate is halved.

[0013] An elbow joint motion pattern recognition device, comprising: High-density surface electromyography signal data acquisition module; used to acquire high-density surface electromyography signal data, which is collected by an electrode array placed on the corresponding muscle groups of elbow flexion, extension, pronation and supination when the subject performs various discrete movements including elbow flexion, extension, pronation and supination. Data augmentation processing module; used to perform data augmentation processing on the acquired raw high-density surface electromyography signal data to generate enhanced signals that simulate the real acquisition environment; The feature extraction module is used to extract features from the enhanced signal. These features include time-domain features, frequency-domain features, and time-frequency-domain features. Time-domain features include mean, standard deviation, maximum value, minimum value, and root mean square. Frequency-domain features include frequency mean, frequency standard deviation, and frequency median. Time-frequency-domain features are extracted by performing wavelet transform on the enhanced signal. Model building module; used to build and train a two-layer long short-term memory network model based on extracted features to recognize various discrete actions; The motion pattern recognition module is used to input the high-density surface electromyography signal data of the elbow joint to be identified into a trained two-layer long short-term memory network model, and output the corresponding motion pattern recognition results, which include elbow joint flexion, extension, pronation and supination.

[0014] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the steps of an elbow joint motion pattern recognition method.

[0015] An electronic device includes: one or more processors; and a memory associated with the one or more processors, the memory storing program instructions that, when read and executed by the one or more processors, perform the steps of an elbow joint motion pattern recognition method.

[0016] The beneficial effects of this invention are: High recognition accuracy: This invention uses a dual-layer long short-term memory network (dual-layer LSTM) to fully leverage its strong modeling ability for time-series data. It can deeply mine the dynamic features of high-density surface electromyography (HD-sEMG) signals and achieve accurate recognition of four movements of the elbow joint: flexion, extension, pronation, and supination. The recognition accuracy is as high as 99.98%, and the F1 score reaches 1.0. The recognition performance is significantly better than that of traditional CNN models.

[0017] Excellent model stability and generalization ability: This invention constructs a two-layer long short-term memory network model, with regularization processing of the first and second dropout layers, and adopts an adaptive learning rate adjustment strategy. Combined with the Adam optimizer and sparse classification cross-entropy loss function, the loss function decreases smoothly during model training, and the accuracy of the validation set is steadily improved. This can effectively prevent overfitting and greatly improve the model stability and generalization ability.

[0018] The method of this invention has outstanding ability to distinguish easily confused movements: For movements such as elbow pronation and supination with similar signal features that are easily confused, the method of this invention relies on the deep mining ability of temporal features by a two-layer long short-term memory network to accurately distinguish and effectively identify them, thus solving the problem of low accuracy in the identification of similar movements by traditional methods.

[0019] Strong robustness in practical applications: In the data preprocessing stage, this invention performs data augmentation processing such as adding Gaussian noise, amplitude scaling, and baseline drift on the original high-density surface electromyography signals to simulate signal interference and changes in the real acquisition environment, thereby improving the model's adaptability to complex real-world scenarios and making the model more robust in real-world applications. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of high-density surface electromyography signal measurement in an embodiment of the present invention; Figure 2 This is a schematic diagram of the electromyographic signals extracted in an embodiment of the present invention; Figure 3 This is an accuracy-loss curve of the CNN model in an embodiment of the present invention; where A is the accuracy-round curve and B is the loss-round curve. Figure 4 This is a confusion matrix diagram of the CNN model in an embodiment of the present invention; Figure 5 This is an accuracy-loss curve of the LSTM model in this embodiment of the invention; where A is the accuracy-round curve and B is the loss-round curve. Figure 6 This is a confusion matrix diagram of the LSTM model in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0024] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0025] Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0026] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, terms such as "set" and "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0027] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] A method for recognizing elbow joint movement patterns includes the following steps: S1. Acquire high-density surface electromyography (sEMG) signal data. This data is collected by an electrode array positioned on the corresponding muscle groups of the elbow joint during various discrete movements, including elbow flexion, extension, pronation, and supination. This embodiment utilizes the publicly available HD-sEMG dataset (M. Rojas-Martínez, LY Serna, M. Jordanic, et al., “High-density surface electromyography signals during isometric contractions of elbow muscles of healthy humans,” Sci. Data, vol. 7, p. 397, 2020.). This dataset contains data from 12 healthy subjects (aged 23-34 years), recording high-density surface electromyography signals collected from the biceps brachii, triceps brachii, and forearm muscle groups during four isometric contraction tasks involving elbow flexion, extension, pronation, and supination. Data acquisition uses an electrode matrix with a sampling frequency of 2048 Hz. Figure 1 and Figure 2 As shown.

[0029] S2. Perform data augmentation processing on the acquired raw high-density surface electromyography (EMG) signal data to generate an enhanced signal that simulates the real acquisition environment; the data augmentation processing includes adding Gaussian noise to the raw high-density surface EMG signal data, performing amplitude scaling, and introducing baseline drift.

[0030] The specific corresponding functions are as follows: Add Gaussian noise: to simulate electronic device noise and environmental noise.

[0031] Amplitude scaling: Simulates changes in the intensity of muscle contraction.

[0032] Baseline drift: Introducing low-frequency waveforms to simulate signal drift caused by sweating or electrode displacement.

[0033] After data augmentation, the signal is segmented using a sliding window with a size of 256 sampling points and a step size of 128 sampling points.

[0034] S3. Extract features from the enhanced signal. Here, features are extracted for each signal window. Features include time-domain features, frequency-domain features, and time-frequency-domain features. Time-domain features include mean, standard deviation, maximum value, minimum value, and root mean square. Frequency-domain features include frequency mean, frequency standard deviation, and frequency median. Time-frequency-domain features are extracted by performing wavelet transform on the enhanced signal. Wavelet transform is applied to extract features such as energy distribution at different scales to more comprehensively characterize the time-varying properties of non-stationary electromyographic signals. S4. Based on the features extracted in the previous step, construct and train a two-layer long short-term memory network model to recognize various discrete actions; Two-layer long short-term memory network models include: The first Long Short-Term Memory (LSTM) layer contains 64 units and has return_sequences=True set to output the complete sequence to the next layer. The first Dropout layer connects to the first Long Short-Term Memory layer; the first Dropout layer has a dropout rate of 0.3 to prevent overfitting. The second Long Short-Term Memory (LSTM) layer contains 32 units. With return_sequences=False set, it only outputs the final state. A second Dropout layer connected to the second long short-term memory layer; dropout rate is 0.3. Fully connected layer; Number of neurons: 4, corresponding to the 4 types of elbow joint movements, serving as the output layer; The Softmax activation function, which connects fully connected layers, is used to output the probability of the corresponding elbow joint flexion, extension, pronation, and supination movement categories.

[0035] The two-layer long short-term memory network model uses the Adam optimizer, with the loss function being sparse classification cross-entropy and a batch size of 32. The dataset is divided into training, validation, and test sets in a 6:2:2 ratio.

[0036] During training, the maximum number of training epochs was set to 50. The Adam optimizer was used with an initial learning rate of 0.001. An adaptive learning rate adjustment strategy was introduced: if the validation set loss did not decrease within 5 consecutive epochs, the learning rate was halved (the minimum learning rate was not lower than 0.0001).

[0037] Perform model performance evaluation: Evaluate the trained model using a test set. For example... Figure 5 , Figure 6 As shown in Table 1, the LSTM model performs excellently: Table 1: LSTM Classification Results

[0038] Accuracy: The test accuracy reached 99.98%, with a loss value of only 0.0005.

[0039] Confusion matrix: Of all the test samples, only 3 samples were misclassified, mainly due to confusion between rotation and buckling.

[0040] Classification report: For the four categories of flexion, extension, pronation, and supination, the model’s precision, recall, and F1 score are all 1.00 (99.98%, 1.00 when rounded to two decimal places).

[0041] S5. Here, the model is applied: the high-density surface electromyography signal data of the elbow joint to be identified is input into the trained two-layer long short-term memory network model, and the corresponding motion pattern recognition results are output. The motion pattern recognition results include elbow joint flexion, extension, pronation and supination.

[0042] An elbow joint motion pattern recognition device, comprising: High-density surface electromyography signal data acquisition module; used to acquire high-density surface electromyography signal data, which is collected by an electrode array placed on the corresponding muscle groups of elbow flexion, extension, pronation and supination when the subject performs various discrete movements including elbow flexion, extension, pronation and supination. Data augmentation processing module; used to perform data augmentation processing on the acquired raw high-density surface electromyography signal data to generate enhanced signals that simulate the real acquisition environment; The feature extraction module is used to extract features from the enhanced signal. These features include time-domain features, frequency-domain features, and time-frequency-domain features. Time-domain features include mean, standard deviation, maximum value, minimum value, and root mean square. Frequency-domain features include frequency mean, frequency standard deviation, and frequency median. Time-frequency-domain features are extracted by performing wavelet transform on the enhanced signal. Model building module; used to build and train a two-layer long short-term memory network model based on extracted features to recognize various discrete actions; The motion pattern recognition module is used to input the high-density surface electromyography signal data of the elbow joint to be identified into a trained two-layer long short-term memory network model, and output the corresponding motion pattern recognition results, which include elbow joint flexion, extension, pronation and supination.

[0043] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the steps of an elbow joint motion pattern recognition method.

[0044] An electronic device includes: one or more processors; and a memory associated with the one or more processors, the memory storing program instructions that, when read and executed by the one or more processors, perform the steps of an elbow joint motion pattern recognition method.

[0045] Comparative Example 1 Following the same dataset and processing flow as Example 1, a CNN model with a dual convolutional block structure was constructed for comparative experiments. The structure of the CNN model is as follows: two convolutional blocks (32→64 3x1 filters, ReLU activation), each convolutional block is followed by a batch normalization and max pooling layer, and finally connected to a global average pooling layer and a 32-neuron fully connected layer (Dropout 0.3), with the classification result output by a Softmax layer.

[0046] The results are as follows Figure 3 , Figure 4 As shown in Table 2, the CNN model achieved a test accuracy of 97.43%, but its recognition of spin actions was relatively weak (F1 score of 0.96), and the accuracy of the validation set fluctuated significantly during training.

[0047] Table 2: CNN Classification Results

[0048] The comparison results show that the LSTM model proposed in this invention is significantly better than the CNN model in terms of recognition accuracy, training stability, and ability to recognize easily confused actions.

[0049] Comparative Example 2 The results of this invention are compared with traditional KNN-based methods (such as those studied by Rahman et al.). The comparison results are shown in Table 3. This invention demonstrates significant advantages in terms of dataset size, signal type richness, feature dimension (fusion of time domain, frequency domain, and time-frequency domain), and final classification performance (99.98% vs 89.68%), filling the gap in high-precision discrete motion recognition of the elbow joint in a laboratory environment.

[0050] Table 3: Comparison between the present invention and traditional methods:

[0051] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for recognizing elbow joint movement patterns, characterized in that, Including the following steps: S1. Acquire high-density surface electromyography (EMG) signal data. The high-density surface EMG signal data is acquired by an electrode array placed on the corresponding muscle groups of elbow flexion, extension, pronation and supination when the subject performs various discrete movements including elbow flexion, extension, pronation and supination. S2. Perform data augmentation processing on the acquired raw high-density surface electromyography signal data to generate an enhanced signal that simulates the real acquisition environment; S3. Extract features from the enhanced signal, including time-domain features, frequency-domain features, and time-frequency-domain features; Time-domain features include mean, standard deviation, maximum value, minimum value, and root mean square; frequency-domain features include frequency mean, frequency standard deviation, and frequency median; time-frequency domain features are extracted by wavelet transform of the enhanced signal. S4. Based on the features extracted in the previous step, construct and train a two-layer long short-term memory network model to recognize various discrete actions; S5. Input the high-density surface electromyography signal data of the elbow joint to be identified into the trained two-layer long short-term memory network model, and output the corresponding motion pattern recognition results. The motion pattern recognition results include elbow joint flexion, extension, pronation and supination.

2. The elbow joint motion pattern recognition method according to claim 1, characterized in that, In step S2, the data augmentation process includes at least one of adding Gaussian noise to the original high-density surface electromyography signal data, performing amplitude scaling, and introducing baseline drift.

3. The elbow joint motion pattern recognition method according to claim 1, characterized in that, Before extracting features from the enhanced signal, the process also includes segmenting the enhanced signal using a sliding window.

4. The elbow joint motion pattern recognition method according to claim 3, characterized in that, The sliding window has a window size of 256 sampling points and a step size of 128 sampling points.

5. The elbow joint motion pattern recognition method according to claim 1, characterized in that, Two-layer long short-term memory network models include: The first long short-term memory layer has 64 units and is used to output complete sequences; The first Dropout layer connects to the first long short-term memory layer; The second long short-term memory layer has 32 units and is connected to the first dropout layer to output the final state; The second Dropout layer connects to the second long short-term memory layer; And the output layer that connects to the second Dropout layer. The output layer uses the Softmax activation function to output the probability of the corresponding elbow joint flexion, extension, pronation and supination movement categories; Fully connected layer; Number of neurons: 4, corresponding to the 4 types of elbow joint movements, serving as the output layer; The Softmax activation function, which connects fully connected layers, is used to output the probability of the corresponding elbow joint flexion, extension, pronation, and supination movement categories.

6. The elbow joint motion pattern recognition method according to claim 5, characterized in that, The two-layer long short-term memory network model uses the Adam optimizer with an initial learning rate of 0.001, a loss function of sparse classification cross-entropy, and a batch size of 32. The dataset is divided into training, validation, and test sets in a 6:2:2 ratio.

7. The elbow joint motion pattern recognition method according to claim 6, characterized in that, An adaptive learning rate adjustment strategy is introduced during training: if the validation set loss does not decrease within 5 consecutive rounds, the learning rate is halved.

8. A device for recognizing elbow joint movement patterns, characterized in that, include: High-density surface electromyography signal data acquisition module; used to acquire high-density surface electromyography signal data, which is collected by an electrode array placed on the corresponding muscle groups of elbow flexion, extension, pronation and supination when the subject performs various discrete movements including elbow flexion, extension, pronation and supination. Data augmentation processing module; Used to perform data augmentation processing on the acquired raw high-density surface electromyography signal data to generate enhanced signals that simulate the real acquisition environment; Feature extraction module; Used to extract features from enhanced signals, including time-domain features, frequency-domain features, and time-frequency-domain features; Time-domain features include mean, standard deviation, maximum value, minimum value, and root mean square; frequency-domain features include frequency mean, frequency standard deviation, and frequency median; time-frequency domain features are extracted by wavelet transform of the enhanced signal. Model building module; used to build and train a two-layer long short-term memory network model based on extracted features to recognize various discrete actions; Motion pattern recognition module; This is used to input the high-density surface electromyography (EMG) signal data of the elbow joint to be identified into a trained two-layer long short-term memory network model, and output the corresponding motion pattern recognition results, which include elbow joint flexion, extension, pronation, and supination.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps of the elbow joint motion pattern recognition method according to any one of claims 1-7.

10. An electronic device, characterized in that, include: One or more processors; And a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the elbow joint motion pattern recognition method according to any one of claims 1-7.

Images