A method and system for predicting human hand movements based on surface electromyography signals

Abstract

This invention discloses a method and system for predicting human hand movements based on surface electromyography (EMG) signals, belonging to the field of hand movement prediction technology. First, the temporal characteristics of the EMG signals are obtained; then, these temporal characteristics are input into a cross-individual lifelong network model to obtain the predicted human hand movements. During the training process of the cross-individual lifelong network model, a model structure and training method that can be adapted to multiple individuals simultaneously in a single training session are proposed. A lifelong learning strategy is designed, and the model is adapted to these adaptations, enabling the method to quickly learn new individuals while maintaining the model's performance on already trained individuals, effectively solving the catastrophic forgetting problem of the model.

Description

Technical Field

[0001] This invention relates to the field of hand movement prediction technology, and in particular to a method and system for predicting human hand movements based on surface electromyography signals using a lifelong learning strategy. Background Technology

[0002] In recent years, with the rapid development of robotics, smart devices, and the internet, the demand for natural, accurate, and robust interaction between humans and smart devices has been increasing. This demand is particularly evident in human-computer interaction scenarios, such as smart homes, industrial production, active prosthetics, machine-assisted surgery, and drone reconnaissance. These fields urgently require easily accessible, user-friendly, and high-quality interaction signals. To meet these requirements, the field is constantly exploring new technologies and signals to enhance the human-computer interaction experience. Among these, surface electromyography (sEMG) signals have attracted considerable attention, playing a crucial role in extracting precise features from biosignals and translating them into control commands to achieve high accuracy and efficiency.

[0003] However, existing methods for training neural networks using deep learning inevitably face the challenge of catastrophic forgetting, meaning that when a model learns a new task, it forgets the previously learned tasks. This phenomenon occurs because during model parameter updates, new data overwrites or influences old data, leading to a decline in the performance of older tasks. Therefore, there is an urgent need in this field for a technical solution that enables lifelong learning. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for predicting human hand movements based on surface electromyography signals, which uses lifelong learning methods to solve the problem of catastrophic forgetting of models.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for predicting human hand movements based on surface electromyography signals, the method comprising:

[0007] Acquire the temporal characteristics of surface electromyography signals;

[0008] The temporal features are input into a cross-individual lifetime network model to obtain human hand movement prediction results;

[0009] The training process of the cross-individual lifelong network model includes:

[0010] The temporal characteristics of the first surface electromyography signal are obtained to obtain the first temporal characteristics;

[0011] Using the first temporal feature as input and the motion sequence as output, a cross-individual lifetime network is trained to obtain the first cross-individual lifetime network model;

[0012] Freeze the first parameters of the first cross-individual lifetime network model; the first parameters are all parameters except for the classifier parameters and the decoder parameters.

[0013] The temporal characteristics and the number of second surface electromyographic signals are obtained to obtain the second temporal characteristics and the number of second signals;

[0014] Update the FC layer in the classifier according to the number of the second signal, and add the corresponding number of Adapter structures in the regression branch;

[0015] Training is performed using the second loss function to obtain a cross-individual lifetime network model; the second loss function is: L lifelong =L r +L' rec +λL' c .

[0016] Optionally, after training according to the second loss function to obtain the cross-individual lifetime network model, the method further includes:

[0017] New data observed during this training task is sampled and stored in memory.

[0018] Optionally, after updating the FC layer in the classifier according to the number of the second signals and adding a corresponding number of Adapter structures in the regression branch, the method further includes:

[0019] The temporal features of the second surface electromyography signal and the training data in the first temporal features are randomly sampled and mixed according to the proportion of individuals to the total number of individuals.

[0020] Optionally, the acquisition of the temporal characteristics of the surface electromyography signal specifically includes:

[0021] The temporal features of surface electromyography (EMG) signals are extracted using the root mean square feature method. The calculation formula is as follows:

[0022]

[0023] sEMG i Let N be the electromyographic signal value of the i-th sampling point in the input window, N be the capacity of the sampling points in the calculation window, and RMS be the temporal feature.

[0024] Optionally, before inputting the temporal features into the cross-individual lifetime network model to obtain the human hand movement prediction result, the method further includes:

[0025] The time series features are standardized using the μ-rate transformation method, and the calculation formula is as follows:

[0026]

[0027] Where RMS is the time series characteristic, μ is a hyperparameter that determines the final range of the signal value, sign indicates that the original value is kept positive or negative, and x is the time series characteristic after standardization.

[0028] Optionally, the step of training a cross-individual lifetime network using the first temporal feature as input and the motion sequence as output to obtain a first cross-individual lifetime network model specifically includes:

[0029] Obtain the hidden feature map of the first temporal feature through the classification branch;

[0030] The classification label is obtained by using the hidden layer feature map through a classifier;

[0031] The hidden features in the hidden layer feature map are treated as feature vectors by the regression branch, and the feature vectors and the classification label are weighted and summed to obtain the hidden feature map of the regression branch;

[0032] The predicted hand movement sequence is obtained from the hidden feature map using a regression predictor;

[0033] The first cross-individual lifetime network model is obtained by training based on the first loss function; the first loss function is:

[0034] L total =L r +L rec +λL c

[0035] Among them, L r For regression loss, L rec To rebuild the losses, L c Let λ be the classification loss and λ be the hyperparameter of the balancing loss function.

[0036] Optionally, the dimension of the feature vector is equal to the dimension of the classification label.

[0037] Optionally, after obtaining the predicted hand motion sequence from the hidden feature map via a regression predictor, the method further includes:

[0038] The hidden feature maps of the classification branch and the regression branch are used as inputs to obtain the reconstruction inputs of the classification branch and the regression branch.

[0039] Optionally, the Adapter process is as follows:

[0040] x down =FC (c,c / 2) (x)

[0041] x nl =GELU(x down )

[0042] Output = FC (c,c / 2) (x nl )+x.

[0043] The present invention also provides a human hand movement prediction system based on surface electromyography signals, the system comprising:

[0044] The temporal feature acquisition module is used to acquire the temporal features of surface electromyography signals;

[0045] The human hand movement prediction module is used to input the temporal features into a cross-individual lifetime network model to obtain human hand movement prediction results.

[0046] The model training module is used to train the cross-individual lifelong network to obtain the cross-individual lifelong network model.

[0047] The training process of the cross-individual lifelong network model includes:

[0048] The temporal characteristics of the first surface electromyography signal are obtained to obtain the first temporal characteristics;

[0049] Using the first temporal feature as input and the motion sequence as output, a cross-individual lifetime network is trained to obtain the first cross-individual lifetime network model;

[0050] Freeze the first parameters of the first cross-individual lifetime network model; the first parameters are all parameters except for the classifier parameters and the decoder parameters.

[0051] The temporal characteristics and the number of second surface electromyographic signals are obtained to obtain the second temporal characteristics and the number of second signals;

[0052] Update the FC layer in the classifier according to the number of the second signal, and add the corresponding number of Adapter structures in the regression branch;

[0053] Training is performed using the second loss function to obtain a cross-individual lifetime network model; the second loss function is: L lifelong =L r +L' rec +λL' c .

[0054] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0055] This invention provides a method for predicting human hand movements based on surface electromyography (EMG) signals. First, the temporal features of the EMG signals are acquired. These temporal features are then input into a cross-individual lifelong network model to obtain the predicted hand movements. The training process of the cross-individual lifelong network model includes: acquiring the temporal features of a first EMG signal to obtain a first temporal feature; training the cross-individual lifelong network using the first temporal feature as input and the movement sequence as output to obtain a first cross-individual lifelong network model; freezing the first parameters of the first cross-individual lifelong network model; the first parameters are all parameters except for the classifier and decoder parameters; acquiring the temporal features of a second EMG signal and the number of second EMG signals to obtain a second temporal feature and a second signal count; updating the fully connected (FC) layer in the classifier based on the second signal count, and adding a corresponding number of adapter structures to the regression branch; training according to a second loss function to obtain the cross-individual lifelong network model; the second loss function is: L lifelong =L r +L' rec +λL' c This invention proposes a model structure and training method that can be adapted to multiple individuals simultaneously during a single training process. It designs a lifelong learning strategy and adapts the model, enabling the method to quickly learn new individuals while maintaining the model's performance on already trained individuals, effectively solving the catastrophic forgetting problem of the model. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram illustrating the temporal representation of the extracted features and motion information from the raw sEMG signal provided in this embodiment of the invention.

[0058] Figure 2 This is a schematic diagram of the training process of the cross-individual lifelong network model provided in Embodiment 1 of the present invention.

[0059] Figure 3 This is a schematic diagram of the structural design of the cross-individual lifelong network model provided in Embodiment 1 of the present invention.

[0060] Figure 4 This is a block diagram of a human hand movement prediction system based on surface electromyography signals, provided in Embodiment 2 of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] The purpose of this invention is to provide a method and system for predicting human hand movements based on surface electromyography signals, which uses lifelong learning methods to solve the problem of catastrophic forgetting of models.

[0063] Compared to other bioelectrical signals, such as electrooculography (EOG) and electroencephalography (EEG), electromyography (sEMG) signals are easier to acquire. Furthermore, unlike traditional invasive methods of acquiring EMG signals, sEMG can be acquired non-invasively using surface-mount electrodes. Because sEMG signals can be easily acquired through wearable devices, they have been widely used in recent decades to predict and estimate human motor intentions.

[0064] sEMG acquires multi-channel time-series electrical signals, while human hand movements can also be captured using kinematic acquisition gloves, with the movement trajectory often described as a time-series image of joint angle changes over time. Therefore, continuous hand prediction based on sEMG signals is often studied as a regression task, with both input and output being multi-channel time-series signals. This method uses the sEMG time-series sequence as model input, allowing the model to calculate the time-series sequence of hand movements to predict them.

[0065] sEMG signal prediction tasks often do not directly use the acquired signal for prediction. Various preprocessing methods have been explored in this field for preliminary feature extraction and denoising of sEMG signals. Depending on the focus of the task regarding signal features, feature extraction methods include time-domain, frequency-domain, time-frequency-domain, and various combinations of features. Commonly used time-domain features include root mean square (RMS) features and zero-crossing features; commonly used frequency-domain features include autoregressive coefficients and spectrograms; and commonly used time-frequency-domain features include short-time Fourier transform and wavelet transform. This invention primarily uses time-domain feature extraction methods, mainly extracting the RMS features of sEMG signals. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 This is a schematic diagram illustrating the temporal representation of the extracted features and motion information from the raw sEMG signal.

[0066] Since this type of task is often regarded as a regression task for sequences, the field generally uses deep learning models for sequences, such as recurrent neural networks represented by the Long Short-Term Memory model, various variant models of the Transformer structure, and convolutional network methods.

[0067] Training neural networks using deep learning methods inevitably faces the challenge of catastrophic forgetting, meaning that when the model learns a new task, it forgets the previously learned tasks. This phenomenon occurs because during model parameter updates, new data overwrites or influences old data, leading to a decline in performance on older tasks. Lifetime learning methods have been researched in recent years to address or mitigate this problem.

[0068] Therefore, this invention proposes a human hand movement prediction method based on surface electromyography (EMG) signals that employs a lifelong learning strategy, and develops an EMG signal prediction tool with higher versatility based on this method.

[0069] Existing sEMG continuous motion prediction methods typically adapt widely used deep model approaches directly to the task. They also introduce transfer learning methods to accelerate training and facilitate knowledge sharing.

[0070] Because sEMG, as a bioelectrical signal, exhibits significant differences in characteristics among individuals, current methods can generally only be applied to a single subject, performing poorly on others. Furthermore, training the model on multiple subjects simultaneously results in poor performance across all individuals. The vast differences in sEMG characteristics between individuals, coupled with the greater difficulty in collecting biosignal data compared to other tasks and the ethical concerns it raises, lead to limited data volume. Unlike common computer vision and natural language processing tasks, existing methods cannot be directly adapted to individuals outside the training set. A common approach to this problem is to introduce transfer learning, but this introduces the problem of catastrophic forgetting, limiting the model's general applicability.

[0071] Therefore, this invention proposes a method for predicting human hand movements using electromyography (sEMG) signals based on a lifelong learning strategy. The improved method enables the model to learn across multiple subjects, and the introduction of lifelong learning technology ensures that knowledge sharing occurs within the model rather than between models, making our method more generalizable and universal. Lifelong learning addresses the problem of catastrophic forgetting; existing models can only be used on a single individual. This invention solves this problem by using a multi-head adapter structure. Simultaneously, the catastrophic forgetting problem still needs to be addressed, which this invention solves by introducing a lifelong learning method. This invention mainly proposes a deep learning-based cross-individual sEMG prediction method for continuous hand movements, as well as a lifelong learning-based model and training framework.

[0072] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] Example 1

[0074] This embodiment provides a method for predicting human hand movements based on surface electromyography signals, the method comprising:

[0075] S1. Obtain the temporal characteristics of surface electromyography signals.

[0076] The temporal features of surface electromyography (EMG) signals are extracted using the root mean square feature method. The calculation formula is as follows:

[0077]

[0078] sEMG i Let N be the electromyographic signal value of the i-th sampling point in the input window, N be the capacity of the sampling points in the calculation window, and RMS be the temporal feature.

[0079] The time series features are standardized using the μ-rate transformation method, and the calculation formula is as follows:

[0080]

[0081] Where RMS is the time series characteristic, μ is a hyperparameter that determines the final range of the signal value, sign indicates that the original value is kept positive or negative, and x is the time series characteristic after standardization.

[0082] S2. Input the time-series features into the cross-individual lifetime network model to obtain the human hand movement prediction results;

[0083] like Figure 2 As shown, the training process of the cross-individual lifelong network model includes the following steps, wherein the structural design diagram of the cross-individual lifelong network model is shown in the figure. Figure 3 As shown.

[0084] A1. Obtain the temporal characteristics of the first surface electromyography signal to obtain the first temporal characteristics.

[0085] A2. Using the first temporal feature as input and the motion sequence as output, train the cross-individual lifelong network to obtain the first cross-individual lifelong network model.

[0086] Steps A1 and A2 above describe the training process of a typical cross-individual lifelong network model, also known as the first training process. After the model is trained, if it needs to be adapted to new individuals, a lifelong learning strategy needs to be used to train it on a new dataset. The training steps are as follows:

[0087] A3. Freeze the first parameters of the first cross-individual lifetime network model; the first parameters are all parameters except for the classifier parameters and decoder parameters.

[0088] A4. Obtain the temporal characteristics of the second surface electromyography (EMG) signal and the number of second surface EMG signals to obtain the second temporal characteristics and the number of second signals.

[0089] A5. Update the FC layer in the classifier according to the number of the second signal, and add the corresponding number of Adapter structures in the regression branch.

[0090] The Adapter process is as follows:

[0091] x down =FC (c,c / 2) (Input Adapter )

[0092] x nl =GELU(x down )

[0093] Output Adapter =FC (c,c / 2) (x nl )+x.

[0094] Among them, FC (c,c / 2) This is a linear layer operation with c input channels and c / 2 output channels. GELU stands for Gaussian Error Linear Unit. Adapter For the adapter's input, Output Adapter represents the output of the adapter, and x represents the new input, i.e., the new individual that needs to be adapted.

[0095] The training data from the second surface electromyography signal and the first time-series signal are randomly sampled and mixed according to the proportion of each individual to the total number of individuals. This step actually involves randomly sampling and mixing the newly added training data with the existing memory data. The probability of random sampling is the probability of each data point to the total data, where each data point refers to the newly added training data, and the total data refers to the newly added training data and the existing memory data.

[0096] A6. Train according to the second loss function to obtain a cross-individual lifetime network model; the second loss function is: L lifelong =L r +L' rec +λL' c , where L′ c With L c The calculation method is the same, but the input changes from only the data x from the new task to the input x' obtained by uniform sampling.

[0097] L c The calculation process is as follows:

[0098]

[0099]

[0100] Then L' c The calculation process can be represented as follows:

[0101] x' = RandomSampleFrom(x,M)

[0102]

[0103]

[0104] Similarly, L rec The process is as follows:

[0105]

[0106]

[0107]

[0108] Then L' rec The calculation process can be represented as follows:

[0109] x' = RandomSampleFrom(x,M)

[0110]

[0111]

[0112]

[0113] In the above formula, E c For the classification branch encoder, E r In the regression branch encoder, x is the model input, x' is the sampled mixed input, C is the classifier, and D is the classifier. c For the classification branch decoder, D r For regression branch decoder, M is memory, and RandomSampleFrom represents the uniform sampling operation of data in this paper. As input for class branch reconstruction, This is the input for the regression branch reconstruction. MSE is the root mean square error loss function, CE is the cross-entropy loss function, and x... r For the regression branch input, For the regression branch output, x c For the classification branch input, For the classification branch output, y cFor the classification branch input, yc′ is the output of the classification branch, and yc′ is the input of the mixed classification branch. The output of the mixed classification branches is shown below. In summary, the outputs with "^" as the superscript are the outputs without "^".

[0114] New data observed during this training task is sampled and stored in memory.

[0115] The training process of using the first temporal feature as input and the motion sequence as output to train a cross-individual lifelong network to obtain the first cross-individual lifelong network model, i.e., the ordinary cross-individual lifelong network model, specifically includes:

[0116] B1. Obtain the hidden feature map of the first temporal feature through the classification branch;

[0117] B2. Obtain classification labels using the hidden layer feature maps through a classifier;

[0118] B3. By treating the hidden features in the hidden layer feature map as feature vectors through the regression branch, and performing a weighted summation operation on the feature vectors and the classification labels to obtain the hidden feature map of the regression branch; wherein, the dimension of the feature vector is equal to the dimension of the classification label.

[0119] B4. Obtain the predicted hand movement sequence from the hidden feature map using a regression predictor;

[0120] B5. Train the network using the first loss function to obtain the first cross-individual lifetime network model; the first loss function is:

[0121] L total =L r +L rec +λL c

[0122] Among them, L r For regression loss, L rec To rebuild the losses, L c For classification loss, λ is a hyperparameter of the balancing loss function. Specifically:

[0123]

[0124]

[0125]

[0126] Where MSE is the root mean square loss, CE is the cross-entropy loss, and x r As the input for the regression branch, x c For the classification branch input, similarly, y represents the actual data. For the output of the model network, y r For the regression branch input, For the regression branch output, Output for the category branch.

[0127] Also includes:

[0128] The hidden feature maps of the classification branch and the regression branch are used as inputs to obtain the reconstruction inputs of the classification branch and the regression branch.

[0129] This embodiment provides a human hand movement prediction method based on surface electromyography (sEMG) signals. It treats sEMG signals and motion information as a time series and proposes a model structure and training method that can be adapted to multiple individuals simultaneously during a single training process, based on temporal convolutional networks and μ-law transforms. A lifelong learning strategy is designed for the model, and the model is adapted to different individuals, enabling the method to quickly learn new individuals while maintaining its performance on already trained individuals. This invention provides a flexible process and framework, making the proposed method modular. Adaptation to specific situations can be achieved by flexibly replacing models within the framework and adjusting specific parameters.

[0130] Example 2:

[0131] like Figure 4 As shown, this embodiment provides a human hand movement prediction system based on surface electromyography signals, the system comprising:

[0132] M1, Temporal Feature Acquisition Module, is used to acquire the temporal features of surface electromyography signals;

[0133] M2, Human hand movement prediction module, is used to input the time-series features into the cross-individual lifetime network model to obtain human hand movement prediction results;

[0134] M3, the model training module, is used to train the cross-individual lifelong network to obtain the cross-individual lifelong network model;

[0135] The training process of the cross-individual lifelong network model includes:

[0136] The temporal characteristics of the first surface electromyography signal are obtained to obtain the first temporal characteristics;

[0137] Using the first temporal feature as input and the motion sequence as output, a cross-individual lifetime network is trained to obtain the first cross-individual lifetime network model;

[0138] Freeze the first parameters of the first cross-individual lifetime network model; the first parameters are all parameters except for the classifier parameters and the decoder parameters.

[0139] The temporal characteristics and the number of second surface electromyographic signals are obtained to obtain the second temporal characteristics and the number of second signals;

[0140] Update the FC layer in the classifier according to the number of the second signal, and add the corresponding number of Adapter structures in the regression branch;

[0141] Training is performed using the second loss function to obtain a cross-individual lifetime network model; the second loss function is: L lifelong =L r +L' rec +λL' c .

[0142] Example 3:

[0143] This embodiment provides a method for predicting human hand movements based on surface electromyography signals. The method includes:

[0144] Step 1: For each input sEMG time-series signal, its data format is number of sampling points T * number of acquisition channels C. The invention uses the root mean square feature calculation method to extract the time-series features of the signal. The calculation method is as follows:

[0145]

[0146] sEMG i Let N be the electromyographic signal value of the i-th sampling point in the input window, and N be the capacity of the sampling points in the calculation window. Each window sliding step corresponds to one sampling point, ensuring that the obtained temporal features are essentially consistent with the original signal. For each real hand movement signal, the data format is the number of sampling points T * the number of hand joints V.

[0147] Step 2: To eliminate range differences in data from different individuals and channels, and to amplify the low-amplitude characteristics of sEMG, the method uses μ-rate transformation to standardize the data, as follows:

[0148]

[0149] Wherein, RMS is the RMS feature obtained in the previous step, μ is a hyperparameter that determines the final range of the signal value, sign indicates that the original value is kept positive or negative, and x is the feature after standardization transformation, which also represents the input of the model in subsequent steps.

[0150] Step 3: Using the standardized features as input and the motion sequence as output, design the model structure. The design diagram is shown below:

[0151] The calculation steps of the entire model are as follows:

[0152] hc =E c (x)

[0153]

[0154]

[0155]

[0156]

[0157]

[0158] E c E r D represents the structure of the classification encoder and regression classifier in the network. c D r This represents the decoder structure. Specifically, C is the classifier, which is the module in the classification branch that operates on the extracted feature maps to obtain the output of the classification branch, and R is the regressor, which is the module in the regression branch that operates on the extracted feature maps to obtain the output of the regression branch.

[0159] Among them, h r h c These are the feature maps extracted from the model input x for the regression task branch and the classification task branch, respectively. The classification task branch consists of an encoder-decoder structure composed of a temporal convolutional network, and its detailed process is as follows:

[0160]

[0161]

[0162]

[0163]

[0164] x is the input, h c The first result is the output; the rest are intermediate results.

[0165] in This refers to dilated causal convolution in a temporal convolutional network, where d is the dilation rate. in c out Here, c represents the number of channels for the input and output data, and c represents the number of channels for the input RMS feature. h is a hyperparameter representing the number of feature channels extracted from the feature map. Similarly, the encoder of the regression task branch can be represented in detail as follows:

[0166]

[0167]

[0168]

[0169]

[0170] h i =Adapter i (h0)(i∈{1,2,…,N})

[0171]

[0172] x is the input, h r The first result is the output; the rest are intermediate results.

[0173] Where N represents the total number of individuals that the model fits. Let represent the probability of being identified as the i-th individual in a single classification, where the sum of all i is 1.

[0174] Meanwhile, the decoder process for both branches is as follows:

[0175]

[0176]

[0177]

[0178]

[0179] h is the input feature map. The input for reconstruction is the normalized RMS features restored by the network decoder, and the rest are intermediate results.

[0180] The operations for calculating the results from the feature maps in both branches are as follows:

[0181] C(h c =Softmax(FC) (c,N) (Adapter(h c )))

[0182]

[0183] In the above process, the final result is The two branches of the decoder only participate in the training process and not in the inference process, which is the prediction result.

[0184] In this step and subsequent lifelong learning strategies, an Adapter structure is introduced to adapt individual knowledge. It is a lightweight, plug-and-play transfer learning module that allows the model to quickly adapt to new task knowledge. The Adapter process used in this invention is as follows:

[0185] x down =FC (c,c / 2) (x)

[0186] x nl =GELU(x down )

[0187] Output = FC (c,c / 2) (x nl )+x

[0188] Step 4: Training Strategy and Lifelong Learning Strategy Design and Implementation. During direct model training, an adapter is requested directly for the current number of tasks, and the model's parameters are trained. Simultaneously, the training data is stored in contiguous memory on an individual basis, storing only the input sEMG signal and not motion information. The training loss function is as follows:

[0189] L total =L r +L rec +λL c

[0190] Among them, L r For regression loss, L rec To rebuild the losses, L c For classification loss, λ is a hyperparameter of the balancing loss function. Specifically:

[0191]

[0192]

[0193]

[0194] Where MSE is the root mean square loss and CE is the cross-entropy loss.

[0195] In summary, the model's inference process and direct training process are as follows:

[0196] The classification branch is obtained by deriving the hidden feature map h from the input sEMG. c The classifier C is composed of h c Get category tags

[0197] The regression branch is derived from the input sEMG, resulting in a vector {h} that treats the hidden features as elements. i}, its dimension is equal to Dimension, by weighting the feature vectors and classification labels, yields the hidden feature map h of the regression branch. r The regression predictor is h rObtain the predicted hand movement sequence That is, the desired result of the task.

[0198] During training, the model additionally requires two branches of decoders, which take the hidden feature maps of the two branches as input to obtain the reconstructed inputs of the two branches. and

[0199] During training, the data calculated above is used to train the system according to the loss function.

[0200] After the model is trained, if it needs to be adapted to new individuals, a lifelong learning strategy needs to be used to train it on a new dataset. The training steps are as follows:

[0201] Freeze all old model parameters (except C(h)) c (and decoder), update C(h) for the number of newly added individuals. c In the FC layer, add the corresponding number of Adapter structures in the regression branch.

[0202] The newly added individual data and the individual data from memory are randomly sampled and mixed according to the proportion of the individual to the total number of individuals, so as to maintain the consistency of the model input size.

[0203] Training is performed based on the new loss function, which is adapted from the normal training process:

[0204] L lifelong =L r +L' rec +λL' c

[0205] L' represents the output calculated from uniform sampling.

[0206] New data observed during this mission is sampled and stored in memory.

[0207] This implementation treats sEMG signals and motion information as time series. Based on temporal convolutional networks and μ-law transforms, this invention proposes a model structure and training method that can be adapted to multiple individuals simultaneously during a single training process. The invention designs a lifelong learning strategy for the model and adapts it, enabling the method to quickly learn new individuals while maintaining the model's performance on already trained individuals. This invention provides a complete and flexible process and framework, making the proposed method modular. Adaptation to specific situations can be achieved by flexibly replacing models within the framework and adjusting specific parameters.

[0208] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0209] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A human hand motion prediction method based on surface electromyography signals, characterized in that, The method includes: Acquire the temporal characteristics of surface electromyography signals; The temporal features are input into a cross-individual lifetime network model to obtain human hand movement prediction results; The training process of the cross-individual lifelong network model includes: The temporal characteristics of the first surface electromyography signal are obtained to obtain the first temporal characteristics; Using the first temporal feature as input and the motion sequence as output, a cross-individual lifetime network is trained to obtain the first cross-individual lifetime network model; Freeze the first parameters of the first cross-individual lifetime network model; the first parameters are all parameters except for the classifier parameters and the decoder parameters. The temporal characteristics and the number of second surface electromyographic signals are obtained to obtain the second temporal characteristics and the number of second signals; Update the FC layer in the classifier according to the number of the second signal, and add the corresponding number of Adapter structures in the regression branch; The cross-individual lifetime network model is obtained by training based on the second loss function; the second loss function is: ; Before inputting the temporal features into the cross-individual lifetime network model to obtain the human hand movement prediction result, the method further includes: The time series features are standardized using the μ-rate transformation method, and the calculation formula is as follows: ； Where RMS represents the time series characteristic, μ is a hyperparameter that determines the final range of the signal value, and sign indicates that the original value should be retained (positive or negative sign). x These are the time-series features after standardization. The process of training a cross-individual lifelong network, using the first temporal feature as input and the motion sequence as output, to obtain the first cross-individual lifelong network model, specifically includes: Obtain the hidden feature map of the first temporal feature through the classification branch; The classification label is obtained by using the hidden layer feature map through a classifier; The hidden features in the hidden layer feature map are treated as feature vectors by the regression branch, and the feature vectors and the classification label are weighted and summed to obtain the hidden feature map of the regression branch; The predicted hand movement sequence is obtained from the hidden feature map using a regression predictor; The first cross-individual lifetime network model is obtained by training based on the first loss function; the first loss function is: ； in, To regress the loss, To rebuild the losses, For classifying losses, These are the hyperparameters for balancing the loss function; The Adapter process is as follows: ； ； ； in, The number of input channels is The number of output channels is Linear layer operations; Represents a Gaussian error linear unit; represents the adapter's output; x represents the new input, i.e., the new individual that needs to be adapted.

2. The method for predicting human hand movements based on surface electromyography signals according to claim 1, characterized in that, After training according to the second loss function to obtain the cross-individual lifetime network model, the process further includes: New data observed during this training task is sampled and stored in memory.

3. The method for predicting human hand movements based on surface electromyography signals according to claim 2, characterized in that, After updating the FC layer in the classifier according to the number of the second signals and adding a corresponding number of Adapter structures in the regression branch, the method further includes: The temporal features of the second surface electromyography signal and the training data in the first temporal features are randomly sampled and mixed according to the proportion of individuals to the total number of individuals.

4. The method for predicting human hand movements based on surface electromyography signals according to claim 1, characterized in that, The temporal characteristics of acquiring surface electromyography signals specifically include: The temporal features of surface electromyography (EMG) signals are extracted using the root mean square feature method. The calculation formula is as follows: ； in Let N be the electromyographic signal value of the i-th sampling point in the input window, N be the capacity of the sampling points in the calculation window, and RMS be the temporal feature.

5. The method for predicting human hand movements based on surface electromyography signals according to claim 1, characterized in that, The dimension of the feature vector is equal to the dimension of the classification label.

6. The method for predicting human hand movements based on surface electromyography signals according to claim 1, characterized in that, After obtaining the predicted hand movement sequence from the hidden feature map via a regression predictor, the method further includes: The hidden feature maps of the classification branch and the regression branch are used as inputs to obtain the reconstruction inputs of the classification branch and the regression branch.

7. A human hand movement prediction system based on surface electromyography signals, characterized in that, The system includes: The temporal feature acquisition module is used to acquire the temporal features of surface electromyography signals; The human hand movement prediction module is used to input the temporal features into a cross-individual lifetime network model to obtain human hand movement prediction results. Before inputting the temporal features into the cross-individual lifetime network model to obtain the human hand movement prediction result, the method further includes: The time series features are standardized using the μ-rate transformation method, and the calculation formula is as follows: ； Where RMS represents the time series characteristic, μ is a hyperparameter that determines the final range of the signal value, and sign indicates that the original value should be retained (positive or negative sign). x These are the time-series features after standardization. The model training module is used to train the cross-individual lifelong network to obtain the cross-individual lifelong network model. The training process of the cross-individual lifelong network model includes: The temporal characteristics of the first surface electromyography signal are obtained to obtain the first temporal characteristics; Using the first temporal feature as input and the motion sequence as output, a cross-individual lifetime network is trained to obtain the first cross-individual lifetime network model; Freeze the first parameters of the first cross-individual lifetime network model; the first parameters are all parameters except for the classifier parameters and the decoder parameters. The temporal characteristics and the number of second surface electromyographic signals are obtained to obtain the second temporal characteristics and the number of second signals; Update the FC layer in the classifier according to the number of the second signal, and add the corresponding number of Adapter structures in the regression branch; The cross-individual lifetime network model is obtained by training based on the second loss function; the second loss function is: ; The process of training a cross-individual lifelong network, using the first temporal feature as input and the motion sequence as output, to obtain the first cross-individual lifelong network model, specifically includes: Obtain the hidden feature map of the first temporal feature through the classification branch; The classification label is obtained by using the hidden layer feature map through a classifier; The hidden features in the hidden layer feature map are treated as feature vectors by the regression branch, and the feature vectors and the classification label are weighted and summed to obtain the hidden feature map of the regression branch; The predicted hand movement sequence is obtained from the hidden feature map using a regression predictor; The first cross-individual lifetime network model is obtained by training based on the first loss function; the first loss function is: ； in, To regress the loss, To rebuild the losses, For classifying losses, These are the hyperparameters for balancing the loss function; The Adapter process is as follows: ； ； ； in, The number of input channels is The number of output channels is Linear layer operations; Represents a Gaussian error linear unit; represents the adapter's output; x represents the new input, i.e., the new individual that needs to be adapted.

Images