A Dexterous Hand Movement Decoding Method Based on Collaborative Manifolds

Abstract

The present invention discloses a dexterous hand movement decoding method based on a collaborative base manifold, including: acquiring electroencephalogram (EEG) signal features and corresponding hand movement data, performing preprocessing after temporal alignment to obtain a training data set; constructing a dexterous hand movement decoding model. In the model, a periodic network module maps the EEG signal features into a phase manifold to obtain EEG features with periodic characteristics; a control network module extracts decoupled spatio-temporal motion control parameters; a collaborative base recruitment module decodes the corresponding hand movement according to the spatio-temporal motion control parameters; constructing an overall loss, and training the dexterous hand movement decoding model using the training data set; extracting the EEG signal features to be predicted, inputting them into the trained dexterous hand movement decoding model, and decoding the corresponding hand movement. Using the present invention, more accurate and refined hand movement decoding can be achieved.

Description

Technical Field

[0001] The invention belongs to the field of motor nerve signal decoding, and in particular relates to a dexterous hand movement decoding method based on collaborative basis manifolds. Background Art

[0002] Invasive brain-computer interfaces build direct communication and control pathways between the brain and external devices, and have shown important application value in medical fields such as motor function rehabilitation. As the core carrier of human interaction with the outside world, human hands have the ability to manipulate objects with high flexibility, precision, and almost no deliberate control. This fine hand movement control function plays an irreplaceable role in daily life. Given the fundamental position of hand function in human behavior, how to achieve continuous and precise reconstruction of hand movement function has become a key technical challenge to help disabled people restore their independent living and professional abilities. At present, this field has become a key frontier cross-cutting direction of global research.

[0003] For example, the Chinese patent document with publication number CN113101021A discloses a robotic arm control method based on the MI-SSVEP hybrid brain-computer interface, which realizes the control of the robotic arm through non-invasive EEG data acquisition, online EEG signal preprocessing, online EEG signal processing and decoding, generating user intended targets and converting the targets into control instructions.

[0004] Current brain-computer interface research has progressed to the stage of initially achieving dexterous control of robotic arms, that is, high-performance control of 3-4 degrees of freedom has been achieved, and autonomous coffee drinking and eating have been achieved. However, the human hand has 27 degrees of freedom, and dexterous control of the hand is extremely difficult. The main challenge lies in the coordination between multiple joints of the hand and the nonlinear interaction. Most of the current hand motion decoding work regards the parameters of each joint as an independent variable, ignoring the synergistic relationship between joints, and thus has limited effect on hand motion decoding. For this reason, some studies have proposed that the brain may regulate high-degree-of-freedom motion in a low-dimensional control space (manifold) and mine low-dimensional representations from neural signals. Although many different manifolds have been found in neural signals, the correspondence between these manifolds and hand motion control parameters is still unclear.

[0005] On the other hand, although hand movement control has a high-dimensional degree of freedom, these degrees of freedom are not completely independent but redundant. Therefore, some work is based on dimensionality reduction methods to identify low-dimensional motion manifolds from complex hand movements and complete the decoding of high-dimensional motions based on this. However, due to the strong dependence of the learning process on assumptions and constraints, different motion manifolds will be obtained under different assumptions, which is obviously inconsistent with the operation control mechanism of the brain. In addition, most studies only consider the motion signals themselves and do not associate them with brain signals. Therefore, it is unknown whether and how the motion manifolds are correlated with neural signals.

[0006] Based on the above background, existing studies mostly explore low-dimensional hand movement control manifolds separately in the neural signal space or the motion space, but often ignore the connection between the two, which may lead to a mismatch between the neural manifold and the motion manifold, thus limiting the decoding effect of high-dimensional hand movements. Summary of the Invention

[0007] The present invention provides a dexterous hand movement decoding method based on a collaborative basis manifold. Through a joint manifold learning method that simultaneously considers neural signals and motion signals, the guidance and constraints of electroencephalogram data are added to the learning of the motion collaborative basis, so that the learned collaborative basis manifold can better reflect the actual control mechanism of the brain, thereby achieving more accurate and refined hand movement decoding.

[0008] A dexterous hand movement decoding method based on a collaborative basis manifold includes:

[0009] (1) Obtain electroencephalogram signal features and corresponding hand movement data, perform preprocessing after time series alignment to obtain a training data set;

[0010] (2) Construct a dexterous hand movement decoding model based on a collaborative basis manifold, including a periodic network module, a control network module, and a collaborative basis recruitment module;

[0011] Among them, the periodic network module is used to map electroencephalogram signal features into a phase manifold to obtain electroencephalogram features with periodic characteristics; the control network module extracts decoupled spatio-temporal motion control parameters from the electroencephalogram features obtained by the periodic network module; the collaborative basis recruitment module recruits corresponding time-varying collaborative bases from the maintained collaborative basis dictionary according to the spatio-temporal motion control parameters obtained by the control network module, and combines and decodes to obtain the corresponding hand movement;

[0012] (3) Construct an overall loss, and use the training data set to train the dexterous hand movement decoding model;

[0013] (4) Extract the electroencephalogram signal features to be predicted, input them into the trained dexterous hand movement decoding model, and decode to obtain the corresponding hand movement.

[0014] In step (1), the spike band energy in the invasive electroencephalogram data is extracted as the electroencephalogram signal feature.

[0015] In step (2), the periodic network module is developed based on the autoencoder framework, including: a time-domain convolutional layer and a deconvolutional layer, a periodic encoding layer and a decoding layer based on the Fourier transform, and a phase manifold mapping layer for extracting periodic motion control features; the training of the periodic network module includes a reconstruction loss of neural signal autoencoding.

[0016] In step (2), the control network module includes two independent feedforward network layers, which map the electroencephalogram features obtained from the periodic network module into spatio-temporal motion control parameters including an amplitude scaling factor and a time offset.

[0017] In step (2), the control network module also includes a decoupling module used for decoupling spatio-temporal motion control parameters during the training phase. The decoupling module obtains motion parameters for independently controlling motion time and space by exchanging parameters between different samples in a batch; the training of the control network module includes a decoupling loss.

[0018] In step (2), the co-basis recruitment module maintains a co-basis dictionary. Based on the amplitude scaling factor and time offset obtained from the control network module, the corresponding time-varying co-bases are selected from the co-basis dictionary; the amplitude is scaled using the corresponding amplitude scaling factor, the time step of the co-basis is selected using the time offset, and then the decoding result of hand motion is obtained by linearly combining all co-bases; the training of the co-basis recruitment module includes a decoding loss of motion data.

[0019] In step (2), the co-basis recruitment module maintains a co-basis dictionary. During the training phase, the co-basis dictionary is used as a learnable parameter tensor and is estimated together with other parts of the network through the backpropagation method.

[0020] During the parameter learning process of the co-basis dictionary, it is constrained by the regularization terms of the dictionary, including the smoothing regularization term in the time dimension and the orthogonality regularization term between dictionary elements; the training of the co-basis recruitment module includes a regularization loss of dictionary learning.

[0021] In step (3), the overall loss constructed includes the reconstruction loss of neural signal autoencoding from the periodic network module, the decoupling loss from the control network module, the decoding loss from the co-basis recruitment module, and the regularization loss of dictionary learning.

[0022] In step (3), during the training process of the dexterous hand motion decoding model, parameter estimation is performed based on the backpropagation method.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] Different from the previous methods of separately constructing neural manifolds or motion manifolds, the present invention adopts a joint manifold learning method, taking into account neural signals and motion signals simultaneously. At the neural signal level, a periodic manifold representation is adopted; at the motion signal level, a cooperative basis is introduced to encode multi-joint hand movements. Meanwhile, the neural representation and the motion representation are associated through motion control parameters, enabling the learning process of the hand motion manifold to be supervised by neural signals, thereby realizing the joint manifold learning of the two modal signals. This method makes the hand movement process more in line with the natural control mode of the brain. Experimental results show that the method of the present invention can achieve more accurate and delicate hand movement decoding. Description of the Drawings

[0025] Figure 1 It is a framework flowchart of a dexterous hand movement decoding method based on a cooperative basis manifold in an embodiment of the present invention.

[0026] Figure 2 It is an experimental paradigm diagram of a data set in an implementation of the present invention. Detailed Implementation Manner

[0027] The present invention will be further described in detail below with reference to the drawings and embodiments. It should be noted that the following embodiments are intended to facilitate the understanding of the present invention and do not impose any limitation on it.

[0028] All clinical and experimental procedures in the embodiments of the present invention have been approved by the Medical Ethics Committee of the Second Affiliated Hospital of Zhejiang University (Ethical Review Number 2019-158, approved on May 22, 2019). The subject is a 74-year-old male who suffered a C4-level cervical spinal cord injury due to a car accident and is quadriplegic, able to move only the parts above his neck, with normal language communication ability and task understanding ability. Two 96-channel Utah intracortical microelectrode arrays (Blackrock Microsystems, Salt Lake City, Utah, USA) were implanted in the left primary motor cortex of the subject to record neural signals. The subject undergoes brain-computer interface training tasks on each working day and rests on weekends.

[0029] As Figure 1 shown, a dexterous hand movement decoding method based on a cooperative basis manifold includes the following steps:

[0030] Step 1, Data preprocessing: Obtain the original invasive electroencephalogram data recorded by the hardware and the corresponding hand movement data; extract the spike band energy in the invasive electroencephalogram data as the electroencephalogram signal feature for dexterous hand movement decoding; perform temporal alignment on the electroencephalogram signal feature and the corresponding hand movement data through a sliding time window to form a time series sample set; divide the data into a training set, a validation set, and a test set according to a reasonable ratio, and standardize the data to obtain the preprocessed electroencephalogram data and the corresponding movement data.

[0031] The experimental paradigm for data acquisition is as Figure 2 shown. The dataset includes the electroencephalogram signals and the corresponding movement signals collected from a subject while watching virtual hand movements in a guidance video and imagining performing the same hand movements. Each experimental trial of the paradigm consists of two phases: "Preparation" and "Execution". In the "Preparation" phase, the type of action to be performed will be prompted through pictures and voices; subsequently, in the "Execution" phase, the virtual hand will continuously perform the action 5 times, and the subject will observe and imagine performing this action himself. The paradigm includes 11 different hand movements, namely: thumb flexion, index finger flexion, middle finger flexion, ring finger flexion, little finger flexion, flexion of the last four fingers, flexion of the last three fingers, five-finger grip, three-finger grip, two-finger grip, and thumb adduction. The execution time for each action is 2.4 seconds; each block will contain the random order appearance of all 11 actions, and each session contains 4 blocks.

[0032] During the action execution, the angles of 15 key joints of the hand were recorded as movement data, with a sampling frequency of 250 hz. After smoothing, symmetrization processing, and upsampling to 1000 Hz, the temporal movement data of 11 different actions were obtained , where M represents the feature dimension of the movement signal, specifically 15, represents the time required to complete a full action, specifically 2400.

[0033] At the same time, two 96-channel Utah intracortical microelectrode arrays implanted in the subject's cortex were used to synchronously collect neural signals. And the Neuroport system (Blackrock Microsystems) was used to amplify, digitize, and sample the neural signals at 30 kHz. In order to denoise the signals, high-pass (0.3 Hz) and low-pass (7.5 kHz) Butterworth filters were used to filter the data. The acquisition work was carried out on three different experimental days: the 1035th day, the 1081st day, and the 1105th day after implantation.

[0034] The original invasive electroencephalogram (EEG) data sampled at 30 kHz is high-pass filtered at 250 Hz, band-pass filtered in the range of 3000 - 1000 Hz, half-wave rectified, and then downsampled to 1000 Hz to obtain the spiking bandpower (SBP) as the EEG signal feature X for decoding. , where represents the feature dimension of the EEG signal, specifically 196, represents the time required to complete a full movement, specifically 2400.

[0035] The EEG data and movement data are preprocessed using a sliding window approach. The window length and step size can be set according to the data characteristics. In this embodiment, the window length is set to 400 ms and the step size is set to 200 ms. The data within the window is averaged to obtain 11-step data. For the movement signal at each time step, the EEG signal of the previous N time steps is used for prediction. In this embodiment . According to the principle of k-fold cross-validation, k - 2 folds of the data are used as the training set, 1 fold of the data is used as the validation set, and 1 fold of the data is used as the test set, repeating k times. In each fold of the data, the movement data and neural data are standardized according to the mean and variance obtained from the training set. Use the average performance of all validation sets in the folds as the criterion to select the optimal model, and use the average performance of the test set of the fold as the final performance metric.

[0036] In this embodiment, k = 4. According to the 4-fold cross-validation method, the data is divided into the training set, validation set, and test set in the ratio of 2:1:1.

[0037] Step 2: Establish a dexterous hand movement decoding model based on collaborative basis manifolds, mainly including a periodic network module, a control network module, and a collaborative basis recruitment module.

[0038] 2-1. Periodic network module: Map the EEG features into a phase manifold to obtain EEG features with periodic characteristics;

[0039] Specifically, the periodic network module is developed based on an autoencoder framework, including a time-domain convolutional layer and a transposed convolutional layer, a periodic encoding and decoding layer based on the Fourier transform, and a phase manifold mapping layer for extracting periodic movement control features. Specifically, the calculation method is as follows:

[0040] First, extract the hidden representation from the neural signal based on the time-domain convolutional layer:

[0041] ;

[0042] Among them, Denote including channels, and denote a latent representation with dimension . Denote a one-dimensional temporal convolutional layer, whose convolutional kernel scale can be set according to the data characteristics. In this embodiment, the number of channels is 196, the number of time steps is 5, and the dimension of the latent representation of the number of channels is selected optimally from 320 to 512 at an interval length of 32. The size of the time dimension of the one-dimensional temporal convolutional layer is selected optimally from {3, 5}.

[0043] Next, model the latent representation with temporal features through a periodic sine function, and its parameters include amplitude ( ), frequency ( ), offset ( ), and phase offset ( ). To estimate these parameters, perform a fast Fourier transform ( ) on the latent representation to obtain Fourier coefficients , where . Then, according to the coefficient magnitudes of each frequency component, calculate the amplitude , frequency , and offset of the latent representation of each dimension by weighted average:

[0044] ;

[0045] where denotes the power of the frequency component , and is the frequency vector. The phase offset is learned through two fully connected layers (FC):

[0046] ;

[0047] To enhance the periodic representation characteristics of the latent representation for motion control, a periodic control manifold is defined, and the corresponding representation is mapped onto the manifold :

[0048] ;

[0049] To avoid the network overfitting only relying on motion data, we add a decoder to the network to reconstruct the original neural signal. The specific approach is to reconstruct the latent representation according to the learned parameters:

[0050] ;

[0051] Among them, represents the time window. Subsequently, one-dimensional deconvolution is used to map back to the original neural signal space:

[0052] ;

[0053] In this embodiment, the time dimension size of the one-dimensional inverse chronological convolution layer is the same as that of the one-dimensional inverse chronological convolution layer.

[0054] The network learns by minimizing the reconstruction loss function between the original neural signal and the reconstructed neural signal:

[0055] ;

[0056] In this embodiment, the mean square error (MSE) is used to measure the loss.

[0057] 2-2. Control network module: Extract decoupled spatio-temporal motion control parameters from the EEG features obtained from the periodic network module;

[0058] Specifically, the control network module includes two independent feedforward network layers, which map the EEG features obtained from the periodic network module into motion control parameters including an amplitude scaling factor and a time offset. Specifically, the calculation method is as follows:

[0059] Two independent feedforward network (FFN) layers are used to predict the amplitude scaling factor ( ) and the time offset ( ), where K represents the number of co-bases. Each FFN consists of two layers of linear transformation and the activation function ReLU in between. To encourage the sparsity of activation, the activation function ReLU is added after the feedforward network layer of the amplitude scaling factor; and to represent the time offset in the style of relative time, we is restricted to the range (0, 1], so the activation function Tanh after scaling and offset transformation is added after the corresponding feedforward network. The calculation method is as follows:

[0060] ;

[0061] In this embodiment, the optimal middle layer dimension size of the FFN is selected from {196, 256}.

[0062] To encourage the control network module to extract decoupled motion time and space control parameters, during the training phase, the control network includes a parameter decoupling module. This module strengthens the independence of spatio-temporal control parameters through a "data augmentation" strategy during training. Specifically, the execution of a single continuous action can be decomposed into two parts: action category (what action to perform) and action style (how to perform the action). Then the corresponding neural control parameters should also have similar decoupling properties, that is: for an action instance, the scaling factor should remain relatively stable throughout the continuous action; at the same time, for movements of different categories but the same style, the change law of its time offset should be basically the same. To achieve these two principles, in each batch of training, this module randomly exchanges the motion control parameters corresponding to samples with the same action category or phase label to obtain new spatio-temporal parameter pairs, and then performs the corresponding co-recruitment process and introduces the corresponding decoding loss to encourage independent control of motion parameters. Specifically, the calculation method is as follows:

[0063] During the training phase, for each batch of samples (the number of samples is ), among the samples with the same action category or motion phase label, randomly exchange the corresponding and to obtain the exchanged results and , which is formulated as follows:

[0064] ;

[0065] where, represents the random permutation operation constrained by the corresponding label, represents the label of the action category, represents the label of the motion phase. In this embodiment, the sample size of this batch is set to 32.

[0066] Then, perform the co-recruitment process again on the randomly exchanged sample pairs to obtain the corresponding motion outputs and . Compare the similarity with the real motion data, thereby introducing the exchange loss :

[0067] ;

[0068] where, represents the co-basis recruitment function, represents the co-basis dictionary it maintains. In this embodiment, the mean square error (MSE) is used to measure the loss.

[0069] 2-3. Synergistic basis recruitment module: Based on the motion control parameters obtained from the control network module, recruit the corresponding time-varying synergistic basis from the maintained synergistic basis dictionary, and perform combined decoding to obtain the corresponding hand motion.

[0070] Specifically, the synergistic basis recruitment module maintains a synergistic basis dictionary. Based on the amplitude scaling factor and time offset obtained from the control network, select the corresponding time-varying synergistic basis from the synergistic basis dictionary, scale the amplitude using the corresponding amplitude scaling factor, select the time step of the synergistic basis using the time offset, and then obtain the decoding result of the hand motion by linearly combining all the synergistic bases. Specifically, the calculation method is as follows:

[0071] The synergistic basis recruitment module internally maintains a synergistic dictionary , and each synergistic basis will perform scaling and time offset operations respectively according to the amplitude and time offset to obtain the synergistic basis transformed by the control parameters, and finally add up the contributions of all the synergistic bases to obtain the motion data output . The corresponding synergistic basis recruitment function can be described in the following form:

[0072] ;

[0073] However, the synergistic basis is discrete in the time dimension, while the time offset is continuous. To solve this mismatch and recruit the synergistic basis in an approximately continuous manner, the method of linear weighted interpolation can be adopted. That is, find the two time steps closest to the time offset , and calculate the weights of the two time steps according to the distance . Thus, the recruitment of the synergistic basis in time can be written as: . Therefore, the corresponding recruitment function can be written as:

[0074] ;

[0075] Subsequently, compare the similarity of the real motion data and introduce the regression decoding loss to supervise the learning of the network parameters:

[0076] ;

[0077] In this embodiment, the mean square error (MSE) is used to measure the loss.

[0078] The synergistic basis recruitment module maintains a spatio-temporal synergistic basis dictionary , where is the number of collaborative bases, is the dimension of the motion output, is the number of time steps (reflecting the time resolution). During the training phase, this dictionary serves as a learnable parameter tensor and, along with other parts of the network, undergoes parameter estimation through the backpropagation method. In this embodiment, the number of collaborative bases is set to 15, the dimension of the motion output is 15, and the number of time steps is set to 20.

[0079] The collaborative base recruitment module maintains a spatio-temporal collaborative base dictionary. During the training phase, the parameter learning process of the dictionary is constrained by the regularization terms of the dictionary, including the smooth regularization term in the time dimension and the orthogonality regularization term between dictionary elements. Specifically, the calculation method is as follows:

[0080] To ensure the smoothness of the collaborative bases in the time dimension, a smooth regularization term loss is defined to penalize large differences between adjacent time steps, thus encouraging a smooth transition of collaboration over time:

[0081] ;

[0082] At the same time, to reduce redundancy between collaborative bases and encourage their orthogonality in the feature space, we unfold it in the spatial dimension to obtain , and then calculate its Gram matrix . The orthogonality regularization term loss is defined as minimizing the distance between and the identity matrix

[0083] ;

[0084] ;

[0085] ;

[0086] where represents the Frobenius norm.

[0087] Step 3: Based on the losses obtained from each module, all losses are summed with reasonable weights to obtain the overall loss of the model for network training.

[0088] Specifically, the overall loss module integrates the reconstruction loss of neural signal autoencoding from the periodic network module, the decoupling loss from the control network module, the decoding loss from the collaborative base recruitment module, and the regularization loss of dictionary learning. Specifically, the calculation method is as follows:

[0089] ;

[0090] Among them, 、 and are hyperparameters used to balance the influence degree of each loss on the model and can be set manually. In this embodiment, is uniformly sampled between 0.85 and 0.95 to select the optimal one. is sampled based on logarithmic uniform distribution between 0.1 and 500 to select the optimal one, is uniformly sampled between 0.1 and 1 to select the optimal one.

[0091] Step 4: Model training and performance evaluation: The network is trained with the training set data, the fitting degree of the model is evaluated with the validation set data, the model with the best regression effect is selected, and finally the model performance is evaluated in the test set.

[0092] Specifically with examples, the main parameters for model learning are set as follows: The batch size is set to 32; The Adam algorithm is used to optimize the parameters, and the learning rate is set to be uniformly sampled between 0.001 and 0.01 and the optimal one is selected according to the model's performance on the validation set, and the weight decay is set to 1e-4; The maximum number of training epochs is set to 5000, and the early stopping method is adopted to alleviate the overfitting problem. Specifically, after each epoch, the R 2 is calculated, and the model with the best performance on the validation set is retained. If R 2 does not show an improvement of more than 1% over the previous performance within 30 epochs, the training is stopped to obtain the final network parameters.

[0093] To verify the effectiveness of the proposed dexterous hand movement decoding method (SynergyNet) based on collaborative base manifold, it is compared with the long short-term memory neural network (Long Short-Term Memory, LSTM) which is the optimal one in the performance of invasive brain-computer interface decoding tasks, and the coefficient of determination (Coefficient of Determination, ), correlation coefficient (Correlation Coefficient, CC) and root mean squared error (Root Mean Squared Error, RMSE) commonly used in regression decoding are used to evaluate the performance (see Table 1). It can be seen from Table 1 that SynergyNet has a significant performance improvement compared with LSTM: It has increased by 8.72%, CC has increased by 3.72%, and RMSE has decreased by 6.80%, indicating that our method has significant advantages in accurately decoding hand movements.

[0094] Table 1

[0095]

[0096] The above-described embodiments have elaborated in detail the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, supplements, and equivalent replacements made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dexterous hand movement decoding method based on collaborative base manifolds, characterized in that Including: (1) Obtain electroencephalogram (EEG) signal features and corresponding hand movement data, preprocess them after temporal alignment to obtain a training dataset; (2) Construct a dexterous hand movement decoding model based on a collaborative basis manifold, including a periodic network module, a control network module, and a collaborative basis recruitment module; Among them, the periodic network module is used to map EEG signal features into a phase manifold to obtain EEG features with periodic characteristics; the control network module extracts decoupled spatio-temporal movement control parameters from the EEG features obtained by the periodic network module; the collaborative basis recruitment module recruits corresponding time-varying collaborative bases from the maintained collaborative basis dictionary according to the spatio-temporal movement control parameters obtained by the control network module, and combines and decodes to obtain the corresponding hand movement; The periodic network module is developed based on an autoencoder framework, including: a time-domain convolutional layer and a transposed convolutional layer, a periodic encoding layer and a decoding layer based on the Fourier transform, and a phase manifold mapping layer for extracting periodic movement control features; the training of the periodic network module includes a reconstruction loss of neural signal autoencoding; The control network module includes two independent feedforward network layers, which map the EEG features obtained from the periodic network module into spatio-temporal movement control parameters including an amplitude scaling factor and a time offset; The control network module also includes a decoupling module for decoupling spatio-temporal movement control parameters during the training phase. The decoupling module obtains movement parameters for independently controlling movement time and space by exchanging parameters between different samples in a batch; the training of the control network module includes a decoupling loss; The collaborative basis recruitment module maintains a collaborative basis dictionary, selects corresponding time-varying collaborative bases from the collaborative basis dictionary based on the amplitude scaling factor and time offset obtained by the control network module; scales the amplitude using the corresponding amplitude scaling factor, selects the time step of the collaborative basis using the time offset, and then obtains the decoding result of the hand movement by linearly combining all collaborative bases; the training of the collaborative basis recruitment module includes a decoding loss of movement data; During the training phase, the collaborative basis dictionary is used as a learnable parameter tensor, and parameter estimation is performed together with other parts of the network through the backpropagation method; (3) Construct an overall loss, and use the training dataset to train the dexterous hand movement decoding model; (4) Extract the EEG signal features to be predicted, input them into the trained dexterous hand movement decoding model, and decode to obtain the corresponding hand movement.

2. The dexterous hand movement decoding method based on collaborative manifolds according to claim 1, characterized in that In step (1), extract the spike band energy in the invasive EEG data as the EEG signal feature.

3. The dexterous hand movement decoding method based on collaborative manifolds according to claim 1, characterized in that During the parameter learning process of the collaborative basis dictionary, it is constrained by the regularization term of the dictionary, including the smoothing regularization term in the time dimension and the orthogonality regularization term between dictionary elements; the training of the collaborative basis recruitment module includes a regularization loss of dictionary learning.

4. The dexterous hand movement decoding method based on collaborative manifolds according to claim 1, wherein In step (3), the constructed overall loss includes the reconstruction loss of neural signal autoencoding from the periodic network module, the decoupling loss from the control network module, the decoding loss from the collaborative basis recruitment module, and the regularization loss of dictionary learning.

5. The dexterous hand movement decoding method based on collaborative manifolds according to claim 1, characterized in that In step (3), during the training process of the dexterous hand movement decoding model, parameter estimation is performed based on the backpropagation method.

Images