A method for decoding precise finger movement trajectories of macaque motor cortex
By constructing a two-layer LSTM model and combining channel attention mechanism and XGBoost adaptive parameter fusion, the problem of insufficient accuracy in decoding hand motion trajectory in macaque motor cortex was solved, achieving higher decoding accuracy and feature extraction capability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI UNIV
- Filing Date
- 2024-09-06
- Publication Date
- 2026-07-14
AI Technical Summary
Existing neural signal decoding methods lack precision in decoding hand movement trajectories in macaque motor cortex, lack the realization of biological neural structures, and lack systematic feature recognition and decoding strategies.
We employ a two-layer LSTM model combined with the channel attention mechanism SEBlock and the adaptive parameter fusion of XGBoost. By training data through a sliding window, we capture key neural signal features and dynamically adjust the weights of different pathways, drawing inspiration from the corticospinal tract and cortical neuron pathways.
It significantly improves decoding accuracy, reducing decoding errors on the x-axis and y-axis by 21.9% and 15.3% respectively compared to existing methods, thereby enhancing decoding performance and feature extraction capabilities.
Smart Images
Figure CN119128438B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of technology for rapidly decoding continuous fine hand movement data recorded in the primary motor cortex of macaques, and particularly to a method for decoding precise finger movement trajectories in the motor cortex of macaques. Background Technology
[0002] Neural signals are ambiguous in relation to hand movements. Current methods for decoding motion trajectories using neural signals mostly rely on traditional machine learning or linear filters, resulting in insufficient decoding accuracy and a lack of fundamental biological neural structure implementations. Accurately decoding motion trajectories from ambiguous neural signals remains challenging, and a systematic approach to neural signal feature recognition and decoding strategies is still lacking. Summary of the Invention
[0003] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for decoding precise finger movement trajectories in the motor cortex of macaques. This invention rapidly decodes continuous fine hand movement data recorded in the primary motor cortex of macaques, effectively improving decoding performance and accuracy, and is inspired by the corticospinal tract pathway and cortical neuronal pathways.
[0004] This invention is achieved through the following technical solution:
[0005] A method for decoding precise finger movement trajectories in the motor cortex of macaques includes the following steps:
[0006] (1) Acquire neural data for decoding and perform data preprocessing;
[0007] (2) Construct a two-layer LSTM model;
[0008] (3) Select the best sliding window size, sliding window step size and the coordinates of the last window as the training data for each group, and train the constructed LSTM model. After training, evaluate the performance and use the root mean square error of the true coordinates and the predicted coordinates as the evaluation index.
[0009] (4) Based on the constructed LSTM model, a channel attention mechanism SEBlock is added after two layers of LSTM, and the sliding window selected in step (3) is put into the LSTM model after adding the channel attention mechanism for retraining to capture key neural signal features.
[0010] (5) Based on the LSTM model after adding the channel attention mechanism in step (4), add adaptive parameter fusion XGBoost; the parallel structure of XGBoost and the LSTM block form a parallel structure, and dynamically adjust the weights of different channels.
[0011] The acquisition of neural data for decoding and the data preprocessing in step (1) are as follows: The neural data for decoding is characterized by a peak count matrix of all N recording cells. For all window sizes, the fixed interval between consecutive time windows is 70 milliseconds. A peak count matrix is formed by connecting 10 consecutive windows. Based on this series of ten pulse count vectors, a neural network is trained to predict the trajectory position of the monkey's hand in the 10th window. Therefore, the sequence of 10 vectors, together with the trajectory position of the hand in the last window, constitutes a data point for training a model. Neural data for decoding and continuous coordinate positions of the hand are obtained from the motor cortex of the macaque. Two sets of neuron peak data and the x and y coordinates of the finger movement trajectory position at the corresponding time point are obtained. The number of neurons N = 42, the length of the statistical time window is 70 ms, and the corresponding peak count matrix and the finger coordinates in the corresponding time dimension are obtained.
[0012] The construction of the two-layer LSTM model described in step (2) is as follows: A two-layer LSTM model is designed to process the spike counting matrix, including an LSTM layer, an XGBoost model, an adaptive parameter fusion parameter, a channel attention layer and a dense layer. The data is split into 70%-30% for training and validation, as well as 10-fold cross-validation. The hidden layer has 128 nodes. The Adam Optimizer is used to train the network with a learning rate of 0.001, a batch size of 64, an epoch of 100, and a dropout of 0.5. The coordinates are output after passing through a fully connected layer and an output layer. The model takes N=42-dimensional neural data and coordinates as a set of training data each time. The neural data of 42 neurons in multiple time windows is input to the model each time to generate a predicted coordinate. Log-cosh is used as the loss function to calculate the loss between the true coordinate and the actual coordinate, and the network weights are updated through backpropagation.
[0013] Step (3) specifically includes the following steps: Evaluate the decoding performance using sliding window sizes of 1 to 12 and a sliding window step size of 1. Finally, select the optimal sliding window size as 10, the sliding window step size as 1, and use the coordinates of the last window as the training data for each group. Train the constructed LSTM model and evaluate its performance after training. Use the root mean square error of the true coordinates and the predicted coordinates as the evaluation metric.
[0014]
[0015]
[0016] error x Calculate the value of rmse for x;
[0017] errory To calculate the value of rmse for y;
[0018] error xy To comprehensively calculate the RMSE values of x and y;
[0019] L represents the number of data samples, which is also the length of the time point.
[0020] Y t Y e X t X e These represent the true value of y, the predicted value of y, the true value of x, and the predicted value of x, respectively.
[0021] k represents a range from 0 to L-1, and is a symbolic representation of the value.
[0022] The channel attention mechanism SEBlock described in step (4) is an adaptive neural network module. First, adaptive average pooling is used to reduce the sequence length of each channel to a single value. Then, the number of channels is reduced from the original number of channels to the original number of channels and then restored through two fully connected layers, and channel weights are generated. The neural network module uses these weights to weight the input features and improve the expressive power of the network.
[0023] Step (5) specifically includes the following: The corticospinal tract is the most direct pathway. Calculate the long axons, dendritic spines, and synaptic terminals of the cortical neuron pathway. It is the direct connection pathway between the cerebral cortex and the spinal cord, responsible for transmitting the brain's motor commands to the spinal cord, thereby controlling the body's voluntary movements. The corticospinal tract is the direct connection pathway between the cerebral cortex and the spinal cord. The cortical neuron pathway refers to the neural network formed by neurons in the cerebral cortex through their processes with other neurons or target organs. Neurons receive signals through dendrites and transmit signals through axons, connecting with other neurons to form a complex network. Each neuron receives information, processes it, and then transmits it out, which is the information flow in the decision tree. The decision tree constructed by XGBoost starts from the root node and gradually splits into multiple branches. Each branch represents a decision path, eventually reaching the leaf node and giving the prediction result. The two are combined and the fine motor commands specified by the cortex are transmitted to a single finger. The parallel structure of XGBoost and the fine-tuning of LSTM block capture directly control the dynamics. By using adaptive parameter fusion parallel structure, the weights of different pathways are dynamically adjusted, improving the decoding accuracy and corresponding to different neural pathways.
[0024] The advantages of this invention are: 1. Feature finding: This invention enhances feature extraction by introducing a channel attention mechanism, which integrates global and local features of neural data;
[0025] 2. Performance Optimization: This invention optimizes performance and significantly improves decoding performance by constructing an LSTM-based model, fusing it with channel attention mechanism and XGBosot's adaptive parameters.
[0026] 3. Brain-inspired model: Inspired by the corticospinal tract and cortical neuronal pathways, this invention designed this model and rigorously tested its decoding performance and accuracy through 5 sets of experiments. The decoding error performance on the x-axis and y-axis is improved by approximately 21.9% and 15.3% respectively compared to the current best decoding CSM-GDA model.
[0027] 4. This invention is applied to the decoding of neural signals in motion trajectories. It can identify the key features that cause neural signals and further optimize performance to improve decoding accuracy. Furthermore, it improves decoding performance by using a biomimetic mechanism to correspond to different neural pathways. Attached Figure Description
[0028] Figure 1 To compare the decoding curves of different models on the x-axis and y-axis;
[0029] Figure 2 This is a diagram of the model structure.
[0030] Figure 3 Diagram of channel attention mechanism;
[0031] Figure 4 For experiments on the motor control of macaques;
[0032] Figure 5 This is a flowchart of the data preprocessing process of the present invention;
[0033] Figure 6 This is a schematic diagram illustrating the decoding performance per inch calculated from the root mean square error of the true coordinates and the predicted coordinates. Detailed Implementation
[0034] A method for decoding precise finger movement trajectories in the motor cortex of macaques includes the following steps:
[0035] (1) Acquire neural data for decoding and perform data preprocessing;
[0036] like Figure 5 As shown, the neural data used for decoding is characterized by a spike count matrix of all N recorded cells. For all window sizes, the fixed interval between consecutive time windows is 70 milliseconds. A spike count matrix is formed by concatenating 10 consecutive windows. Based on this series of ten spike count vectors, a neural network is trained to predict the trajectory position of the monkey's hand in the 10th window. Therefore, the sequence of every 10 vectors, along with the trajectory position of the hand in the last window, constitutes a data point for training the model. The experimental design and collection of publicly available data (such as...) Figure 4 As shown, neural data for decoding and continuous hand coordinates are obtained from the motor cortex of macaques. Two sets of neuronal spike data and the x and y coordinates of the finger movement trajectory at the corresponding time points are obtained. The number of neurons N = 42 and the length of the statistical time window is 70ms. The corresponding spike count matrix and the finger coordinates in the corresponding time dimension are obtained.
[0037] (2) Construct a two-layer LSTM model;
[0038] like Figure 2 As shown, a two-layer LSTM model is designed to process the spike counting matrix, including an LSTM layer, an XGBoost model, an adaptive parameter fusion parameter, a channel attention layer, and a dense layer. The training uses 70%-30% of the data splits for training and validation, as well as 10-fold cross-validation. The hidden layer has 128 nodes. The Adam Optimizer is used to train the network with a learning rate of 0.001, a batch size of 64, 100 epochs, and a dropout of 0.5. The coordinates are output after passing through a fully connected layer and an output layer. The model uses N=42 dimensional neural data and coordinates as a set of training data each time. It inputs neural data from 42 neurons across multiple time windows to generate a predicted coordinate. Log-cosh is used as the loss function to calculate the loss between the true and actual coordinates, and backpropagation is used to update the network weights.
[0039] (3) Select the best sliding window size, sliding window step size and the coordinates of the last window as the training data for each group, and train the constructed LSTM model. After training, evaluate the performance and use the root mean square error of the true coordinates and the predicted coordinates as the evaluation index.
[0040] The decoding performance was evaluated using sliding window sizes from 1 to 12 with a step size of 1. The optimal sliding window size was ultimately selected as 10 with a step size of 1. The coordinates of the last window were used as the training data for each group, and the constructed LSTM model was trained on it. Performance was then evaluated, using the root mean square error between the true and predicted coordinates as the evaluation metric.
[0041] like Figure 6 As shown, selecting the optimal time window, the decoding performance on the y-axis is Per.
[0042] Per = 1 - (error / Max) error )
[0043] Max error =max(RMSE(Z) t -mean Z e ))
[0044] error = RMSE(Z) t -Z e )
[0045]
[0046] error x Calculate the value of rmse (root mean square error) for x;
[0047] error y To calculate the value of rmse for y;
[0048] error xy To comprehensively calculate the RMSE values of x and y;
[0049] L represents the number of data samples, which is also the length of the time point.
[0050] Y t Y e X t X e These represent the true value of y, the predicted value of y, the true value of x, and the predicted value of x, respectively.
[0051] k represents a range from 0 to L-1, and is a symbolic representation of the value. The decoding performance of per is... Figure 6 The percentage shown on the y-axis, Z t Represents the actual coordinates (x, y), Z e The predicted coordinates (x, y) represent the calculated RMS / SE values between the actual and predicted coordinates.
[0052] Max error The maximum value of RMSE is calculated by subtracting the mean of the predicted coordinates from the actual coordinates.
[0053] (4) Based on the constructed LSTM model, a channel attention mechanism SEBlock is added after the two LSTM layers, such as... Figure 3 As shown, the sliding window selected in step (3) is placed into the LSTM model after the channel attention mechanism is added for retraining to capture key neural signal features;
[0054] The channel attention mechanism SEBlock described in step (4) is an adaptive neural network module. First, adaptive average pooling is used to reduce the sequence length of each channel to a single value. Then, the number of channels is reduced from the original number of channels to the original number of channels and then restored through two fully connected layers, and channel weights are generated. The neural network module uses these weights to weight the input features and improve the expressive power of the network.
[0055] (5) Based on the LSTM model after adding the channel attention mechanism in step (4), add adaptive parameter fusion XGBoost; the parallel structure of XGBoost and the LSTM block form a parallel structure, and dynamically adjust the weights of different channels.
[0056] Inspired by neural pathways, adaptive parameter fusion XGBoost was added based on step 4. The corticospinal tract is the most direct pathway, corresponding to the model structure proposed in step 4. Cortical neuron pathway,
[0057] This pathway performs simple calculations on the long axons, dendritic spines, and synaptic terminals, transmitting crucial cortical-specific fine motor commands to individual fingers. The parallel architecture of XGBoost, combined with LSTM block capture, allows for direct control of the dynamics through fine-tuning. By adaptively fusing the parallel architecture with different parameters and dynamically adjusting the weights of different pathways, this method further improves decoding accuracy and corresponds to different neural pathways.
[0058] The corticospinal tract pathway is the most direct pathway, such as... Figure 4 The arrow on the left side of the diagram represents the direct connection pathway between the cerebral cortex and the spinal cord, responsible for transmitting motor commands from the brain to the spinal cord, thereby controlling voluntary body movements. The corticospinal tract is the direct connection pathway between the cerebral cortex and the spinal cord, and due to its directness, it is considered key to controlling fine motor skills (such as hand movements). This can be compared to the LSTM neural network used here for decoding neural signals related to hand trajectories. Cortical neuronal pathways refer to the neural networks formed by neurons in the cerebral cortex through their processes (such as axons and dendrites) with other neurons or target organs (such as muscles), such as… Figure 4 The arrow on the right illustrates this. Neurons connect to other neurons via dendrites (for receiving signals) and axons (for transmitting signals), forming a complex network. Each neuron receives information, processes it, and then transmits it out, similar to the information flow in a decision tree. XGBoost constructs a decision tree, with each tree starting from the root node and progressively splitting into multiple branches, each representing a decision path, ultimately reaching a leaf node to provide a prediction. Combining these two approaches, XGBoost transmits fine motor commands specified by the cortex to a single finger. The parallel structure of XGBoost, along with fine-tuning such as LSTM block capture, directly controls the dynamics. By adaptively fusing the parallel structure with parameters, it dynamically adjusts the weights of different pathways, improving decoding accuracy and corresponding to different neural pathways.
[0059] The adaptive parameter `nn.Parameter` is a special type of tensor in PyTorch used to define learnable parameters in a neural network. It can be optimized through backpropagation and is automatically updated when the optimizer is invoked. The `α` in the formula is the parameter defined here. `V1` and `V2` represent the outputs of XGBoost and the LSTM combined with channel attention, respectively, which are the predicted coordinates.
[0060] f = V1*α + V2*(1-α)
[0061] This part is adaptive parameter fusion. V1 and V2 represent the outputs of XGBoost and LSTM combined with channel attention mechanism, respectively, and f is the final output combining the two models. α combines and fine-tunes the parallel structure block of LSTM and XGBoost, and the resulting coordinates are used to directly control the dynamics of the finger.
[0062] This invention enhances feature extraction by introducing a channel attention mechanism, which integrates global and local features of neural data.
[0063] This invention optimizes performance and significantly improves decoding performance by constructing an LSTM-based model, fusing it with channel attention mechanisms and XGBosot's adaptive parameters.
[0064] Inspired by the corticospinal tract and cortical neuronal pathways, this invention designed the model and rigorously tested its decoding performance and accuracy through five sets of experiments. The decoding error performance on the x-axis and y-axis is improved by approximately 21.9% and 15.3% respectively compared to the current best decoding CSM-GDA model.
[0065] Table 1
[0066] X-axis(Y-axis)estimation error,unit:CM
[0067]
[0068] Table 1X-axis(Y-axis)estimation error,unit:CM
[0069] Estimation error of the two-dimensional plane,unit:CM
[0070]
[0071] Table 2Estimation error of the two-dimensional plane,unit:CM
[0072] Table 1: Comparison of RMSE errors on the x and y axes of different models through five experiments. The model of this invention has the lowest error and the best performance, outperforming traditional decoding methods.
[0073] Table 2
[0074] X-axis(Y-axis)estimation error,unit:CM
[0075]
[0076] Table 3X-axis(Y-axis)estimation error,unit:CM
[0077] Estimation error of the two-dimensional plane,unit:CM
[0078]
[0079] Table 4Estimation error of the two-dimensional plane,unit:CM
[0080] Table 2: Comparison of ablation experiment models. By comparing the RMSE errors of different models on the x-axis and y-axis in four experiments, it can be found that the decoding performance is further improved after adding different modules.
[0081] like Figure 1 The figures show a comparison of the decoding curves of different models on the x and y axes. Figures A and J represent the results of training the first 70% of the data and predicting the remaining 30%. On the left, A, C, E, and G compare the true and predicted values on the x-axis, respectively: SCRNN (simple convolutional recurrent neural network), the linear model, XGBoost (extreme gradient boosting), and our own Brain-FMC (brain-inspired neural network for fine motion control). The right side compares the true and predicted values on the y-axis. The two bars at the bottom represent the decoding errors (RMSE) of the four models on the x and y axes. The figures clearly show that the Brain-FMC model of this invention significantly reduces the decoding error when decoding the fine trajectory of a finger on the x and y axes.
Claims
1. A method for decoding precise finger movement trajectories in the motor cortex of macaques, characterized in that: Specifically, the steps include the following: (1) Acquire neural data for decoding and perform data preprocessing; (2) Construct a two-layer LSTM model; (3) Select the best sliding window size, sliding window step size and the coordinates of the last window as the training data for each group, and train the constructed LSTM model. After training, evaluate the performance and use the root mean square error of the true coordinates and the predicted coordinates as the evaluation index. (4) Based on the constructed LSTM model, a channel attention mechanism SEBlock is added after two layers of LSTM, and the sliding window selected in step (3) is put into the LSTM model after adding the channel attention mechanism for retraining to capture key neural signal features. (5) Based on the LSTM model after adding the channel attention mechanism in step (4), add adaptive parameter fusion XGBoost; the parallel structure of XGBoost and the LSTM block form a parallel structure, and dynamically adjust the weights of different channels.
2. The method for decoding precise finger movement trajectories in the motor cortex of macaques according to claim 1, characterized in that: The acquisition of neural data for decoding and the data preprocessing in step (1) are as follows: The neural data for decoding is characterized by a peak count matrix of all N recording cells. For all window sizes, the fixed interval between consecutive time windows is 70 milliseconds. A peak count matrix is formed by connecting 10 consecutive windows. Based on this series of ten pulse count vectors, a neural network is trained to predict the trajectory position of the monkey's hand in the 10th window. Therefore, the sequence of 10 vectors, together with the trajectory position of the hand in the last window, constitutes a data point for training a model. Neural data for decoding and continuous coordinate positions of the hand are obtained from the motor cortex of the macaque. Two sets of neuron peak data and the x and y coordinates of the finger movement trajectory position at the corresponding time point are obtained. The number of neurons N = 42, the length of the statistical time window is 70 ms, and the corresponding peak count matrix and the finger coordinates in the corresponding time dimension are obtained.
3. The method for decoding precise finger movement trajectories in the motor cortex of macaques according to claim 2, characterized in that: The construction of the two-layer LSTM model described in step (2) is as follows: A two-layer LSTM model is designed to process the spike counting matrix, including an LSTM layer, an XGBoost model, an adaptive parameter fusion parameter, a channel attention layer and a dense layer. The data is split into 70%-30% for training and validation, as well as 10-fold cross-validation. The hidden layer has 128 nodes. The Adam Optimizer is used to train the network with a learning rate of 0.001, a batch size of 64, an epoch of 100, and a dropout of 0.
5. The coordinates are output after passing through a fully connected layer and an output layer. The model takes N=42-dimensional neural data and coordinates as a set of training data each time. The neural data of 42 neurons in multiple time windows is input to the model each time to generate a predicted coordinate. Log-cosh is used as the loss function to calculate the loss between the true coordinate and the actual coordinate, and the network weights are updated through backpropagation.
4. The method for decoding precise finger movement trajectories in the motor cortex of macaques according to claim 3, characterized in that: Step (3) specifically includes the following steps: Evaluate the decoding performance using sliding window sizes of 1 to 12 and a sliding window step size of 1. Finally, select the optimal sliding window size as 10, the sliding window step size as 1, and use the coordinates of the last window as the training data for each group. Train the constructed LSTM model and evaluate its performance after training. Use the root mean square error of the true coordinates and the predicted coordinates as the evaluation metric. error x To calculate the value of rmse for x; error y To calculate the value of rmse for y; error xy To comprehensively calculate the RMSE values of x and y; L represents the number of data samples, which is also the length of the time point. Y t Y e X t X e These represent the true value of y, the predicted value of y, the true value of x, and the predicted value of x, respectively. k represents a range from 0 to L-1, and is a symbolic representation of the value.
5. The method for decoding precise finger movement trajectories in the motor cortex of macaques according to claim 3, characterized in that: The channel attention mechanism SEBlock described in step (4) is an adaptive neural network module. First, adaptive average pooling is used to reduce the sequence length of each channel to a single value. Then, the number of channels is reduced from the original number of channels to the original number of channels and then restored through two fully connected layers, and channel weights are generated. The neural network module uses these weights to weight the input features and improve the expressive power of the network.
6. The method for decoding precise finger movement trajectories in the motor cortex of macaques according to claim 4, characterized in that: Step (5) specifically includes the following: The corticospinal tract is the most direct pathway. Calculate the long axons, dendritic spines, and synaptic terminals of the cortical neuron pathway. It is the direct connection pathway between the cerebral cortex and the spinal cord, responsible for transmitting the brain's motor commands to the spinal cord, thereby controlling the body's voluntary movements. The corticospinal tract is the direct connection pathway between the cerebral cortex and the spinal cord. The cortical neuron pathway refers to the neural network formed by neurons in the cerebral cortex through their processes with other neurons or target organs. Neurons receive signals through dendrites and transmit signals through axons, connecting with other neurons to form a complex network. Each neuron receives information, processes it, and then transmits it out, which is the information flow in the decision tree. The decision tree constructed by XGBoost starts from the root node and gradually splits into multiple branches. Each branch represents a decision path, eventually reaching the leaf node and giving the prediction result. The two are combined and the fine motor commands specified by the cortex are transmitted to a single finger. The parallel structure of XGBoost and the fine-tuning of LSTM block capture directly control the dynamics. By using adaptive parameter fusion parallel structure, the weights of different pathways are dynamically adjusted, improving the decoding accuracy and corresponding to different neural pathways.