Method for decoding accurate finger movement track of macaque movement cortex

Abstract

The invention discloses a method for decoding an accurate finger movement track of a macaque movement cortex, and the method comprises the steps: obtaining neural data for decoding, and carrying out the data preprocessing; constructing an LSTM (Long Short Term Memory) model; selecting the size of a sliding window, training the constructed LSTM model, and performing performance evaluation after training; and (3) on the basis of the constructed LSTM model, adding a channel attention mechanism, and putting the sliding window selected in the step (3) into the LSTM model added with the channel attention mechanism for retraining to capture key neural signal features. The method is applied to a neural signal decoding motion trail, key features causing neural signals can be recognized, performance optimization can be further carried out, decoding precision is improved, different neural pathways correspond to a bionic mechanism, and decoding performance is improved at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of fast decoding of continuous fine hand movement data recorded in the primary motor cortex of a macaque, and in particular to a method for decoding precise finger movement trajectories of the motor cortex of a macaque. Background Art

[0002] Neural signals are ambiguous for hand movements. When using neural signals to decode motion trajectories, most existing methods use traditional machine learning or linear filters, which have insufficient decoding accuracy and lack basic biological neural structure implementation. It is still challenging to accurately decode motion trajectories from ambiguous neural signals, and there is still a lack of systematic methods for neural signal feature recognition and decoding strategies. Summary of the invention

[0003] The purpose of the present invention is to make up for the defects of the prior art and provide a method for decoding the precise finger movement trajectory of the motor cortex of macaques. The present invention quickly decodes the continuous fine hand movement data recorded in the primary motor cortex of macaques, effectively improves the decoding performance and accuracy, and is inspired by the corticospinal tract pathway and the cortical neuron pathway.

[0004] The present invention is achieved through the following technical solutions:

[0005] A method for decoding precise finger motion trajectories in the motor cortex of a macaque comprises the following steps:

[0006] (1) Obtain neural data for decoding and perform data preprocessing;

[0007] (2) Construct a two-layer LSTM model;

[0008] (3) Select the optimal sliding window size, sliding window step size, and the coordinates of the last window as the training data for each group, and train the constructed LSTM model. After training, perform performance evaluation and use the root mean square error between the true coordinates and the predicted coordinates as the evaluation index;

[0009] (4) Based on the constructed LSTM model, a channel attention mechanism SEBlock is added after the two layers of LSTM, and the sliding window selected in step (3) is put into the LSTM model after adding the channel attention mechanism for retraining to capture the key neural signal features;

[0010] (5) Adaptive parameter fusion XGBoost is added to the LSTM model after adding the channel attention mechanism in step (4); the parallel structure of XGBoost and the LSTM block form a parallel structure to dynamically adjust the weights of different channels.

[0011] The step (1) of obtaining neural data for decoding and performing data preprocessing is as follows: the neural data for decoding is characterized by the spike count matrix of all N recording cells. For all window sizes, the fixed interval between consecutive time windows is 70 milliseconds. A peak count matrix is ​​formed by connecting 10 consecutive windows. Based on this series of ten pulse count vectors, a neural network is trained to predict the trajectory position of the monkey's hand in the 10th window. Therefore, each sequence of 10 vectors, together with the trajectory position of the hand in the last window, constitutes a data point of a training model; the neural data for decoding and the continuous coordinate position of the hand are obtained from the motor cortex of the macaque, and the neuronal spike data of two groups of macaques and the x, y coordinates of the finger movement trajectory position at the corresponding time point are obtained, where the number of neurons N = 42 and the length of the statistical time window is 70ms, and the corresponding spike count matrix and the finger coordinates of the corresponding time dimension are obtained.

[0012] The two-layer LSTM model constructed in step (2) is specifically as follows: a 2-layer LSTM model is designed to process the spike count matrix, including an LSTM layer, an XGBoost model, an adaptive parameter fusion parameter, a channel attention layer and a dense layer, a 70%-30% data split is trained, which is used for training and verification respectively, and a ten-fold cross validation, the hidden layer nodes are 128, and the Adam Optimizer is used to train the network, the learning rate is 0.001, the batch size is 64, the epoch is 100, and the dropout is 0.5. The coordinates are output through the fully connected layer and the output layer. The model uses N=42-dimensional neural data and coordinates as a set of training data each time, and inputs the neural data of 42 neurons in multiple time windows to the model each time to generate a predicted coordinate, uses Log-cosh as the loss function, calculates the loss between the true coordinate and the actual coordinate, and updates the network weights by back propagation.

[0013] Step (3) specifically includes the following steps: evaluating the decoding performance with sliding window sizes of 1 to 12 and a sliding window step of 1, finally selecting the best sliding window size of 10 and a sliding window step of 1, and the coordinates of the last window as the training data for each group, and training the constructed LSTM model. After training, the performance is evaluated, and the root mean square error between the real coordinates and the predicted coordinates is used as the evaluation index:

[0014]

[0015]

[0016] error x Calculate the value of rmse for x;

[0017] errory Calculate the value of rmse for y;

[0018] error xy To comprehensively calculate the rmse values ​​of x and y;

[0019] L represents the number of data samples, that is, the length of the time point.

[0020] Y t , Y e , X t , X e Represent the true y value, predicted y value, true x value, and predicted x value respectively;

[0021] k represents the range from 0 to L-1, and is a symbol representing the value.

[0022] The channel attention mechanism SEBlock described in step (4) is an adaptive neural network module. It first uses adaptive average pooling to reduce the sequence length of each channel to a single value. Then, it uses two fully connected layers to reduce the number of channels from the original number of channels to the original number of channels and then restores it, and generates channel weights. The neural network module uses these weights to weight the input features and improve the network's expressiveness.

[0023] Step (5) specifically includes the following contents: the corticospinal tract is the most direct pathway, which calculates the long axons, dendritic spines and synaptic terminals of the cortical neuron pathway, and is a direct connection pathway between the cerebral cortex and the spinal cord, responsible for transmitting the brain's motor commands to the spinal cord, thereby controlling the body's voluntary movements. The corticospinal tract is a direct connection pathway between the cerebral cortex and the spinal cord. The cortical neuron pathway refers to the neural network formed by neurons in the cerebral cortex through their protrusions and other neurons or target organs. Neurons are connected to other neurons through dendrites receiving signals and axons transmitting signals, forming a complex network. Each neuron receives information, processes it and then transmits it, that is, the information flow in the decision tree. The decision tree constructed by XGBoost, each tree starts from the root node and gradually splits into multiple branches. Each branch represents a decision path, and finally reaches the leaf node and gives a prediction result. The two are combined and transmit the fine motor commands specified by the cortex to a single finger. The parallel structure of XGBoost and the LSTM block capture such fine-tuning direct control dynamics. The parallel structure is fused through adaptive parameters to dynamically adjust the weights of different pathways, thereby improving the decoding accuracy and corresponding to different neural pathways.

[0024] The advantages of the present invention are: 1. Feature search: The present invention introduces a channel attention mechanism to integrate the global and local features of neural data and enhance feature extraction;

[0025] 2. Performance optimization: The present invention can optimize the performance and greatly improve the decoding performance by constructing an LSTM model based on the channel attention mechanism and the adaptive parameter fusion of XGBosot;

[0026] 3. Brain-inspired model: The present invention is inspired by the corticospinal tract and cortical neuronal pathways to design this model, and strictly tests the decoding performance and accuracy through 5 groups of experiments. The decoding error performance on the x-axis and y-axis is improved by about 21.9% and 15.3% respectively compared with the current best decoding CSM-GDA model.

[0027] 4. The present invention is applied to the neural signal decoding motion trajectory, which can identify the key features that lead to the neural signal, and can further optimize the performance, improve the decoding accuracy, and correspond to different neural pathways through the bionic mechanism, while improving the decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 To compare the decoding curves of different models on the x-axis and y-axis;

[0029] Figure 2 It is the model structure diagram;

[0030] Figure 3 This is the channel attention mechanism diagram;

[0031] Figure 4 For macaque movement control experiments;

[0032] Figure 5 This is a data preprocessing flow chart of the present invention;

[0033] Figure 6 Schematic diagram of decoding performance per calculated by calculating the root mean square error between the real coordinates and the predicted coordinates. DETAILED DESCRIPTION

[0034] A method for decoding precise finger motion trajectories in the motor cortex of a macaque comprises the following steps:

[0035] (1) Obtain neural data for decoding and perform data preprocessing;

[0036] like Figure 5 As shown, the neural data used for decoding is characterized by the spike count matrix of all N recorded cells. For all window sizes, the fixed interval between consecutive time windows is 70 milliseconds. The spike count matrix is ​​formed by connecting 10 consecutive windows. Based on this series of ten spike count vectors, a neural network is trained to predict the trajectory position of the monkey's hand in the 10th window. Therefore, each sequence of 10 vectors, together with the trajectory position of the hand in the last window, constitutes a data point for training the model; the experimental design and collection of public data (such as Figure 4 As shown in the figure, neural data for decoding and the continuous coordinate positions of the hand are obtained from the motor cortex of the macaque, the neuron spike data of the two groups of macaques and the x, y coordinates of the finger motion trajectory positions at the corresponding time points are obtained, where the number of neurons N = 42, the length of the statistical time window is 70ms, and the corresponding spike count matrix and the finger coordinates of the corresponding time dimension are obtained.

[0037] (2) Construct a two-layer LSTM model;

[0038] like Figure 2 As shown in the figure, a 2-layer LSTM model is designed to process the spike count matrix, including an LSTM layer, an XGBoost model, an adaptive parameter fusion parameter, a channel attention layer and a dense layer. The data is split into 70%-30% for training and validation, respectively, and ten-fold cross validation. The hidden layer has 128 nodes. Adam Optimizer is used to train the network with a learning rate of 0.001, a batch size of 64, an epoch of 100, and a dropout of 0.5. The coordinates are output through the fully connected layer and the output layer. The model uses N=42-dimensional neural data and coordinates as a set of training data each time. The neural data of 42 neurons in multiple time windows are input to the model each time to generate a predicted coordinate. Log-cosh is used as the loss function to calculate the loss between the true coordinate and the actual coordinate, and the network weights are updated by back propagation.

[0039] (3) Select the optimal sliding window size, sliding window step size, and the coordinates of the last window as the training data for each group, and train the constructed LSTM model. After training, perform performance evaluation and use the root mean square error between the true coordinates and the predicted coordinates as the evaluation index;

[0040] Evaluate the decoding performance with sliding window sizes of 1 to 12 and a sliding window step of 1. Finally, select the best sliding window size of 10 and a sliding window step of 1. The coordinates of the last window are used as the training data for each group, and train the constructed LSTM model. After training, perform performance evaluation and use the root mean square error between the real coordinates and the predicted coordinates as the evaluation indicator:

[0041] like Figure 6 As shown in the figure, the best time window is selected, and the decoding performance on the y-axis is Per

[0042] Per=1-(error / Max error )

[0043] Max error =max(RMSE(Z t -mean Z e ))

[0044] error=RMSE(Z t -Z e )

[0045]

[0046] error x Calculate the value of rmse (root mean square error) for x;

[0047] error y Calculate the value of rmse for y;

[0048] error xy To comprehensively calculate the rmse values ​​of x and y;

[0049] L represents the number of data samples, that is, the length of the time point.

[0050] Y t , Y e , X t , X e Represent the true y value, predicted y value, true x value, and predicted x value respectively;

[0051] k represents the value from 0 to L-1, which is a symbol for the value. The per decoding performance is Figure 6 The y-axis percentages shown, Z t Refers to the real coordinates (x, y), Z e represents the predicted coordinates (x, y0, error represents the rmse of the actual and predicted coordinates,

[0052] Max error It represents the maximum value of rmse calculated by subtracting the mean of the predicted coordinates from the true coordinates.

[0053] (4) Based on the constructed LSTM model, a channel attention mechanism SEBlock is added after the two layers of LSTM, such as Figure 3 As shown, the sliding window selected in step (3) is put into the LSTM model after adding the channel attention mechanism for retraining to capture the key neural signal features;

[0054] The channel attention mechanism SEBlock described in step (4) is an adaptive neural network module. It first uses adaptive average pooling to reduce the sequence length of each channel to a single value. Then, it uses two fully connected layers to reduce the number of channels from the original number of channels to the original number of channels and then restores it, and generates channel weights. The neural network module uses these weights to weight the input features and improve the network's expressiveness.

[0055] (5) Adaptive parameter fusion XGBoost is added to the LSTM model after adding the channel attention mechanism in step (4); the parallel structure of XGBoost and the LSTM block form a parallel structure to dynamically adjust the weights of different channels.

[0056] Inspired by the neural pathway, adaptive parameters are added to fusion XGBoost based on step 4. The corticospinal tract is the most direct pathway, corresponding to the model structure proposed in step 4. Cortical neuron pathway,

[0057] The long axons, dendritic spines, and synaptic terminals of this pathway perform simple computations and transmit important cortical-specified fine motor commands to individual fingers. XGBoost's parallel structure and LSTM blocks capture such fine-tuned direct control dynamics. By adaptively fusion-parallel structures and dynamically adjusting the weights of different pathways, this method further improves decoding accuracy and corresponds to different neural pathways.

[0058] The corticospinal pathway is the most direct pathway, e.g. Figure 4 The arrow in the middle left part is the direct connection pathway between the cerebral cortex and the spinal cord, which is responsible for transmitting the brain's motor commands to the spinal cord, thereby controlling the body's voluntary movements. The corticospinal tract is a direct connection pathway between the cerebral cortex and the spinal cord. Because of its directness, it is considered to be the key to controlling fine movements (such as hand movements). Analogously, the LSTM neural network here processes neural signals to decode hand trajectories. The cortical neuronal pathway refers to the neural network formed by neurons in the cerebral cortex through their protrusions (such as axons and dendrites) and other neurons or target organs (such as muscles), such as Figure 4 Arrow on the right. Neurons are connected to other neurons through dendrites (receiving signals) and axons (transmitting signals), forming a complex network. Each neuron receives information, processes it, and then transmits it, which is similar to the flow of information in a decision tree. XGBoost builds a decision tree. Each tree starts from the root node and gradually splits into multiple branches. Each branch represents a decision path, and finally reaches the leaf node to give a prediction result. The two are combined and the fine motor commands specified by the cortex are transmitted to individual fingers. The parallel structure of XGBoost and the LSTM block capture such fine-tuning to directly control the dynamics. The parallel structure is fused through adaptive parameters to dynamically adjust the weights of different pathways, improve the decoding accuracy and correspond to different neural pathways.

[0059] Adaptive parameter nn.Parameter is a special type of tensor in PyTorch that is used to define learnable parameters in a neural network. It can be optimized through backpropagation and is automatically updated when the optimizer is called. The α in the formula is the parameter defined here. V 1 and V 2They represent the output of XGBoost and LSTM combined with the channel attention mechanism, that is, the predicted coordinates.

[0060] f=V 1 *α+V 2 *(1-α)

[0061] This part is adaptive parameter fusion, V 1 and V 2 They represent the output of XGBoost and LSTM combined with channel attention mechanism, and f is the final output of the two models. α combines the LSTM block and the parallel structure block of XGBoost and fine-tunes them. The obtained coordinates are used to directly control the dynamics of the fingers.

[0062] The present invention introduces a channel attention mechanism to fuse the global and local features of neural data and enhance feature extraction.

[0063] The present invention can optimize the performance and greatly improve the decoding performance by constructing an LSTM model based on the channel attention mechanism and the adaptive parameter fusion of XGBosot.

[0064] The present invention is inspired by the corticospinal tract and cortical neuronal pathways to design this model, and strictly tests the decoding performance and accuracy through 5 groups of experiments. The decoding error performance on the x-axis and y-axis is improved by about 21.9% and 15.3% respectively compared with the current best decoding CSM-GDA model.

[0065] Table 1

[0066] X-axis(Y-axis)estimation error,unit:CM

[0067]

[0068] Table 1X-axis(Y-axis)estimation error,unit:CM

[0069] Estimation error of the two-dimensional plane,unit:CM

[0070]

[0071] Table 2Estimation error of the two-dimensional plane,unit:CM

[0072] Table 1: Comparison of RMSE errors of different models on the x-axis and y-axis through 5 experiments. The model of the present invention has the lowest error and the best performance, which is better than the traditional decoding method.

[0073] Table 2

[0074] X-axis(Y-axis)estimation error,unit:CM

[0075]

[0076] Table 3X-axis(Y-axis)estimation error,unit:CM

[0077] Estimation error of the two-dimensional plane,unit:CM

[0078]

[0079] Table 4Estimation error of the two-dimensional plane,unit:CM

[0080] Table 2: Ablation experiment comparison model. Through 4 experiments to compare the RMSE errors of different models on the x-axis and y-axis, it can be found that the decoding performance is further improved after adding different modules.

[0081] like Figure 1 The figure shows the comparison of decoding curves of different models on the x-axis and y-axis. Figures AJ are the result graphs of the prediction of the first 70% of the first set of data and the 30% after training. A, C, E, and G on the left are the comparison of the true value and the predicted value of the x-axis, respectively comparing SCRNN simple convolutional recurrent neural network, linear model linear, XGBoost extreme gradient boosting, and our own Brain-FMC brain-inspired neural network for fine motor control. The right side is the comparison of the true value and the predicted value of the y-axis. The two bar graphs at the bottom represent the RMSE of the decoding error of the above four models on the x-axis and y-axis respectively. It can be intuitively seen from the figure that the decoding error of the Brain-FMC model of the present invention is significantly reduced when decoding the fine trajectories of the finger on the x-axis and y-axis.

Claims

1. A method for decoding precise finger motion trajectories in the motor cortex of macaques, characterized in that: The specific steps include: (1) Obtain neural data for decoding and perform data preprocessing; (2) Construct a two-layer LSTM model; (3) Select the optimal sliding window size, sliding window step size, and the coordinates of the last window as the training data for each group, and train the constructed LSTM model. After training, perform performance evaluation and use the root mean square error between the true coordinates and the predicted coordinates as the evaluation index; (4) Based on the constructed LSTM model, a channel attention mechanism SEBlock is added after the two layers of LSTM, and the sliding window selected in step (3) is put into the LSTM model after adding the channel attention mechanism for retraining to capture the key neural signal features; (5) Adaptive parameter fusion XGBoost is added to the LSTM model after adding the channel attention mechanism in step (4); the parallel structure of XGBoost and the LSTM block form a parallel structure to dynamically adjust the weights of different channels.

2. A method for decoding accurate finger motion trajectories in the motor cortex of a macaque according to claim 1, characterized in that: The step (1) of obtaining neural data for decoding and performing data preprocessing is as follows: the neural data for decoding is characterized by the spike count matrix of all N recording cells. For all window sizes, the fixed interval between consecutive time windows is 70 milliseconds. A peak count matrix is ​​formed by connecting 10 consecutive windows. Based on this series of ten pulse count vectors, a neural network is trained to predict the trajectory position of the monkey's hand in the 10th window. Therefore, each sequence of 10 vectors, together with the trajectory position of the hand in the last window, constitutes a data point of a training model; the neural data for decoding and the continuous coordinate position of the hand are obtained from the motor cortex of the macaque, and the neuronal spike data of two groups of macaques and the x, y coordinates of the finger movement trajectory position at the corresponding time point are obtained, where the number of neurons N = 42 and the length of the statistical time window is 70ms, and the corresponding spike count matrix and the finger coordinates of the corresponding time dimension are obtained.

3. A method for decoding accurate finger motion trajectories in the motor cortex of macaques according to claim 2, characterized in that: The two-layer LSTM model constructed in step (2) is specifically as follows: a 2-layer LSTM model is designed to process the spike count matrix, including an LSTM layer, an XGBoost model, an adaptive parameter fusion parameter, a channel attention layer and a dense layer, a 70%-30% data split is trained, which is used for training and verification respectively, and a ten-fold cross validation, the hidden layer nodes are 128, and the Adam Optimizer is used to train the network, the learning rate is 0.001, the batch size is 64, the epoch is 100, and the dropout is 0.

5. The coordinates are output through the fully connected layer and the output layer. The model uses N=42-dimensional neural data and coordinates as a set of training data each time, and inputs the neural data of 42 neurons in multiple time windows to the model each time to generate a predicted coordinate, uses Log-cosh as the loss function, calculates the loss between the true coordinate and the actual coordinate, and updates the network weights by back propagation.

4. A method for decoding accurate finger motion trajectories in the motor cortex of a macaque according to claim 3, characterized in that: Step (3) specifically includes the following steps: evaluating the decoding performance with sliding window sizes of 1 to 12 and a sliding window step of 1, finally selecting the best sliding window size of 10 and a sliding window step of 1, and the coordinates of the last window as the training data for each group, and training the constructed LSTM model. After training, the performance is evaluated, and the root mean square error between the real coordinates and the predicted coordinates is used as the evaluation index: error x Calculate the value of rmse for x; error y Calculate the value of rmse for y; error xy To comprehensively calculate the rmse value of x and y; L represents the number of data samples, that is, the length of the time point. Y t , Y e , X t , X e Represent the true y value, predicted y value, true x value, and predicted x value respectively; k represents the range from 0 to L-1, and is a symbol representing the value.

5. A method for decoding accurate finger motion trajectories in the motor cortex of macaques according to claim 3, characterized in that: The channel attention mechanism SEBlock described in step (4) is an adaptive neural network module. It first uses adaptive average pooling to reduce the sequence length of each channel to a single value. Then, it uses two fully connected layers to reduce the number of channels from the original number of channels to the original number of channels and then restores it, and generates channel weights. The neural network module uses these weights to weight the input features and improve the network's expressiveness.

6. A method for decoding accurate finger motion trajectories in the motor cortex of macaques according to claim 4, characterized in that: Step (5) specifically includes the following contents: the corticospinal tract is the most direct pathway, which calculates the long axons, dendritic spines and synaptic terminals of the cortical neuron pathway, and is a direct connection pathway between the cerebral cortex and the spinal cord, responsible for transmitting the brain's motor commands to the spinal cord, thereby controlling the body's voluntary movements. The corticospinal tract is a direct connection pathway between the cerebral cortex and the spinal cord. The cortical neuron pathway refers to the neural network formed by neurons in the cerebral cortex through their protrusions and other neurons or target organs. Neurons are connected to other neurons through dendrites receiving signals and axons transmitting signals, forming a complex network. Each neuron receives information, processes it and then transmits it, that is, the information flow in the decision tree. The decision tree constructed by XGBoost, each tree starts from the root node and gradually splits into multiple branches. Each branch represents a decision path, and finally reaches the leaf node and gives a prediction result. The two are combined and transmit the fine motor commands specified by the cortex to a single finger. The parallel structure of XGBoost and the LSTM block capture such fine-tuning direct control dynamics. The parallel structure is fused through adaptive parameters to dynamically adjust the weights of different pathways, thereby improving the decoding accuracy and corresponding to different neural pathways.

Images