Neuron signal space-time joint decoding method and system for brain-computer interface and electronic device

Abstract

The application provides a neuron signal space-time joint decoding method and system for a brain-computer interface and an electronic device, relates to the field of brain-computer interface control, and utilizes a space adjacency matrix and a time sequence adjacency matrix of a target neuron to accurately acquire a space-time joint relationship between neuron signals and brain intentions, so that end-to-end accurate mapping from neuron signal acquisition to brain intention decoding is realized, various noise interferences can be effectively overcome, and the decoding accuracy of the neuron signals is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface control, and in particular to a spatiotemporal joint decoding method, system and electronic device for neuronal signals for brain-computer interfaces. Background Technology

[0002] In brain-computer interface systems, decoding neural group activity is a core technical component, and its decoding accuracy directly determines control stability and interaction fluency. The conventional decoding process involves acquiring spike potential signals from multiple neurons using a microelectrode array, extracting and decoding the temporal firing rate data of each neuron to obtain the brain's intention feature information corresponding to the neuronal signals. However, in real-world scenarios, random noise in the neuronal's own electrical activity, signal amplitude fluctuations caused by electrode micro-drift, individual differences in neuronal response characteristics at different times, and firing pattern shifts due to changes in brain state can all significantly impact the decoding of neural signals.

[0003] To address these issues, existing technologies primarily employ manual feature design to preprocess neural signals (e.g., noise filtering, firing rate calibration, and spike potential sorting) before inputting them into traditional population vector models for decoding; or they optimize acquisition conditions to reduce the impact of signal interference. However, these two approaches still cannot fundamentally resolve decoding errors caused by firing feature distortion. Manual features rely on expert experience, have limited extraction accuracy, and cannot adapt to dynamically changing neural activity; optimizing acquisition conditions requires additional hardware costs and is difficult to implement in complex environments such as long-term implantation or mobile scenarios. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a spatiotemporal joint decoding method, system and electronic device for neuronal signals for brain-computer interfaces. This method uses the spatial adjacency matrix and temporal adjacency matrix of the target neuron to accurately obtain the spatiotemporal joint relationship between neuronal signals and brain intentions, thereby realizing end-to-end accurate mapping from neuronal signal acquisition to brain intention decoding. It can effectively overcome various noise interferences and significantly improve the decoding accuracy of neuronal signals.

[0005] In a first aspect, embodiments of the present invention provide a spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces, the method comprising: The neuronal signals of the neural clusters corresponding to the target brain-computer interface are acquired through a multi-channel electrode array in each acquisition channel. The firing rate of the target neuron is calculated using the number of action potentials of neuron signals within a preset time window, and a spatial adjacency matrix of the target neuron is constructed based on the spatial Euclidean distance between the target neuron and other neurons. After truncating the neuron signal using a preset truncation length, the time series segment of the target neuron is obtained, and the temporal adjacency matrix of the target neuron is constructed based on the temporal dependency between the time series segments. After calculating the attention weights of the target neuron based on the spatial adjacency matrix and the temporal adjacency matrix, the spatiotemporal joint feature map of the target neuron under the attention weights is calculated using the firing rate and time series segments. The decoding results of neuron signals are obtained based on the spatiotemporal joint feature map.

[0006] Optionally, the steps of calculating the firing rate of the target neuron using the number of action potentials of neuron signals within a preset time window, and constructing a spatial adjacency matrix of the target neuron based on the spatial Euclidean distance between the target neuron and other neurons, include: The peak potentials of neuron signals in each acquisition channel within a preset time window are obtained, and the number of action potentials corresponding to the target neuron is obtained using the peak potentials. The firing rate of the target neuron is obtained based on the ratio of the number of action potentials to the length of the time window. The hyperparameters of the target neuron are determined by the channel difference between the first acquisition channel of the target neuron and the second acquisition channel of other neurons, and the spatial Euclidean distance between the target neuron and other neurons is calculated. The spatial adjacency matrix of the target neuron is constructed using spatial Euclidean distance and hyperparameters; where, the spatial adjacency matrix... ; For spatial Euclidean distance. For hyperparameters, The target neuron in the spatial topological adjacency matrix First acquisition channel The target neuron and the second acquisition channel Other neurons below.

[0007] Optionally, the step of truncating the neuron signal using a preset truncation length to obtain a time-series segment of the target neuron, and constructing a temporal adjacency matrix of the target neuron based on the temporal dependencies between the time-series segments, includes: After sequentially truncating the neuron signals from each acquisition channel using a preset truncation length, continuous time series segments of the target neuron are obtained, and a temporal feature matrix of the neuron signal is constructed based on the continuous time series segments and the number of acquisition channels. Determine the temporal dependencies between time series segments based on the temporal feature matrix; Based on the temporal dependencies, a temporal adjacency matrix of the target neuron is constructed using consecutive time series segments.

[0008] Optionally, the attention weights of the target neuron are calculated based on the spatial adjacency matrix and the temporal adjacency matrix, including: Obtain the graph attention network corresponding to the spatial adjacency matrix and temporal adjacency matrix, and obtain the firing rate and node features of time series segments in the graph attention network; Obtain linear transformation weights and attention vectors corresponding to node features based on graph attention networks; The linear transformation output of adjacent nodes in the graph attention network is obtained by multiplying the linear transformation weights with the node features. The attention coefficients of neighboring nodes are calculated using the linear transformation output and the attention vector. The attention weight of the target neuron in the graph attention network is calculated based on the attention coefficient.

[0009] Optionally, obtain the graph attention network corresponding to the spatial adjacency matrix and temporal adjacency matrix, including: Using the target neuron as the graph node, the firing rate as the node feature, and a spatial graph attention network is constructed based on the spatial association weights between neurons in the spatial adjacency matrix. Using time windows as graph nodes and the collective firing rate of neuron signals as node features, a temporal graph attention network is constructed based on the temporal evolution data of neuron signals in the temporal adjacency matrix.

[0010] Optionally, the loss function used in the construction of the graph attention network is: ; Where N is the batch size. The decoding result of the graph attention network output, The true label for the node features.

[0011] Optionally, the spatiotemporal joint feature map of the target neuron under attention weights can be calculated using the firing rate and time series segments, including: Obtain the first and second attention weights of the release rate and time series segment in the graph attention network, respectively; The spatial feature aggregation result of the target neuron corresponding to the first attention weight is calculated based on the output result of the first linear transformation under the spatial adjacency matrix; The temporal feature aggregation result of the target neuron corresponding to the second attention weight is calculated based on the output result of the second linear transformation under the temporal adjacency matrix; After convolutionally fusing and concatenating the spatial feature aggregation results with the temporal feature aggregation results in the channel dimension of the graph attention network, the spatiotemporal joint feature map of the target neuron is obtained.

[0012] Optionally, the step of obtaining the decoding result of the neuron signal based on the spatiotemporal joint feature map includes: Obtaining a multidimensional feature matrix of spatiotemporal joint feature map based on spatial adjacency matrix and temporal adjacency matrix; After flattening the multidimensional feature matrix by adjusting the number of dimensions of the spatiotemporal joint feature map, a one-dimensional feature vector of the spatiotemporal joint feature map is obtained. The correlation calculation results between node features in a one-dimensional feature vector are calculated using motion intention category parameters and visual stimulus category parameters. The decoding result of the neuron signal corresponding to the correlation calculation result is calculated by using a linear activation function associated with the spatiotemporal joint feature map.

[0013] Secondly, this invention provides a spatiotemporal joint decoding system for neuronal signals for brain-computer interfaces, the system comprising: Neuron signal acquisition module: used to acquire neuronal signals of the target brain-computer interface corresponding to the neural cluster in each acquisition channel through a multi-channel electrode array; Spatial Adjacency Matrix Construction Module: Used to calculate the firing rate of the target neuron using the number of action potentials of neuron signals within a preset time window, and to construct the spatial adjacency matrix of the target neuron based on the spatial Euclidean distance between the target neuron and other neurons; Temporal adjacency matrix construction module: used to truncate the neuron signal using a preset truncation length to obtain the time sequence segment of the target neuron, and construct the temporal adjacency matrix of the target neuron according to the temporal dependency relationship between the time sequence segments; Spatiotemporal joint feature map calculation module: After calculating the attention weights of the target neuron based on the spatial adjacency matrix and the temporal adjacency matrix, it uses the firing rate and time series segments to calculate the spatiotemporal joint feature map of the target neuron under the attention weights; Spatiotemporal joint decoding and acquisition module for neuronal signals for brain-computer interfaces: used to obtain the decoding results of neuronal signals based on spatiotemporal joint feature maps.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, which includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, and the processor executing the computer-executable instructions to implement the steps of the spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces provided in the first aspect.

[0015] This invention provides a spatiotemporal joint decoding method, system, and electronic device for neuronal signals in brain-computer interfaces. In decoding the brain intent of neuronal signals in a neural cluster corresponding to a target brain-computer interface, the method first acquires neuronal signals from the neural cluster in each acquisition channel using a multi-channel electrode array. Then, it calculates the firing rate of the target neuron using the number of action potentials in the neuronal signals within a preset time window, and constructs a spatial adjacency matrix of the target neuron based on the Euclidean distance between the target neuron and other neurons. Furthermore, it truncates the neuronal signals using a preset truncation length to obtain time-series segments of the target neuron, and constructs a temporal adjacency matrix of the target neuron based on the temporal dependencies between these segments. Next, it calculates the attention weight of the target neuron based on the spatial and temporal adjacency matrices, and then calculates the spatiotemporal joint feature map of the target neuron under the attention weight using the firing rate and time-series segments. Finally, it obtains the decoding result of the neuronal signal based on the spatiotemporal joint feature map. This method utilizes the spatial adjacency matrix and temporal adjacency matrix of the target neuron to accurately obtain the spatiotemporal joint relationship between neuronal signals and brain intentions, thereby achieving end-to-end accurate mapping from neuronal signal acquisition to brain intention decoding. It can effectively overcome various noise interferences and significantly improve the decoding accuracy of neuronal signals.

[0016] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 A flowchart of the first spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces provided in this embodiment of the invention; Figure 2 A flowchart of a second spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces provided in an embodiment of the present invention; Figure 3A flowchart illustrating the third spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces provided in this embodiment of the invention; Figure 4 This is a schematic diagram of the structure of the fourth brain-computer interface-oriented spatiotemporal joint decoding system for neuronal signals provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a spatiotemporal joint decoding system for neuronal signals for brain-computer interfaces, provided in an embodiment of the present invention. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0020] icon: 100 - Neuron signal acquisition module; 200 - Spatial adjacency matrix construction module; 300 - Temporal adjacency matrix construction module; 400 - Spatiotemporal joint feature map calculation module; 500 - Spatiotemporal joint decoding and acquisition module for neuron signals for brain-computer interfaces; 101 - Processor; 102 - Memory; 103 - Bus; 104 - Communication interface. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] To facilitate understanding of this embodiment, a spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces, disclosed in this embodiment of the invention, will first be described in detail. This method is as follows: Figure 1 As shown, it includes: Step S101: Acquire neuronal signals of the neural cluster corresponding to the target brain-computer interface in each acquisition channel through a multi-channel electrode array.

[0023] By pre-implanting a multi-channel microelectrode array device (i.e., a brain-computer interface) into the target brain region, continuous real-time signal acquisition is performed on the corresponding neural cluster, acquiring raw neuronal electrical signals from each acquisition channel. The acquired neuronal signals include complete raw data such as spike potential signals and continuous electrical activity signals of each neuron within the neural cluster, covering the overall electrical activity state of the neural cluster. This provides a comprehensive, raw, and complete data source for subsequent neural feature extraction, matrix construction, and intent decoding, ensuring the integrity and accuracy of subsequent feature analysis.

[0024] Step S102: Calculate the firing rate of the target neuron using the number of action potentials of neuron signals within a preset time window, and construct the spatial adjacency matrix of the target neuron based on the spatial Euclidean distance between the target neuron and other neurons.

[0025] First, a fixed preset time window is set to adapt to the electrical activity rhythm of the neural cluster. The total number of action potentials (spiking potentials) generated by the target neurons within each time window is counted. Combined with the duration parameter of the time window, the real-time firing rate of each target neuron is accurately calculated, thereby quantifying the real-time activation intensity and firing activity of a single neuron.

[0026] Secondly, the spatial distribution coordinates of all neurons within the neural cluster are accurately located, and the spatial Euclidean distance between the target neuron and all other neurons in the cluster is calculated one by one to quantify the spatial positional relationship between neurons.

[0027] Based on the spatial distance correlation between neurons, a spatial adjacency matrix is ​​constructed through normalization and weight assignment to accurately reflect the spatial distribution coupling relationship and the degree of spatial correlation of neurons within a neural cluster, thus fully characterizing the spatial dimensional features of neural signals.

[0028] Step S103: After truncating the neuron signal using a preset truncation length, the time sequence segment of the target neuron is obtained, and the temporal adjacency matrix of the target neuron is constructed based on the temporal dependency relationship between the time sequence segments.

[0029] By pre-setting the signal truncation length and temporal truncation interval to match the dynamic response characteristics of neurons, the continuous raw neuronal signals acquired in step S101 are segmented to obtain several groups of time-continuous, uniform-length, and ordered target neuronal time-series segments, avoiding feature errors caused by inconsistent signal durations and temporal disorder. Based on this, the temporal correlation patterns between each group of time-series segments are deeply explored. Combining the temporal continuity and sequential dependence characteristics of neural activity, the temporal feature relationships such as the evolutionary correlation of signal firing patterns, amplitude changes, and activity linkages of time-series segments at different times are quantified. Based on the temporal dependence weights of each time-series segment, a temporal adjacency matrix is ​​constructed that can comprehensively characterize the dynamic evolution of neuronal signals in the temporal dimension, accurately covering the temporal dynamic features of neural activity.

[0030] Step S104: After calculating the attention weights of the target neuron based on the spatial adjacency matrix and the temporal adjacency matrix, the spatiotemporal joint feature map of the target neuron under the attention weights is calculated using the firing rate and time series segments.

[0031] By integrating the constructed spatial and temporal adjacency matrices of neurons, and considering two core dimensions—spatial coupling and correlation of neural clusters and dynamic temporal evolution of neural signals—the dynamic attention weights corresponding to each target neuron are adaptively calculated. These attention weights can autonomously strengthen the weight ratio of effective neural features while weakening the influence of ineffective interference features such as random noise, signal drift, and feature distortion, thus achieving intelligent screening and optimization of neural features. Subsequently, the real-time firing rate data of neurons obtained in step S102 is fused with the standardized time series segments extracted in step S103. Based on the adaptive attention weights, the spatial and temporal features of neurons are weighted, fused, mapped in multiple dimensions, and aggregated. Finally, a high-precision spatiotemporal joint feature map of neurons, possessing both spatial correlation and temporal dynamic characteristics, is generated, achieving deep integration of multidimensional effective features of neural signals.

[0032] Step S105: Obtain the decoding result of the neuron signal based on the spatiotemporal joint feature map.

[0033] The final generated spatiotemporal joint feature map of neurons is input into a pre-constructed neural decoding mapping table (or model, module, database). Based on the intrinsic correlation logic between neuronal signals and brain intentions carried by the spatiotemporal joint feature map, the core intention information of brain perception, movement, and control corresponding to multi-dimensional aggregated neural features is deeply mined. This completes the end-to-end accurate mapping from raw neuronal electrical signals to brain intention features, and finally outputs spatiotemporal joint decoding results of neuronal signals for brain-computer interfaces that are highly resistant to interference, highly accurate, and stable. This effectively solves the technical problem that traditional decoding methods are easily affected by neural dynamic changes and signal interference.

[0034] The aforementioned spatiotemporal joint decoding method for neuronal signals in brain-computer interfaces can be implemented using a two-branch parallel graph attention network (GAT) architecture, as detailed below. Figure 2 As shown, the specific process is as follows: The input layer provides two types of basic input data, corresponding to the spatial and temporal dimensions of neuronal activity, respectively, forming a parallel dual-branch input structure.

[0035] 1. Input layer branch 1 (spatial dimension input).

[0036] Input data includes a neuron firing rate matrix. Spatial adjacency matrix .in, The number of neurons within a neural cluster. The time sampling dimension for the distribution rate (corresponding to the number of preset time windows, such as...) The firing rate matrix quantifies the firing intensity of each neuron in each time window, directly reflecting the real-time activation state of the neuron; the spatial adjacency matrix is ​​constructed based on the spatial Euclidean distance between neurons, characterizing the spatial coupling relationship of neurons within the neural cluster, and providing topological constraints for spatial dimension feature interactions.

[0037] 2. Input layer branch 2 (time-series dimension input).

[0038] Input data includes neuron time series matrices. With temporal adjacency matrix .in, The time series matrix is ​​the length of the time series segments after the neuronal signal is segmented according to a preset length; the time series matrix preserves the dynamic evolution details of each neuronal signal and fully presents the temporal change pattern of neural activity; the temporal adjacency matrix is ​​constructed based on the sequential dependency relationship between time series segments, characterizes the temporal correlation law of neuronal activity, and provides topological constraints for the feature interaction of the temporal dimension.

[0039] 3. Spatial GAT feature extraction layer (spatial dimension adaptive feature extraction).

[0040] This layer is based on the Graph Attention Network (GAT) to extract spatial features from the input data of branch 1. Through linear transformation, attention calculation and feature aggregation, it adaptively captures the spatial collaborative firing pattern of neural clusters.

[0041] Linear transformation: The firing rate matrix of the input neuron. A linear transformation is performed to map the original firing rate features to a high-dimensional latent space through a learnable weight matrix, resulting in an initial spatial feature vector. This achieves preliminary dimensionality enhancement and abstraction of the features, providing standardized feature inputs for subsequent attention mechanism computation.

[0042] Attention Calculation: Based on the topological constraints of the spatial adjacency matrix, the attention weights between each neuron and its spatial neighbors are calculated. Through the attention mechanism, the model can adaptively learn the spatial influence weights of different neurons on the target neuron, automatically strengthen neuron features strongly correlated with the decoding intent, and weaken the influence of ineffective features caused by random neuron noise and electrode micro-drifts, thus achieving adaptive filtering of noise interference.

[0043] Feature aggregation: The attention weights are weighted and aggregated with the initial spatial features after linear transformation to obtain the neuron feature output in the spatial dimension. ( (This refers to the latent space feature dimension). This output integrates the spatial coupling information within the neural cluster, enabling the extraction and optimization of effective spatial dimension features, and solving the problem that traditional artificial features cannot adapt to dynamic neural activity.

[0044] 4. Temporal GAT feature extraction layer (adaptive feature extraction based on temporal dimension).

[0045] This layer runs in parallel with the spatial GAT layer. Based on the graph attention network, it extracts temporal features from the input data of branch 2. Through the same linear transformation-attention calculation-feature aggregation process, it captures the temporal dynamic evolution of neuronal activity.

[0046] Linear transformation: For the input neuron time series matrix A linear transformation is performed to map the original temporal features to a high-dimensional latent space, resulting in an initial temporal feature vector. This completes the initial abstraction and dimensional alignment of the temporal features, providing a foundation for subsequent attention calculations.

[0047] Attention Calculation: Based on the topological constraints of the temporal adjacency matrix, the attention weight between each time series segment and its preceding and following time series segments is calculated. The model can adaptively learn the temporal dependencies of neuronal activity, capture the dynamic evolution of firing patterns, weaken the interference of firing pattern shifts caused by changes in brain state, and improve the robustness of temporal features.

[0048] Feature aggregation: The attention weights are weighted and aggregated with the initial temporal features after linear transformation to obtain the neuron feature output in the temporal dimension. ( (This represents the latent space feature dimension, aligned with the spatial feature dimension). This output integrates the temporal correlation information of neuronal activity, realizing the extraction and optimization of effective features in the temporal dimension, effectively solving the shortcomings of traditional decoding methods that cannot adapt to the dynamic changes in neural activity.

[0049] 5. Feature fusion layer (spatiotemporal joint feature construction).

[0050] This layer uses a splicing or weighted fusion method to combine the spatial features output by the spatial GAT layer. Temporal characteristics of the output of the temporal GAT layer The process involves fusion to construct a spatiotemporal joint feature of neurons that combines spatial correlation and temporal dynamics. The fusion process includes splicing fusion and weighted fusion, as detailed below: splicing and fusion will and By directly concatenating the features along their dimensions, we obtain a dimension of... The combined features retain all information of both spatial and temporal features; Weighted fusion uses learnable weight parameters to... and By performing weighted summation, the contribution ratio of spatial features and temporal features in decoding is adaptively adjusted to achieve complementary enhancement of the two features.

[0051] The fused spatiotemporal joint features comprehensively cover the spatiotemporal coupling information of neural activity, providing a complete and robust feature foundation for subsequent decoding.

[0052] 6. Flatten layer (feature dimension normalization).

[0053] This layer will use the two-dimensional spatiotemporal joint feature matrix output by the feature fusion layer (such as...) or The feature vector (which, depending on the fusion method) is flattened and converted into a one-dimensional feature vector. This step eliminates the two-dimensional structure of the features, facilitating processing by subsequent fully connected layers, while fully preserving all information of the spatiotemporal joint features, providing standardized input for high-level feature mapping of the decoding model.

[0054] 7. Fully connected layer + activation function (decoding intent feature mapping).

[0055] This layer uses multi-layer fully connected transformations and non-linear activation functions to perform high-level abstraction and mapping of decoding intent on the flattened one-dimensional feature vector. The fully connected layer uses learnable weight matrices and bias terms to map the low-dimensional flattened features to a high-dimensional latent space that matches the decoding task, achieving further abstraction of features and mapping of decoding intent.

[0056] Specifically, activation functions (such as ReLU, Sigmoid, etc.) can be introduced to give the model nonlinear fitting capabilities, enhance the model's ability to model the complex mapping relationship between neural signals and brain intentions, improve the expressive power and generalization of the decoding model, and solve the problem that traditional linear models cannot fit complex neural activity patterns.

[0057] 8. Output layer (generating decoding results).

[0058] This layer, based on the high-level latent features output by the fully connected layer, selects the corresponding output layer (such as a Softmax layer or a linear layer) according to the decoding task type (such as motion control, perceptual intent classification, etc.) to generate the final decoding result, which is the brain intent feature information corresponding to the neuronal signal. This step completes the end-to-end mapping from the original neural signal to the decoding result, outputting a decoding result with strong anti-interference ability, high accuracy, and good stability, effectively overcoming decoding errors caused by various noises and signal distortions.

[0059] Optionally, step S102, which calculates the firing rate of the target neuron using the number of action potentials of neuron signals within a preset time window and constructs the spatial adjacency matrix of the target neuron based on the spatial Euclidean distance between the target neuron and other neurons, includes the following steps: Step S201: Obtain the peak potential of the neuron signal in each acquisition channel within the preset time window, and use the peak potential to obtain the number of action potentials corresponding to the target neuron.

[0060] Peak potential detection is performed on the raw neuron signals acquired from each acquisition channel within a preset time window. Effective peak potentials exceeding a preset amplitude threshold are identified and filtered out, and invalid peak interference such as noise and artifacts are removed. Based on the effective peak potentials, the total number of action potentials under the acquisition channel corresponding to the target neuron within the time window is calculated, and the precise quantitative counting of the action potentials of the target neuron is completed.

[0061] Step S202: Obtain the firing rate of the target neuron based on the ratio of the number of action potentials to the length of the time window.

[0062] The number of action potentials of the target neuron obtained in step S201 is compared with the length of the preset time window to obtain the firing rate of the target neuron in the current time window. This firing rate serves as the core quantitative indicator of neuronal firing activity, used to characterize the real-time activation intensity of the target neuron and provide standardized data for assigning features to spatial graph nodes.

[0063] Step S203: Determine the hyperparameters of the target neuron by using the channel difference between the first acquisition channel of the target neuron and the second acquisition channels of other neurons, and calculate the spatial Euclidean distance between the target neuron and other neurons.

[0064] Using the first acquisition channel corresponding to the target neuron as a benchmark, the second acquisition channels corresponding to other neurons in the neural cluster are obtained. The channel number difference between the first acquisition channel and the second acquisition channel is calculated, and the hyperparameters used to construct the spatial adjacency matrix are adaptively determined based on the channel difference. At the same time, according to the spatial coordinates of each acquisition channel in the implanted brain region, the spatial Euclidean distance between the target neuron and other neurons in the cluster is calculated, and the physical spatial proximity relationship between neurons is accurately quantified.

[0065] Step S204: Construct the spatial adjacency matrix of the target neuron using spatial Euclidean distance and hyperparameters.

[0066] Based on the spatial Euclidean distance and adaptive hyperparameters obtained in step S203, a spatial topological adjacency matrix corresponding to the target neuron signal is constructed according to a preset mapping rule. In this matrix, the correlation element values ​​between the target neuron corresponding to the first acquisition channel and other neurons corresponding to the second acquisition channel are determined by their spatial Euclidean distance and hyperparameters. This matrix comprehensively depicts the spatial topological connectivity relationships of all neurons within the neural cluster, providing spatial topological constraints for the spatial graph attention network. Specifically, the spatial adjacency matrix... ; For spatial Euclidean distance. For hyperparameters, The target neuron in the spatial topological adjacency matrix First acquisition channel The target neuron and the second acquisition channel Other neurons below.

[0067] Optionally, step S103, which involves truncating the neuron signal to obtain a time-series segment of the target neuron using a preset truncation length and constructing a temporal adjacency matrix of the target neuron based on the temporal dependencies between the time-series segments, includes the following steps: Step S301: After sequentially truncating the neuron signals of each acquisition channel using a preset truncation length, a continuous time series segment of the target neuron is obtained, and a temporal feature matrix of the neuron signal is constructed based on the continuous time series segment and the number of acquisition channels.

[0068] By using a preset time-series truncation length, the original electrical signals of neurons from all acquisition channels of the multi-channel electrode array are truncated along the time axis in an equal-length, continuous, and orderly manner, eliminating signal truncation errors and time-series offset interference, and obtaining multiple sets of time-series segments of uniform length and continuous time sequence corresponding to the target neuron.

[0069] Using the aforementioned time series segments as basic units, and combining the total number of acquisition channels, the length of the time series segments, and the dimension of the time series sampling points, a dimensionally regular and time-aligned neuronal signal time series feature matrix is ​​constructed to achieve standardized integration of multi-channel time series signals, providing a unified data carrier for subsequent time series dependency analysis.

[0070] Step S302: Determine the temporal dependencies between time series segments based on the temporal feature matrix.

[0071] Based on the constructed temporal feature matrix, the temporal sequence, sequential relationship, and temporal correlation strength of each time series segment are analyzed along the time axis. The temporal dependency rules between time series segments at different times are clarified, the temporal connection logic and topological constraints of the time series graph nodes are defined, and the dynamic evolution law and sequential response dependency characteristics of neuronal electrical activity in the time dimension are accurately characterized.

[0072] Step S303: Construct the temporal adjacency matrix of the target neuron based on the temporal dependency relationship through continuous time series segments.

[0073] Using each continuous time series segment as a node in the time series graph, and strictly following the temporal dependency relationship determined in step S302, the temporal adjacency matrix corresponding to the target neuron is constructed according to the temporal topology connection rules.

[0074] This temporal adjacency matrix fully represents the sequential dependence and dynamic evolution topology of neuronal signals in the time dimension, providing core topological constraints for the subsequent temporal feature extraction and temporal attention weight calculation of temporal graph attention networks, and ensuring the accuracy and robustness of temporal dimension feature extraction.

[0075] In a spatiotemporal dual-graph attention network, the core implementation process involves parallel computation of the spatial and temporal attention weights of the target neuron based on the spatial adjacency matrix and the temporal adjacency matrix, respectively. This process strictly adheres to the computational logic of graph attention networks (GAT), with independent parameters for the spatial and temporal branches and synchronous computation processes. The final output is an adaptive attention weight that matches the spatiotemporal characteristics of the neuron. Optionally, the calculation of the attention weights of the target neuron based on the spatial and temporal adjacency matrices includes the following steps: Step S401: Obtain the graph attention network corresponding to the spatial adjacency matrix and temporal adjacency matrix, and obtain the firing rate and node features of the time series segments in the graph attention network.

[0076] The existing spatial graph attention network and temporal graph attention network are invoked respectively, and the spatial adjacency matrix is ​​input into the spatial graph attention network and the temporal adjacency matrix is ​​input into the temporal graph attention network to provide topological constraints.

[0077] Simultaneously, neuron firing rate is extracted as a node feature of the spatial graph attention network. Extracting time-series segments of neurons as node features for a time-series graph attention network. This completes the matching and binding of bi-branch node features with graph structures, providing basic data for attention weight calculation.

[0078] Step S402: Obtain the linear transformation weights and attention vectors corresponding to the node features based on the graph attention network.

[0079] Configure independent learnable parameters, including linear transformation weights, for the spatial graph attention network and the temporal graph attention network. (Spatial branching is) Timing branches are ) and attention vector (Spatial branching is) Timing branches are ).

[0080] The above parameters are trainable parameters of the network. They are optimized iteratively through backpropagation during model training to achieve high-dimensional mapping of node features and quantitative calculation of the correlation strength between neurons.

[0081] Step S403: Calculate the linear transformation output of adjacent nodes in the graph attention network by multiplying the linear transformation weights with the node features.

[0082] Node features of each branch With corresponding linear transformation weights Perform matrix multiplication to obtain the linear transformation output of adjacent nodes in the graph attention network. ,Right now This step completes the dimensional mapping and feature abstraction of node features, transforming the original firing rate features and temporal segment features into high-dimensional latent features adapted for attention computation, while preserving the core information of spatiotemporal association between neurons.

[0083] Step S404: Calculate the attention coefficients of neighboring nodes using the linear transformation output and the attention vector.

[0084] Output the linear transformation result The input is fed into the attention computation unit and combined with the attention vector. The attention coefficients between adjacent nodes in the spatial and temporal branches are calculated. The LeakyReLU activation function is introduced during the calculation to perform nonlinear processing on the attention coefficients, highlighting effective feature associations, suppressing noise interference, and accurately quantifying the strength of functional associations and temporal dependencies between adjacent neurons / adjacent temporal segments. Specifically... ;in, The index of the target node. This is an index of neighboring nodes connected to the target node.

[0085] Step S405: Calculate the attention weight of the target neuron in the graph attention network based on the attention coefficient.

[0086] Attention coefficients obtained in step S404 The Softmax function is used for normalization to eliminate the numerical differences in attention coefficients between different nodes, thereby obtaining the spatial attention weights of the target neuron in the spatial graph attention network and the temporal attention weights in the temporal graph attention network.

[0087] This attention weight can adaptively strengthen spatiotemporal features that are strongly correlated with brain intentions, while weakening interference factors such as neuronal noise, electrode drift, and firing pattern shift, providing precise weighting constraints for subsequent spatiotemporal joint feature extraction.

[0088] Optionally, the spatiotemporal joint feature map of the target neuron under attention weights is calculated using the firing rate and time series segments, including the following steps: Step S501: Obtain the first attention weight and the second attention weight of the firing rate and the time series segment in the graph attention network, respectively.

[0089] Two types of attention weights corresponding to the target neurons are extracted from the spatial graph attention network branch and the temporal graph attention network branch, respectively, providing a core basis for subsequent feature weighting and aggregation.

[0090] First attention weight (spatial attention weight): This is the spatial attention weight of the neuron obtained under the spatial GAT branch, based on the firing rate node features of the target neuron, the topological constraints of the spatial adjacency matrix, and after calculation of the attention coefficients and Softmax normalization. It is denoted as... This weight quantifies the spatial correlation strength between other neurons j and target neuron i within the neural cluster, reflecting the degree of functional coupling between neurons. It has been normalized to ensure that the sum of the weights of all neighboring neurons corresponding to the same target neuron is 1.

[0091] The second attention weight (temporal attention weight): This is the temporal attention weight obtained under the temporal GAT branch, based on the node features of the neuron's time-series segment and the topological constraints of the temporal adjacency matrix, after attention coefficient calculation and Softmax normalization. It is denoted as... This weight quantifies the temporal dependence of different time windows j on the target time window i, reflecting the dynamic evolution of neuronal activity. It is also normalized to ensure that the sum of the weights of all adjacent time segments corresponding to the same target time window is 1.

[0092] Both types of attention weights are parameters that can be adaptively optimized during network training, automatically strengthening features that are strongly related to the decoding intent and weakening the influence of noise and distortion interference features.

[0093] Step S502: Calculate the spatial feature aggregation result of the target neuron corresponding to the first attention weight based on the output result of the first linear transformation under the spatial adjacency matrix.

[0094] Based on the output of the first linear transformation corresponding to the spatial adjacency matrix, and combined with the first attention weight, the spatial dimensional features of the target neuron are aggregated.

[0095] The output of the first linear transformation is the firing rate node feature of the target neuron, followed by the linear transformation weights of the spatial GAT branch. The high-dimensional latent features obtained after mapping are denoted as ,in This is a neuron firing rate matrix with dimension 1. ( For the number of neurons, (For the time sampling dimension of the issuance rate), the transformed Dimensions ( (This is the latent space feature dimension), which has completed preliminary dimension abstraction and noise filtering.

[0096] For each target neuron i, traverse all neighboring neurons j in the spatial adjacency matrix that are connected to neuron i, and transform the linear transformation features of the neighboring neurons. With the corresponding first attention weight After multiplying and summing, the spatial feature aggregation result of the target neuron is obtained. The calculation logic is as follows: ,in Let be the set of spatially adjacent neurons of the target neuron i.

[0097] The spatial feature aggregation result obtained after aggregation has the following dimensions: This result integrates the co-firing information of the target neuron and all spatially adjacent neurons, adaptively enhances the neuronal features that contribute highly to the decoding intent, suppresses the influence of invalid features caused by electrode drift and random noise, and fully characterizes the spatial coupling activity pattern of the neural cluster.

[0098] Step S503: Calculate the temporal feature aggregation result of the target neuron corresponding to the second attention weight based on the output result of the second linear transformation under the temporal adjacency matrix.

[0099] Based on the output of the second linear transformation corresponding to the temporal adjacency matrix, and combined with the second attention weight, the temporal dimension features of the target neuron are aggregated.

[0100] The output of the second linear transformation is the node features of the neuron's time-series segment, after linear transformation weights via the temporal GAT branch. The high-dimensional latent features obtained after mapping are denoted as ,in This is a time series matrix of neurons, with dimensions [missing information]. ( (length of time segment), after transformation Dimensions (Hidden Space Dimension) (Aligned with spatial branches), this feature has completed the dimensional abstraction and artifact filtering of the temporal signal.

[0101] For each target time window i, iterate through all adjacent time segments j in the temporal adjacency matrix that are connected to time window i, and transform the linear characteristics of the adjacent time segments. With the corresponding second attention weight After multiplying and summing, the aggregated temporal features of the target time window are obtained. The calculation logic is as follows: in Let i be the set of temporally adjacent time segments within the target time window i. To achieve subsequent fusion with spatial features, the temporal features need to be regularized along the neuron dimension to match the spatial features. This ensures dimensional alignment of spatiotemporal features.

[0102] The dimensions of the time-series feature aggregation result obtained after aggregation are: This result integrates the dynamic firing information of target neurons in different time windows, adaptively strengthens the temporal evolution pattern that is strongly correlated with the decoding intention, weakens the temporal feature distortion caused by changes in brain state and shifts in firing pattern, and fully characterizes the dynamic temporal pattern of neuronal activity.

[0103] Step S504: After convolutionally fusing and splicing the spatial feature aggregation result and the temporal feature aggregation result in the channel dimension of the graph attention network, the spatiotemporal joint feature map of the target neuron is obtained.

[0104] The spatial feature aggregation result obtained in step S502 and the temporal feature aggregation result obtained in step S503 are fused together in the channel dimension of the graph attention network to generate a spatiotemporal joint feature map of the target neuron, as follows.

[0105] Feature dimension alignment: pre-aggregating spatial features Aggregation results with time series features The dimensions are standardized to ensure that the number of neurons N and the latent feature dimension d are perfectly matched, thus eliminating the fusion error caused by the dimensional difference.

[0106] Channel dimension fusion processing: Feature fusion can be achieved using the following two optional methods: Direct splicing and fusion: Spatial features and temporal features are directly spliced ​​along the channel dimension to obtain a dimension of The joint feature matrix fully preserves all information of spatial and temporal features; Convolutional fusion / weighted fusion: Using learnable 1×1 convolutional layers or weight matrices, channel interaction and dimensionality reduction are applied to the concatenated features, or a weighted sum of two features with learnable weights is directly applied to achieve adaptive complementary enhancement of spatiotemporal features, resulting in a feature of dimension 1. Optimized joint features.

[0107] Spatiotemporal joint feature map output: The feature matrix obtained after fusion processing is the spatiotemporal joint feature map of the target neuron. This feature map simultaneously carries the spatial collaborative firing information and temporal dynamic evolution information of the neural cluster, realizing the deep integration of effective features of the spatiotemporal two dimensions of neuronal activity. It can resist noise and drift interference in the spatial dimension and pattern shift interference in the temporal dimension at the same time, providing high-quality and robust feature input for the subsequent decoding layer.

[0108] Optionally, the step of obtaining the decoding result of the neuron signal based on the spatiotemporal joint feature map includes the following steps: Step S601: Obtain the multidimensional feature matrix of the spatiotemporal joint feature map based on the spatial adjacency matrix and the temporal adjacency matrix.

[0109] Based on the pre-constructed spatial adjacency matrix and temporal adjacency matrix, the spatiotemporal joint feature map generated by weighted aggregation and channel fusion of spatial graph attention branches and temporal graph attention branches is retrieved and formatted into a multi-dimensional feature matrix with standardized structure and uniform dimensions.

[0110] This multidimensional feature matrix fully carries the spatial coupling and correlation features, temporal dynamic evolution features, and core effective features after attention weighting of the neuron population. It retains the suppression characteristics of noise, electrode drift, and firing pattern shift, and is the core feature carrier for subsequent decoding calculations.

[0111] Step S602: Flatten the multidimensional feature matrix by adjusting the number of dimensions of the spatiotemporal joint feature map to obtain a one-dimensional feature vector of the spatiotemporal joint feature map.

[0112] Based on the inherent dimensional parameters of the spatiotemporal joint feature map (number of neurons, number of feature channels, and latent space feature dimension), a flattening transformation operation is performed on the multidimensional feature matrix to stretch the two-dimensional / high-dimensional structured features in an orderly manner along the feature channels and neuron dimensions, transforming them into continuous and uninterrupted one-dimensional feature vectors.

[0113] The flattening process only adjusts the arrangement of features without losing any spatiotemporal joint feature information, making high-dimensional spatiotemporal features adapt to the input format requirements of fully connected layers, and providing standardized feature input for the associative mapping of brain intentions.

[0114] Step S603: Calculate the correlation between node features in the one-dimensional feature vector using the motion intention category parameter and the visual stimulus category parameter.

[0115] The flattened one-dimensional feature vector is input into the decoding fully connected layer, and preset motion intention category parameters and visual stimulus category parameters (which are classification weight parameters learned after model training and corresponding to different brain intentions) are loaded. Through the linear weighting operation of the fully connected layer, the association matching results between the features of each node in the one-dimensional feature vector and different brain intention categories are calculated.

[0116] The correlation results quantify the degree of matching between spatiotemporal joint features and various motion intentions and visual stimulus intentions, completing the initial mapping from high-dimensional spatiotemporal features to the intention category space.

[0117] Step S604: Calculate the decoding result of the neuron signal corresponding to the correlation calculation result through the linear activation function associated with the spatiotemporal joint feature map.

[0118] The linear activation function adapted to the spatiotemporal joint feature map is invoked. The feature association calculation result obtained in step S603 is input into the linear activation function for mapping calculation, eliminating the numerical offset of the feature mapping, and outputting the standardized decoding prediction value.

[0119] This predicted value is the final decoding result corresponding to the neuronal signal, which directly represents the target brain's intention, including the category of motor intention and the category of visual stimulus recognition. It can be directly converted into brain-computer interface control commands for real-time control of intelligent prostheses, rehabilitation equipment, and interactive terminals, achieving high-precision and high-robust neural signal decoding.

[0120] The flowchart of the spatiotemporal joint decoding method of neuronal signals for brain-computer interfaces using a graph attention network model is as follows: Figure 3 As shown, the entire processing flow from raw neuronal signal acquisition to brain intent decoding output is fully presented. The core relies on the dual-branch graph attention network (GAT) to achieve adaptive extraction and fusion of spatiotemporal features, and finally outputs high-precision decoding results. The role and key operations of each link are as follows: 1. Acquisition end: Acquisition of raw neural signals.

[0121] Core operation: Acquire electrical signals of neural clusters in the implanted target brain region through a microelectrode array and amplifier, and amplify the weak neuronal spike potential signals.

[0122] Function: To acquire multi-channel, high-fidelity raw neuronal signals, providing a basic data source for subsequent processing.

[0123] 2. Data processing: signal preprocessing and topology construction.

[0124] Core operations: Denoise the acquired signals and perform three key processing steps: calculate the neuron firing rate to quantify the real-time firing activity of a single neuron; extract standardized length time-series segments of neurons; and construct spatial adjacency matrices (based on the Euclidean distance between neurons) and temporal adjacency matrices (based on the sequential dependency of time segments).

[0125] Function: To eliminate signal noise interference and provide node characteristics and topology constraints for subsequent dual GAT networks.

[0126] 3. Feature extraction: Spatiotemporal feature extraction using dual-branch GAT.

[0127] Core operation: A parallel dual-branch graph attention network is used to extract two types of features respectively. Spatial GAT, based on firing rate and spatial adjacency matrix, extracts spatial cooperative association features between neurons and outputs spatial features. ; Temporal GAT, based on time series segments and temporal adjacency matrices, extracts the temporal dynamic evolution features of neuronal activity and outputs temporal features. .

[0128] Function: Adaptively learns the spatial coupling and temporal dependencies between neurons, automatically strengthens effective features, and weakens noise and pattern shift interference.

[0129] 4. Feature fusion: Joint construction of spatiotemporal features.

[0130] Core operation: Spatial features are combined through splicing or weighted fusion methods. With time series characteristics The system integrates and generates joint features that simultaneously contain spatial correlation information and temporal dynamic information.

[0131] Function: To achieve deep integration of effective features of neuronal activity in both spatiotemporal dimensions, forming a more robust feature representation.

[0132] 5. Decoding computation: Feature regularization and intent mapping.

[0133] The core operation is to first convert the high-dimensional joint features into a one-dimensional feature vector through a flattening operation, and then map the one-dimensional features to the feature space corresponding to the brain's intention through a fully connected layer.

[0134] Function: To complete the standardization of feature formats and achieve the initial mapping from high-dimensional spatiotemporal features to intent features.

[0135] 6. Model Training: Parameter Optimization and Model Convergence.

[0136] Core operation: The MSE (mean squared error) loss function is used to target the error between the decoding result and the real brain intention, and the model parameters are iteratively optimized through backpropagation.

[0137] Function: To enable the model to learn a stable mapping relationship between neuronal signals and brain intentions, thereby improving decoding accuracy and generalization ability.

[0138] 7. Output: Decoding result generated.

[0139] Core operation: Output the final brain-computer interface decoding result. This refers to the brain's intention (such as motor commands, visual stimulus categories, etc.) corresponding to neuronal signals.

[0140] Function: Generates decoded outputs that can be directly used for brain-computer interface control, realizing the conversion of neural signals into control commands for external devices.

[0141] Based on the above, the flowchart of the spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces, including model training, is as follows: Figure 4 As shown, this paper presents the complete implementation process of a spatiotemporal joint decoding method for neuronal signals in brain-computer interfaces (BCIs) based on spatiotemporal bibranch graph attention networks (GAT). It covers the entire closed loop from signal processing, model building, offline training to online deployment of decoding, aiming to solve the problems of low accuracy and poor robustness caused by noise, electrode drift, and dynamic shifts in neural activity in traditional decoding methods. The core roles and key operations of each step are explained below: 1. Data acquisition and preprocessing.

[0142] This is the foundational preparatory step in the decoding process, the core of which is to provide standardized, low-interference feature data for subsequent model processing. It involves acquiring multimodal neuronal electrical signals and performing signal denoising (filtering out power frequency interference, baseline drift, and other artifacts); calculating neuron firing rates to quantify firing activity; extracting standardized length time segments; and constructing a spatial adjacency matrix (based on the Euclidean distance between neurons) and a temporal adjacency matrix (based on the sequential dependencies of time segments).

[0143] Specifically, data acquisition mainly involves the acquisition of neuronal firing rates and time series data. The acquisition process uses a multi-channel microelectrode array to acquire neural signals, with a total of [number missing] microelectrodes deployed. One acquisition channel, corresponding to Individual neurons or neural clusters are used as graph nodes, with a sampling frequency of Continuous data collection duration is The original neural electrical signals were used to obtain the original spatiotemporal data matrix. Based on this, two core modules of the dataset are constructed, corresponding to the spatial GAT and temporal GAT inputs, respectively.

[0144] On the one hand, with a fixed time window By performing a sliding calculation on the original signal, the firing rate of the neuron in the i-th channel at time t is obtained. ,in To determine the number of action potentials generated within a window, the firing rate sequences of all channels are combined into a spatial feature matrix. And based on the Euclidean distance between the passageways Constructing a spatial topological adjacency matrix The calculation formula is: The release rate feature, together with the spatial adjacency matrix, constitutes the spatial branch of the dataset, serving as the input to the spatial GAT. On the other hand, directly extracting a length of [length] from the original signal Continuous time series segments form a time series feature matrix. And construct a temporal graph adjacency matrix based on temporal dependencies. It is used to describe the temporal correlation structure of neuronal responses. This temporal signal and the temporal graph structure constitute the temporal branch of the dataset, which serves as the input of the temporal GAT. Finally, it completes the construction of a complete spatiotemporal dataset containing spatial topology, firing rate features, temporal signals, and temporal correlations.

[0145] During data preprocessing, the constructed original spatiotemporal dataset undergoes spatial and temporal preprocessing respectively to adapt to the input of the dual-branch GAT model. For the distribution rate features corresponding to the spatial GAT... Z-score standardization is used for normalization, and the formula is as follows: ,in and The mean and standard deviation of the firing rates for all channels are given, respectively, to eliminate the impact of differences in amplitude between different channels on the calculation of attention weights; the spatial adjacency matrix is ​​used. Add self-loops and perform symmetric regularization To improve the stability of spatial feature propagation; among them, It is a degree matrix; For the identity matrix, through Connect each node to itself, and through Eliminating the impact of node degree differences yields a standardized adjacency matrix. , which serves as the topological constraint input for spatial GAT.

[0146] For the original timing signal corresponding to the timing GAT First, bandpass filtering is used to remove power frequency noise and baseline drift. Then, time length alignment and amplitude normalization are performed. At the same time, the time adjacency matrix is ​​adjusted. Synchronous regularization is performed to remove abnormal channels and invalid time series segments, resulting in a preprocessed dataset with uniform scale, controllable noise, and standard structure, ensuring the consistency of the input data quality between the spatial GAT and the temporal GAT.

[0147] 2. Model building.

[0148] The core technology of the entire method is the decoding model built with a dual-branch GAT architecture. Specifically, the decoding model contains two graph attention networks. One network uses the target neuron as a graph node, with firing rate as a node feature, and constructs a spatial graph attention network based on the spatial association weights between neurons in the spatial adjacency matrix. The other network uses the time window as a graph node, with the collective firing rate of neuron signals as a node feature, and constructs a temporal graph attention network based on the temporal evolution data of neuron signals in the temporal adjacency matrix.

[0149] The layer receives two types of features (issue rate features) Time series characteristics The feature extraction process involves a dual-branch GAT layer (spatial GAT extracts spatial collaborative features of neurons, and temporal GAT extracts temporal dynamic features of neural activity); then, it passes through a feature fusion layer (splicing / weighting fusion to generate spatiotemporal joint features), a flattening layer (converting high-dimensional features into one-dimensional vectors), a fully connected layer (mapping features to the intent space), and an output layer.

[0150] The GAT feature extraction layer models the correlation between neuron firing rate features and time-series features. Both modules share the same graph attention calculation logic, but their parameters are independent, ensuring decoupling of spatial correlation and temporal dynamic feature extraction while avoiding parameter redundancy. Both GAT layers employ the same calculation process: first, the input node features are dimensionally mapped using a linear transformation; then, attention coefficients between neurons are calculated based on the adjacency matrix; attention weights are obtained through LeakyReLU activation and softmax normalization; finally, the neighbor node features are weighted and aggregated to obtain graph embedding features.

[0151] Let the features of the input node be The adjacency matrix is The linear transformation weights are The attention vector is The output of the linear transformation is Attention coefficient is Normalized attention weights are ; Node aggregation output is Among them, the spatial GAT input represents the neuron firing rate feature. Spatial adjacency matrix Temporal GAT input is the temporal features of neurons. With temporal adjacency matrix After dual-branch GAT feature extraction, the spatial branch and temporal branch output feature dimensions remain consistent, both being... .

[0152] The features output from the spatial GAT and temporal GAT are concatenated along the channel dimension to obtain a fused spatiotemporal joint feature map. A Flatten layer flattens the two-dimensional map feature matrix into a one-dimensional feature vector, realizing the transformation from high-dimensional features to global feature representation. Subsequently, multiple fully connected layers are connected to perform nonlinear combination and high-order feature mapping on the spatiotemporal joint features, further exploring the correlations between features.

[0153] By decoupling the spatial and temporal characteristics of neurons through a dual-branch parallel structure, adaptive enhancement of effective features and suppression of interfering features are achieved, thus constructing an end-to-end mapping path from neural signals to brain intentions.

[0154] 3. Model training.

[0155] By optimizing model parameters through offline training, the model learns a stable mapping relationship between neural signals and brain intentions.

[0156] The core operation involves dividing the preprocessed dataset into training, validation, and test sets; using the MSE (mean squared error) loss function, the model parameters are iteratively optimized through backpropagation until the model converges.

[0157] Function: Enables the model to automatically learn the spatial correlation weights and temporal dependency weights between neurons during training, strengthens the response to intent-related features, weakens the impact of noise and signal distortion, and improves decoding accuracy and generalization ability.

[0158] The dataset partitioning process primarily uses complete experimental trials as the basic unit to divide the preprocessed spatiotemporal dataset, ensuring that each trial simultaneously retains... The dataset contains the complete spatial topology and continuous temporal dynamics of each channel, avoiding disruption of the graph structure and temporal dependencies required for dual GATs due to splitting. Following standard model validation strategies, the dataset is divided into training, validation, and test sets in a 7:2:1 ratio. During the partitioning process, the three subsets maintain approximately consistent distributions in channel distribution, task labels, firing rate statistics, and temporal response patterns to avoid data skew leading to model evaluation distortion. Each partitioned subset independently contains the firing rate features and spatial adjacency matrix required for spatial GATs, and the temporal signals and temporal adjacency matrix required for temporal GATs. The training set is used to jointly learn the feature extraction weights and spatiotemporal fusion parameters of the two GAT branches. The validation set is used to adjust model hyperparameters and monitor overfitting. The test set is used to objectively evaluate the generalization performance and decoding accuracy of the dual GAT spatiotemporal fusion decoding method on completely independent data.

[0159] The loss function used in the construction of the graph attention network is: Where N is the batch size, The decoding result of the graph attention network output, The true labels are the node features. Since the same type of motor imagery or behavioral intention in the dataset corresponds to diverse spatiotemporal response patterns of neurons, the network forces the model to learn invariant representations of neural signals during backpropagation. This weakens the influence of signal distortion caused by non-task-related factors such as noise, individual differences, and acquisition status, allowing the model to focus more on the spatial associations and temporal dynamic features of neurons driven by true intentions. In the latent space, it learns stable mapping relationships: as long as the behavioral intention category is consistent, even if there are some fluctuations in the neural signals, the model output can still stably approximate the true labels.

[0160] 4. Model deployment and decoding application.

[0161] The trained model is deployed to real-world scenarios to achieve real-time decoding output.

[0162] The core operation involves collecting neuronal signals online in real time and performing the same preprocessing procedure as the training phase. Then, standardized features are input into the trained model, and the model directly outputs the brain's intention or motor control commands through inference.

[0163] Function: To realize the transformation from offline models to online applications, and can directly drive intelligent prostheses, rehabilitation equipment, interactive terminals, etc., to achieve high-precision and high-robust real-time control of brain-computer interfaces.

[0164] As can be seen from the spatiotemporal joint decoding method of neuronal signals for brain-computer interfaces mentioned in the above embodiments, this method uses the spatial adjacency matrix and temporal adjacency matrix of the target neuron to accurately obtain the spatiotemporal joint relationship between neuronal signals and brain intentions, thereby realizing end-to-end accurate mapping from neuronal signal acquisition to brain intention decoding. It can effectively overcome various noise interferences and significantly improve the decoding accuracy of neuronal signals.

[0165] Corresponding to the spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces provided in the foregoing embodiments, this invention provides a spatiotemporal joint decoding system for neuronal signals for brain-computer interfaces, such as... Figure 5 As shown, the system includes: Neuron signal acquisition module 100: used to acquire neuronal signals of the neural cluster corresponding to the target brain-computer interface in each acquisition channel through a multi-channel electrode array; Spatial adjacency matrix construction module 200: used to calculate the firing rate of the target neuron using the number of action potentials of neuron signals within a preset time window, and to construct the spatial adjacency matrix of the target neuron based on the spatial Euclidean distance between the target neuron and other neurons; Temporal adjacency matrix construction module 300: used to truncate the neuron signal using a preset truncation length to obtain the time sequence segment of the target neuron, and to construct the temporal adjacency matrix of the target neuron according to the temporal dependency relationship between the time sequence segments; Spatiotemporal joint feature map calculation module 400: After calculating the attention weights of the target neuron based on the spatial adjacency matrix and the temporal adjacency matrix, it uses the firing rate and time series segments to calculate the spatiotemporal joint feature map of the target neuron under the attention weights; The spatiotemporal joint decoding and acquisition module 500 for neuronal signals for brain-computer interfaces is used to acquire the decoding results of neuronal signals based on spatiotemporal joint feature maps.

[0166] As can be seen from the spatiotemporal joint decoding system for neuronal signals for brain-computer interfaces mentioned in the above embodiments, the system uses the spatial adjacency matrix and temporal adjacency matrix of the target neuron to accurately obtain the spatiotemporal joint relationship between neuronal signals and brain intentions, thereby achieving end-to-end accurate mapping from neuronal signal acquisition to brain intention decoding. It can effectively overcome various noise interferences and significantly improve the decoding accuracy of neuronal signals.

[0167] The spatiotemporal joint decoding system for neuronal signals for brain-computer interfaces provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces.

[0168] This embodiment also provides an electronic device, the structural schematic diagram of which is shown below. Figure 6 As shown, the device includes a processor 101 and a memory 102; wherein, the memory 102 is used to store one or more computer instructions, which are executed by the processor to implement the above-mentioned steps of spatiotemporal joint decoding of neuronal signals for brain-computer interface.

[0169] Figure 6 The electronic device shown also includes a bus 103 and a communication interface 104, with the processor 101, communication interface 104 and memory 102 connected via the bus 103.

[0170] The memory 102 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. The bus 103 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0171] The communication interface 104 is used to connect to at least one user terminal and other network units through a network interface, and to send encapsulated IPv4 packets or IPv4 packets to the user terminal through the network interface.

[0172] Processor 101 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 101 or by instructions in software form. The processor 101 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 102. The processor 101 reads the information in memory 102 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0173] This invention also provides a storage medium storing a computer program, which, when run by a processor, executes the steps of the spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces described in the foregoing embodiments.

[0174] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, devices, and methods can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0175] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0176] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0177] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces, characterized in that, The method includes: The neuronal signals of the neural clusters corresponding to the target brain-computer interface are acquired through a multi-channel electrode array in each acquisition channel. The firing rate of the target neuron is calculated using the number of action potentials of the neuron signal within a preset time window, and a spatial adjacency matrix of the target neuron is constructed based on the spatial Euclidean distance between the target neuron and other neurons. The time series segments of the target neuron are obtained by truncating the neuron signal using a preset truncation length, and the temporal adjacency matrix of the target neuron is constructed based on the temporal dependencies between the time series segments. After calculating the attention weight of the target neuron based on the spatial adjacency matrix and the temporal adjacency matrix, the spatiotemporal joint feature map of the target neuron under the attention weight is calculated using the firing rate and the time series segment; The decoding result of the neuron signal is obtained based on the spatiotemporal joint feature map.

2. The spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces according to claim 1, characterized in that, The steps of calculating the firing rate of the target neuron using the number of action potentials of the neuron signals within a preset time window, and constructing the spatial adjacency matrix of the target neuron based on the spatial Euclidean distance between the target neuron and other neurons, include: The peak potential of the neuron signal in each acquisition channel within a preset time window is obtained, and the number of action potentials corresponding to the target neuron is obtained using the peak potential. The firing rate of the target neuron is obtained based on the ratio of the number of action potentials to the length of the time window; The hyperparameters of the target neuron are determined by using the channel difference between the first acquisition channel of the target neuron and the second acquisition channel of other neurons, and the spatial Euclidean distance between the target neuron and the other neurons is calculated. The spatial adjacency matrix of the target neuron is constructed using the spatial Euclidean distance and the hyperparameters; wherein, the spatial adjacency matrix... ; Let be the Euclidean distance in the space. For the hyperparameter, The spatial topological adjacency matrix of the target neuron The first acquisition channel The target neuron below and the second acquisition channel The other neurons mentioned below.

3. The spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces according to claim 1, characterized in that, The steps of truncating the neuron signal using a preset truncation length to obtain a time series segment of the target neuron, and constructing a temporal adjacency matrix of the target neuron based on the temporal dependencies between the time series segments, include: After sequentially truncating the neuron signals from each acquisition channel using a preset truncation length, a continuous time series segment of the target neuron is obtained, and a temporal feature matrix of the neuron signal is constructed based on the continuous time series segment and the number of acquisition channels. The temporal dependencies between the time series segments are determined based on the temporal feature matrix. Based on the time sequence dependency, the temporal adjacency matrix of the target neuron is constructed using the continuous time series segments.

4. The spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces according to claim 1, characterized in that, The attention weights of the target neuron are calculated based on the spatial adjacency matrix and the temporal adjacency matrix, including: Obtain the graph attention network corresponding to the spatial adjacency matrix and the temporal adjacency matrix, and obtain the firing rate and the node features of the time series segment in the graph attention network; Based on the graph attention network, obtain the linear transformation weights and attention vectors corresponding to the node features; The linear transformation output of adjacent nodes in the graph attention network is obtained by multiplying the linear transformation weights with the node features. The attention coefficients of the neighboring nodes are calculated using the output of the linear transformation and the attention vector. The attention weight of the target neuron in the graph attention network is calculated based on the attention coefficient.

5. The spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces according to claim 4, characterized in that, Obtaining the graph attention network corresponding to the spatial adjacency matrix and the temporal adjacency matrix includes: Using the target neuron as a graph node, the firing rate as a node feature, and a spatial graph attention network constructed based on the spatial association weights between neurons in the spatial adjacency matrix; Using the time window as a graph node, the collective firing rate of the neuron signal as a node feature, and based on the temporal evolution data of the neuron signal in the temporal adjacency matrix, a temporal graph attention network is constructed.

6. The spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces according to claim 4, characterized in that, The loss function used in the construction of the graph attention network is: ; Where N is the batch size. The decoding result output by the graph attention network. The true label for the node feature.

7. The spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces according to claim 4, characterized in that, Calculating the spatiotemporal joint feature map of the target neuron under the attention weights using the firing rate and the time series segment includes: The firing rate and the time series segment are respectively obtained in the graph attention network as the first attention weight and the second attention weight; The spatial feature aggregation result of the target neuron corresponding to the first attention weight is calculated based on the output result of the first linear transformation under the spatial adjacency matrix. The temporal feature aggregation result of the target neuron corresponding to the second attention weight is calculated based on the output result of the second linear transformation under the temporal adjacency matrix; The spatial feature aggregation result and the temporal feature aggregation result are convolved and fused together in the channel dimension of the graph attention network to obtain the spatiotemporal joint feature map of the target neuron.

8. The spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces according to claim 4, characterized in that, The step of obtaining the decoding result of the neuron signal based on the spatiotemporal joint feature map includes: The multidimensional feature matrix of the spatiotemporal joint feature map is obtained based on the spatial adjacency matrix and the temporal adjacency matrix. After flattening the multidimensional feature matrix by adjusting the number of dimensions of the spatiotemporal joint feature map, a one-dimensional feature vector of the spatiotemporal joint feature map is obtained. The correlation calculation results between the node features in the one-dimensional feature vector are calculated using the motion intention category parameter and the visual stimulus category parameter. The decoding result of the neuron signal corresponding to the correlation calculation result is calculated using the linear activation function associated with the spatiotemporal joint feature map.

9. A spatiotemporal joint decoding system for neuronal signals for brain-computer interfaces, characterized in that, The system includes: Neuron signal acquisition module: used to acquire neuronal signals of the target brain-computer interface corresponding to the neural cluster in each acquisition channel through a multi-channel electrode array; Spatial adjacency matrix construction module: used to calculate the firing rate of the target neuron using the number of action potentials of the neuron signal within a preset time window, and to construct the spatial adjacency matrix of the target neuron based on the spatial Euclidean distance between the target neuron and other neurons; Temporal adjacency matrix construction module: used to truncate the neuron signal using a preset truncation length to obtain the time sequence segment of the target neuron, and construct the temporal adjacency matrix of the target neuron according to the temporal dependency relationship between the time sequence segments; Spatiotemporal joint feature map calculation module: used to calculate the attention weight of the target neuron based on the spatial adjacency matrix and the temporal adjacency matrix, and then use the firing rate and the time series segment to calculate the spatiotemporal joint feature map of the target neuron under the attention weight; A spatiotemporal joint decoding and acquisition module for neuronal signals for brain-computer interfaces: used to acquire the decoding result of the neuronal signals based on the spatiotemporal joint feature map.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the steps of the spatiotemporal joint decoding method for neuronal signals for brain-computer interfaces according to any one of claims 1 to 8.

Images