A spike signal decoding model construction method applied to an invasive brain-computer interface

Abstract

The application belongs to the technical field of brain-computer interface spike signal decoding, and particularly relates to a spike signal decoding model construction method applied to an invasive brain-computer interface, which comprises: a pulse neural network comprising: a space-time feature extraction module, which is used for extracting spike interval time features of each channel signal in a multi-channel spike signal through one-dimensional convolution in the space-time feature extraction module, so as to obtain a pulse feature matrix; a channel fusion unit, which is used for fusing inter-channel feature data in the pulse feature matrix in different ways through n two-dimensional convolutions, so as to obtain n one-dimensional pulse feature maps; a time convolution unit, which is used for extracting features of each one-dimensional pulse feature map through two-dimensional convolution, so as to obtain n new one-dimensional pulse feature maps; and a refinement convolution unit, which is used for performing feature interaction between all new one-dimensional pulse feature maps for multiple times, so as to obtain multiple one-dimensional pulse interaction feature maps. The method can reduce the calculation complexity while ensuring the accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of brain-computer interface spike signal decoding, and more particularly relates to a spike signal decoding model construction method applied to an invasive brain-computer interface. BACKGROUND

[0002] Brain computer interfaces (BCIs) enable direct communication between the brain and external devices by bypassing the traditional neuromuscular pathway. Initially developed for patients with severe motor disabilities, BCIs have now expanded to brain decoding for able-bodied users in areas such as gaming, emotion recognition, and military applications.

[0003] Depending on the location of electrode placement, brain-computer interfaces can be divided into three types: non-invasive, semi-invasive, and invasive. Non-invasive brain-computer interfaces are based on electroencephalogram (EEG) signals and have been widely used in brain-computer interfaces due to their convenience and low acquisition cost. However, their low signal-to-noise ratio and variability limit their further application in complex scenarios. Semi-invasive brain-computer interfaces use electrocorticogram (ECoG) to record signals implanted in the skull but outside the brain tissue, making a certain trade-off between signal quality and surgical risk. Invasive brain-computer interfaces include electrodes or microelectrode arrays directly implanted into brain tissue, providing the highest temporal and spatial resolution and temporal accuracy of signals, making high-precision control of prostheses or communication devices possible. However, invasive brain-computer interfaces also face problems such as infection risk, scar tissue, and long-term stability caused by the body's immune response.

[0004] Artificial neural networks (ANNs) have achieved remarkable success in the field of BCIs. For example, Lawhern et al. proposed a compact EEGNet that combines temporal convolution with spatial deep convolution and has proven effective in various electroencephalogram-based paradigms. Although artificial neural networks have achieved remarkable results, the high computational cost limits their further clinical application. In mobile or implanted brain-computer interface applications, excessive power can reduce the lifespan of the device and may harm human function due to heat. At the same time, the low signal-to-noise ratio, non-stationarity, and significant inter-subject variability of EEG signals limit their application in complex tasks. In contrast, intracortical spike signals have high temporal accuracy and high temporal and spatial resolution. In addition, models designed for EEG signals may not be optimal for processing spike data, which emphasizes the need to develop spiking neural network (SNN)-based architectures for effective decoding.

[0005] Spiking neural network is considered as the third generation of neural network model, with rich time dynamics characteristics. Neurons in spiking neural network communicate through binary spikes, and can use multiple encoding schemes. The calculation process in spiking neural network is event-driven, and only responds to the peak value event of input, thereby ensuring energy efficiency. In recent years, it has gradually attracted attention in the field of brain-computer interface. Its event-driven calculation method and biological-inspired model characteristics provide a new way for decoding of brain signals. Although there have been studies exploring the application of spiking neural network in brain-computer interface, most of them are concentrated in the field of non-invasive BCI based on EEG signals, while the decoding research of invasive BCI based on spike signals is still relatively few. Calibrating the decoder and decoding high-dimensional neural signals face the challenge of high time and high computational burden. The calibration decoder needs the user to participate in the experiment for a long time to collect enough data to ensure that the decoder captures stable neural activity patterns, and needs to be adjusted and verified repeatedly, which affects the convenience and user experience. Decoding high-dimensional neural signals requires processing a large amount of data, which contains complex temporal and spatial dependencies, such as the time interval and spatial channel coordination characteristics of spike signals. Complex feature extraction leads to a significant increase in computational load, not only increasing the difficulty of real-time decoding, but also putting higher requirements on hardware performance and power consumption, especially in embedded systems, which becomes a deployment bottleneck. SUMMARY

[0006] In view of the above defects or improvement needs of the prior art, the present application provides a spike signal decoding model construction method applied to an invasive brain-computer interface, which aims to solve the problem of high computational burden when decoding high-dimensional neural signals.

[0007] To achieve the above-mentioned purpose, according to one aspect of the present application, a spike signal decoding model construction method applied to an invasive brain-computer interface is provided, comprising:

[0008] The pulse neural network is constructed, comprising a space-time feature extraction module, a channel fusion module with multiple layers of space-time convolution, and a classification module; the space-time feature extraction module is used to extract the spike interval time features of each channel signal in the multi-channel spike signal through one-dimensional convolution in the space-time feature extraction module, and obtain a pulse feature matrix through a spike neuron; the channel fusion module comprises a channel fusion unit, a time convolution unit, and a refinement convolution unit; the channel fusion unit is used to fuse the inter-channel feature data in the pulse feature matrix in different ways through n two-dimensional convolutions, and correspondingly obtain n one-dimensional pulse feature maps through a spike neuron; the time convolution unit is used to extract features of each one-dimensional pulse feature map through two-dimensional convolution, and obtain n new one-dimensional pulse feature maps through a spike neuron; the refinement convolution unit is used to perform feature interaction between all the new one-dimensional pulse feature maps multiple times without changing the shape of the feature map, to obtain multiple one-dimensional pulse interaction feature maps, so as to obtain decoding information through the classification module.

[0009] The pulse neural network is constructed, comprising a space-time feature extraction module, a channel fusion module with multiple layers of space-time convolution, and a classification module; the space-time feature extraction module is used to extract the spike interval time features of each channel signal in the multi-channel spike signal through one-dimensional convolution in the space-time feature extraction module, and obtain a pulse feature matrix through a spike neuron; the channel fusion module comprises a channel fusion unit, a time convolution unit, and a refinement convolution unit; the channel fusion unit is used to fuse the inter-channel feature data in the pulse feature matrix in different ways through n two-dimensional convolutions, and correspondingly obtain n one-dimensional pulse feature maps through a spike neuron; the time convolution unit is used to extract features of each one-dimensional pulse feature map through two-dimensional convolution, and obtain n new one-dimensional pulse feature maps through a spike neuron; the refinement convolution unit is used to perform feature interaction between all the new one-dimensional pulse feature maps multiple times without changing the shape of the feature map, to obtain multiple one-dimensional pulse interaction feature maps, so as to obtain decoding information through the classification module.

[0010] Further, the pulse neural network is provided with a reverse synapse connection at the refinement convolution unit.

[0011] Further, the space-time feature extraction module is also used to multiply each channel feature in the extracted pulse feature matrix by an attention weight vector based on an attention mechanism, to obtain a pulse feature matrix subjected to a channel-level attention mechanism.

[0012] Further, the pulse neural network is provided with a residual connection at the refinement convolution unit.

[0013] Further, the training sample set is obtained by data enhancement on a multi-channel spike signal sample.

[0014] The data enhancement method is a time point mask, specifically:

[0015] A binary mask matrix with the same shape as the multi-channel spike signal sample is preset, and is initialized as a full 1 matrix; a probability of mask or not is randomly generated for each element in the binary mask matrix, to mask the element, to obtain a new binary mask matrix; the new binary mask matrix is multiplied by the spike signal sample element by element, to simulate the absence of spikes, to obtain a new spike signal.

[0016] And / or, the data enhancement method is a time period mask, specifically:

[0017] A binary mask matrix with the same shape as the single-channel spike signal is preset, and initialized as a full 1 matrix; a probability of masking or not masking is generated for any segment of elements in the binary mask matrix, so as to mask the segment of elements and obtain a new binary mask matrix; and the new binary mask matrix is multiplied with each channel of the multi-channel spike signal sample element by element, so as to simulate the absence of spikes and obtain a new spike signal.

[0018] And / or, the data enhancement manner is channel masking, in particular:

[0019] A binary mask matrix with the same shape as the single-time-point multi-channel spike signal is preset, and initialized as a full 1 matrix; a probability of masking or not masking is generated for each element in the binary mask matrix, so as to mask the element and obtain a new binary mask matrix; and the new binary mask matrix is multiplied with each single-time-point multi-channel spike signal in the multi-channel spike signal sample element by element, so as to simulate the absence of spikes and obtain a new spike signal.

[0020] According to another aspect of the present application, a spike signal decoding method applied to an invasive brain-computer interface is provided, which uses a spike signal decoding model constructed by the spike signal decoding model construction method as described above to perform spike signal decoding.

[0021] According to another aspect of the present application, an invasive brain-computer interface system is provided, in which a pattern recognition module uses the spike signal decoding method as described above to perform pattern recognition.

[0022] According to another aspect of the present application, a computer readable storage medium is provided, which includes a stored computer program, wherein when the computer program is run by a processor, the storage medium controls the device where the storage medium is located to perform the steps of the method as described above and / or the steps of the method as described above.

[0023] According to another aspect of the present application, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the steps of the method as described above and / or the steps of the method as described above are implemented.

[0024] Overall, compared with the prior art, the technical scheme provided by the present application mainly has the following beneficial effects:

[0025] 1. The application provides a spike signal decoding model construction method applied to an invasive brain-computer interface, which adopts a pulse neural network with multi-layer spatio-temporal convolution. The network effectively solves the problem of high computational burden in decoding high-dimensional neural signals through multi-layer module design and the event-driven characteristics of the pulse neural network, which is embodied in the following aspects: (1) effective feature representation of the spatio-temporal feature extraction module: the spatio-temporal feature extraction module of the network extracts the spike interval time features of each channel in the multi-channel spike signal through one-dimensional convolution. This feature representation directly captures the timing information of the pulse signal, and at the same time, the spike neurons convert these features into a pulse feature matrix. (2) efficient computation design of the channel fusion module: first, the channel fusion unit uses multiple two-dimensional convolutions to extract fusion features between channels in different ways, and generates a one-dimensional pulse feature map through spike neurons. Then, the time convolution unit extracts the time features of each one-dimensional pulse feature map through two-dimensional convolution, and finally, the refinement convolution unit strengthens the information flow between one-dimensional pulse feature maps without changing the shape of the feature map, while maintaining the sparsity of the calculation, avoiding the redundant overhead in full connection calculation. This event-driven computing architecture, combined with modular design, enables the channel fusion module to efficiently process high-dimensional neural signals, significantly reducing computational burden and power consumption. (3) phased feature extraction and information interaction: the network decomposes high-dimensional signal processing into multiple stages of feature extraction and fusion, and each stage uses convolution operation and spike neurons for local calculation instead of global processing. This layer-by-layer refined feature extraction method reduces the computational complexity caused by direct mapping of high-dimensional signals and improves the decoding efficiency. (4) event-driven computation reduces power consumption and redundancy: the network's computation is only triggered when a spike event (i.e., the pulse signal value is 1) occurs in the input signal, and no computation is performed when the signal value is 0, thereby significantly reducing unnecessary computational overhead. In traditional artificial neural networks, all time steps participate in computation, regardless of whether they contain meaningful information; while in pulse neural networks, only time steps with signal value 1 are calculated. In addition, this sparse computing characteristic propagates layer by layer, as the output of the spike neuron is also a sparse signal, and the subsequent layer is only activated when receiving a pulse input. As the network depth increases, the sparsity accumulates layer by layer, further reducing the overall computational load and energy consumption, making it particularly suitable for processing high-dimensional, sparse spike signals. Therefore, the application effectively solves the problem of high computational burden in decoding high-dimensional neural signals.

[0026] 2. The application further proposes to set a reverse synapse connection at the refined convolution unit, making the response of each neuron more stable and orderly, avoiding instability caused by sudden noise or external disturbance, i.e. implementing a local synapse stabilization mechanism. This mechanism allows neurons to maintain a stable activation state for a long time, which is particularly critical for the time interval information unique to spike signals, as these time intervals carry most of the neural activity information. Specifically, by re-inputting the output of the refined convolution layer into the layer, the previous cumulative membrane potential is utilized, further improving the learning efficiency of the layer, not only increasing the depth interaction of features, but also continuously optimizing and dynamically adjusting the membrane potential of spike neurons. This design enables neurons to more efficiently accumulate and update membrane potential when processing sparse spike signals, thereby improving the response capability of neurons to spike signals and enhancing the preservation and decoding accuracy of timing information by the network. By optimizing the update of the membrane potential of the fourth layer of spike neurons, the performance and energy efficiency are significantly improved. Experiments show that this reverse connection improves model performance, reduces energy consumption, and effectively alleviates the calibration time problem required for decoding high-dimensional neural signals in brain-computer interface systems.

[0027] 3. The application further proposes a channel-level attention mechanism that dynamically adjusts the activation weight of each channel according to the signal strength or information contribution of each channel. For important channels, a higher weight is given to enhance their ability to express key information; for channels that are not related or have more noise, their weight is reduced, effectively reducing the interference of noise on the network. Through this adaptive weighting method, the network can more accurately extract task-related features and improve overall decoding performance.

[0028] 4. The application further proposes three data enhancement methods, enabling the pulse neural network to efficiently and accurately decode intracortical neural signals. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a pulse neural network architecture provided by an embodiment of the application for application in an invasive brain-computer interface;

[0030] Figure 2 is a pulse neural network structure provided by an embodiment of the application for application in an invasive brain-computer interface;

[0031] Figure 3 is a data enhancement schematic diagram provided by an embodiment of the application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and examples. It should be understood that the specific examples described herein are only intended to explain the present application and not to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0033] Embodiment one

[0034] A spike signal decoding model construction method applied to an invasive brain-computer interface, as shown in Figure 1 , comprising:

[0035] A pulse neural network is constructed, which includes a spatial-temporal feature extraction module, a channel fusion module with multiple layers of spatial-temporal convolution, and a classification module; the spatial-temporal feature extraction module is used to extract the spike interval time features of each channel signal in the multi-channel spike signal through one-dimensional convolution in it, and obtain a pulse feature matrix through a spike neuron; the channel fusion module includes a channel fusion unit, a time convolution unit and a refinement convolution unit, the channel fusion unit is used to fuse the inter-channel feature data in the pulse feature matrix in different ways through n two-dimensional convolutions, and correspondingly obtain n one-dimensional pulse feature maps through a spike neuron; the time convolution unit is used to extract features of each one-dimensional pulse feature map through two-dimensional convolution, and obtain n new one-dimensional pulse feature maps through a spike neuron; the refinement convolution unit is used to make feature interaction between all the new one-dimensional pulse feature maps multiple times under the condition of not changing the shape of the feature map, to obtain multiple one-dimensional pulse interaction feature maps, so as to obtain decoding information through the classification module.

[0036] A training sample set is constructed to train the pulse neural network, and a spike signal decoding model is obtained.

[0037] Regarding the spatial-temporal feature extraction module, in order to fully extract the spatial-temporal dynamic features of the spike data, a one-dimensional convolution layer (the convolution kernel size is, for example, 64) is used in the spatial-temporal feature extraction module in this embodiment, aiming to capture the time features (time interval) within each individual channel, and the first one-dimensional convolution in the pulse neural network promotes the comprehensive extraction of cross-channel information, ensuring that the inter-channel dependency is well represented. After that, as preferred, a channel-level attention mechanism is applied to assign appropriate weights to each channel, enhancing relevant features. Then the output is introduced into nonlinearity through a spike neuron (such as a PLIF neuron), so that the model can process discrete peak information.

[0038] Regarding the channel fusion module, the channel fusion module with multi-layer spatio-temporal convolution is the core to enhance the ability of the network to capture the cross-channel interaction in the input data. The module includes three different units: a channel fusion unit, a temporal convolution unit, and a refinement convolution unit. Multiple spatio-temporal convolution units are used to fully extract the spatio-temporal information of the spike signal.

[0039] wherein the channel fusion unit uses a preset two-dimensional convolution with a kernel size of (C, 1) to fuse the channel data, C is the number of channels of the multi-channel spike signal, and other parameters of the convolution kernel are parameters to be learned, so as to obtain a plurality of feature maps, thereby promoting the feature interaction between channels. After convolution, batch normalization and average pooling are applied to normalize the data and reduce the feature mapping size, and finally a plurality of one-dimensional pulse feature maps are obtained.

[0040] Then, the two key convolution units (the temporal convolution unit and the refinement convolution unit) are processed, and each unit aims to capture and refine different features. The temporal convolution unit uses a two-dimensional convolution with a kernel size of (1, K) to capture sequential features across the time dimension (i.e., further extract time features), allowing the network to extract relevant features along the time dimension, obtaining a plurality of new one-dimensional pulse feature maps after time feature extraction. The refinement convolution layer uses a two-dimensional convolution with a kernel size of (1, 1) to further refine the mutual mapping between feature maps, enhancing the model's ability to focus on basic features without changing the structure (i.e., shape) of the feature map. In each unit, a spike neuron (preferably, a PLIF neuron) is applied at the output end, which helps effective peak activity in the network. The combination of this sequential feature extraction and fine mapping ensures that the network captures complex patterns while maintaining computational efficiency.

[0041] Regarding the classification module, after the spike signal undergoes hierarchical feature extraction and convolution operations, the classification module uses global average pooling to aggregate the extracted features. This operation simplifies each feature map to a single value, thereby generating a fixed-dimensional feature vector aligned with the target class count, thereby simplifying the complexity of the model while preserving basic information. To alleviate overfitting, a dropout layer is introduced to encourage the model to rely on multiple paths through the network and enhance generalization. Finally, the processed features are passed through a fully connected layer for final classification.

[0042] As a preferred embodiment, the above-mentioned pulse neural network is provided with a reverse synapse connection at the refinement convolution unit.

[0043] Inspired by biological neuron connections, a local synaptic stabilization mechanism is added to the refinement convolution unit. The local synaptic stabilization mechanism optimizes the membrane potential of deep neurons (i.e., spike neurons of the refinement convolution unit) to improve the learning efficiency of neurons and the performance of the network.

[0044] Local synaptic stabilization (LSS) is a mechanism that introduces stable synaptic connections into the network, allowing each neuron's response to be updated stably and orderly without being disturbed by sudden noise or external perturbations. Its main features are: (1) Temporal and spatial stability: LSS mechanism can adjust the update rule of membrane potential, so that the neuron can maintain a stable activation state for a long time. This is crucial for the temporal interval information in spike signals, as the interval information in spike signals carries most of the precision of neural activity. (2) Noise suppression: LSS mechanism effectively suppresses the influence of noise on decoding by smoothing the response pattern of neurons, enhancing the robustness of the model. Through smooth signal transmission, LSS ensures that the network can ignore meaningless noise and focus on important spike activity.

[0045] LSS can ensure that the timing information of spike signals is not disturbed or lost during transmission, enhancing the decodability of the signal. By improving the stability within the network, the LSS mechanism enables the network to better adapt to various changes and noise in experiments, thereby improving decoding accuracy.

[0046] As a preferred implementation, the spatio-temporal feature extraction module is further configured to multiply each channel feature in the extracted pulse feature matrix by an attention weight vector based on an attention mechanism, to obtain a pulse feature matrix subjected to a channel-level attention mechanism.

[0047] Channel-wise Attention (CA) dynamically allocates weights to each channel, allowing the network to focus on neuron activity with key information while suppressing irrelevant or redundant channel signals. Compared to traditional global attention mechanisms, channel-level attention is more suitable for the sparse nature of spike signals, reducing noise interference while significantly improving network energy efficiency.

[0048] Channel-wise Attention (CA) dynamically adjusts the importance of each channel, allowing the network to focus on information-rich signal channels and suppress unimportant or noisy channels. This mechanism optimizes the network in the following ways: adaptive selection: based on the signal strength or information contribution of each channel, the activation weight of the channel is adjusted. The signal of important channels will get higher weight, enhancing the expression ability of the signal, while the weight of irrelevant channels is suppressed, reducing the influence of noise. Save calculation and energy: since the attention mechanism dynamically adjusts the weight of the channel, it effectively reduces the calculation of irrelevant channels, thus significantly reducing the amount of calculation and energy consumption.

[0049] By focusing on important channels, the CA mechanism reduces the need for irrelevant calculations, reduces the amount of calculation, and thus reduces power consumption. By dynamically weighting the signal channels, the network can selectively activate channels related to the task, thereby avoiding useless calculations and energy waste.

[0050] In a preferred embodiment, a local synaptic stabilization spiking neural network with channel-wise attention (LSS-CA-SNN) aims to fully exploit the information potential in spike signals and improve decoding efficiency and reduce energy consumption.

[0051] Combining two innovative mechanisms: the local synaptic stabilization mechanism simulates the synaptic stabilization characteristics of biological neurons, optimizes the membrane potential update of deep neurons after the convolution layer, makes it more robustly respond to the dynamic changes of spike signals, and improves the extraction ability of spike interval features. The channel-level attention mechanism dynamically allocates the weights of each channel, allowing the network to focus on neuron activity with key information while suppressing irrelevant or redundant channel signals. Compared with traditional global attention mechanisms, channel-level attention is more suitable for the sparse characteristics of spike signals, reducing noise interference while significantly improving the energy efficiency of the network. Thus, in a preferred embodiment, the pulse neural network combines the above two innovations to achieve dual improvement of accuracy and energy efficiency in the invasive spike signal decoding task, which has important theoretical significance and practical application value.

[0052] As a preferred embodiment, the above pulse neural network is provided with a residual connection at the detailed convolution unit.

[0053] To solve the problem of gradient disappearance in deep layers, a residual connection is introduced in the last layer of the pulse neural network, ensuring effective use of deep features and effective propagation of gradients.

[0054] A more specific example of a pulse neural network can be seen in Figure 2 .

[0055] As a preferred embodiment, the above training sample set is obtained by data augmentation on multi-channel spike signal samples;

[0056] Wherein, the data augmentation method is time point mask, specifically:

[0057] Pre-set a binary mask matrix with the same shape as the multi-channel spike signal sample, initialized as a full 1 matrix; a mask or not probability is randomly generated for each element in the binary mask matrix to mask the element, obtaining a new binary mask matrix; the new binary mask matrix is element-wise multiplied with the spike signal sample to simulate the absence of spikes, obtaining a new spike signal;

[0058] And / or, the data enhancement method is time period mask, specifically:

[0059] Pre-set a binary mask matrix with the same shape as the single-channel spike signal, initialized as a full 1 matrix; a mask or not probability is generated for any segment of elements in the binary mask matrix to mask the segment of elements, obtaining a new binary mask matrix; the new binary mask matrix is element-wise multiplied with each channel of the multi-channel spike signal sample to simulate the absence of spikes, obtaining a new spike signal;

[0060] And / or, the data enhancement method is channel mask, specifically:

[0061] Pre-set a binary mask matrix with the same shape as the single time point multi-channel spike signal, initialized as a full 1 matrix; a mask or not probability is generated for each element in the binary mask matrix to mask the element, obtaining a new binary mask matrix; the new binary mask matrix is element-wise multiplied with each single time point multi-channel spike signal in the multi-channel spike signal sample to simulate the absence of spikes, obtaining a new spike signal.

[0062] Consider that the training data is {X i}∈B C×T , where B represents the binary field {0, 1}, i represents the i-th training data, C represents the number of channels (corresponding to the number of neurons), and T represents the discrete time points. As shown in Figure 3 , data augmentation is performed by randomly masking from the time point dimension, time period dimension and channel dimension to simulate the absence of spikes and increase the data pattern:

[0063] Time point mask: define a binary mask matrix M t ∈B C×T , initialized as a full 1 matrix, and then mask each element by a probability p t to simulate the absence of spikes, M t can be obtained by the following formula: (corresponding to randomly generating a random probability value for each point)

[0064]

[0065] The enhanced data x i ’ can be represented as X i' = X i ⊙M t where ⊙ denotes element-wise multiplication.

[0066] Time period mask: define a binary mask matrix M s ∈B 1×T , by a probability p s Randomly mask all channels in a time period [t s , t s +l], M s can be obtained as follows:

[0067]

[0068] Enhanced data x i ' can be obtained by broadcasting mechanism x i ' = X i ⊙M s .

[0069] Channel mask: define a binary mask matrix M c ∈B C×1 , by a probability p c Randomly mask each channel, M c can be obtained as follows:

[0070]

[0071] Enhanced data X i ' can be obtained by broadcasting mechanism x i ' = X i ⊙M c .

[0072] With the above SpikeDrop (spike mask) data enhancement, the pulse neural network can efficiently and accurately decode the neural signals in the cortex.

[0073] Experimental verification shows that on the invasive spike data from two adult male rhesus monkeys, the decoding method based on the pulse neural network proposed in the present application has obvious advantages in classification performance and energy efficiency compared with the traditional artificial neural network. In addition, the experiment also verifies that the SpikeDrop data enhancement strategy proposed in the present application has good applicability to various neural network architectures, not only improves the generalization performance of the pulse neural network model, but also has certain performance improvement for the traditional artificial neural network. The experimental results of Table 1 clearly show the comprehensive superiority of the decoding method based on the pulse neural network in the present application in terms of performance and energy consumption.

[0074] Table 1: Classification accuracy and energy consumption of different methods on invasive spike data

[0075]

[0076] In Table 1, the best results are bolded, and the second highest results are underlined. The precision and energy consumption data in microjoules (uJ) are included. The content in the parentheses shows the energy consumption ratio of different networks relative to the proposed spiking neural network in the same scenario.

[0077] In summary, the present application proposes a spiking neural network method to address the decoding problem of spike signals in invasive brain-computer interfaces, which can reduce the computational complexity. In addition, this method innovatively combines local synaptic consolidation mechanism and channel-level attention mechanism, which has made significant progress in improving decoding performance, energy efficiency and model stability. Specifically, the local synaptic consolidation mechanism optimizes the membrane potential and spike activation of neurons, enhancing the ability of the spiking neural network to process complex temporal dynamics and ensuring decoding accuracy over a long period of time. At the same time, the channel-level attention mechanism focuses on extracting the most information- valuable features from spike signals and suppressing noise interference from irrelevant channels, thereby significantly reducing computational redundancy and energy consumption. This method not only solves the contradiction between high-precision decoding and low-power operation in traditional artificial neural networks, but also enhances the model's ability to capture spike signal features, providing a new direction for the efficient development of the invasive brain-computer interface field. The present application is expected to promote the wider application of brain-computer interface systems in the fields of medical rehabilitation, emotion recognition and human-computer interaction in the future.

[0078] In summary, the present application proposes a spiking neural network method to address the decoding problem of spike signals in invasive brain-computer interfaces, which can reduce the computational complexity. In addition, this method innovatively combines local synaptic consolidation mechanism and channel-level attention mechanism, which has made significant progress in improving decoding performance, energy efficiency and model stability. Specifically, the local synaptic consolidation mechanism optimizes the membrane potential and spike activation of neurons, enhancing the ability of the spiking neural network to process complex temporal dynamics and ensuring decoding accuracy over a long period of time. At the same time, the channel-level attention mechanism focuses on extracting the most information- valuable features from spike signals and suppressing noise interference from irrelevant channels, thereby significantly reducing computational redundancy and energy consumption. This method not only solves the contradiction between high-precision decoding and low-power operation in traditional artificial neural networks, but also enhances the model's ability to capture spike signal features, providing a new direction for the efficient development of the invasive brain-computer interface field. The present application is expected to promote the wider application of brain-computer interface systems in the fields of medical rehabilitation, emotion recognition and human-computer interaction in the future.

[0079] As a further illustration, the embodiment method also has the following advantages: (1) improve the robustness of the model. In brain-computer interface (BCI), the signal is affected by noise, interference and experimental environment changes, resulting in poor robustness of the model and easy misdecoding. LSS establishes stable synaptic connections between neurons and adjusts the update rule of membrane potential, making the network's response to different stimuli more stable. This mechanism enables the network to ignore some burst noise and focus on meaningful spike activity, reducing the occurrence of misdecoding. By stabilizing the activation pattern of neurons and reducing overfitting, LSS effectively improves the robustness of the network in uncertain environments.(2) Energy optimization. Traditional deep neural networks often require a large amount of computing resources when processing high-dimensional spatiotemporal signals, resulting in high power consumption, which is particularly prominent in real-time BCI applications, especially in invasive brain-computer interfaces. The channel-level attention mechanism adjusts the weight of each neuron channel, enabling the network to focus on important channel signal activity and suppress unimportant channels, thereby reducing irrelevant calculations and energy consumption. This mechanism effectively reduces the network's computational load and power consumption by adaptively activating the most important neurons, thereby achieving the goal of energy saving.(3) Strengthen the extraction of time interval information of spike signals. Compared to pulse neural networks (SNN), traditional neural networks may ignore the time interval information in spike signals, which is crucial for decoding accuracy. The PLIF neuron models the time interval of spike signals by adaptively adjusting the membrane potential of neurons to respond to subtle time changes. In addition, the spatiotemporal convolution layer extracts multi-level features of spike signals, allowing for in-depth mining of potential information carried by time intervals. Through the combination of PLIF and spatiotemporal convolution, LSS-CA-SNN can more accurately decode spike signals, improving the sensitivity to temporal features and thus improving overall decoding accuracy.(4) Improve the training stability of the model. SNN and deep neural networks are prone to problems such as gradient vanishing and gradient explosion during training, leading to unstable training, especially in deep neural networks. The local synaptic stabilization mechanism optimizes the update rule of neuron membrane potential, making the network training process smoother and reducing training instability caused by gradient fluctuations. At the same time, the PLIF neuron adjusts the membrane potential and activation function of the neuron, making the gradient change of the spike signal during propagation more stable, avoiding the problem of gradient vanishing or explosion. In this way, the model can maintain stable convergence during training.

[0080] Embodiment Two

[0081] A spike signal decoding method applied to an invasive brain-computer interface, characterized in that a spike signal decoding model constructed by the spike signal decoding model construction method described above is used for spike signal decoding.

[0082] The related technical solutions are the same as above, and will not be repeated here.

[0083] Embodiment three

[0084] An invasive brain-computer interface system, wherein the pattern recognition module uses the spike signal decoding method as described above to perform pattern recognition.

[0085] The related technical solutions are the same as above, and will not be repeated here.

[0086] Embodiment four

[0087] A computer-readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the steps of the above method.

[0088] Specifically, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0089] The related technical solutions are the same as above, and will not be repeated here.

[0090] Embodiment five

[0091] The embodiment of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps of the above-mentioned embodiment method of the present application.

[0092] The related technical solutions are the same as above, and will not be repeated here.

[0093] Those skilled in the art can easily understand that the above description is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for constructing a spike signal decoding model for invasive brain-computer interfaces, characterized in that, include: A spiking neural network is constructed, comprising a spatiotemporal feature extraction module, a channel fusion module with multi-layer spatiotemporal convolution, and a classification module. The spatiotemporal feature extraction module extracts the peak interval time features of each channel signal in a multi-channel spike signal through one-dimensional convolution, and obtains a spiking feature matrix through spike neurons. The channel fusion module includes a channel fusion unit, a temporal convolution unit, and a thinning convolution unit. The channel fusion unit fuses the inter-channel feature data in the spiking feature matrix in different ways through n two-dimensional convolutions, and obtains n one-dimensional spiking feature maps through spike neurons. The temporal convolution unit extracts features from each one-dimensional spiking feature map through two-dimensional convolution, and obtains n new one-dimensional spiking feature maps through spike neurons. The thinning convolution unit performs feature interactions multiple times between all the new one-dimensional spiking feature maps without changing the shape of the feature maps, obtaining multiple one-dimensional spiking interaction feature maps, which are then used by the classification module to obtain decoding information. The spiking neural network is trained by constructing a training sample set to obtain a spike signal decoding model; In the spiking neural network, reverse synaptic connections are provided at the refined convolutional units; The spatiotemporal feature extraction module is also used to perform a dot product between the attention weight vector and each channel feature in the extracted pulse feature matrix based on the attention mechanism, so as to obtain a pulse feature matrix after the channel-level attention mechanism. The training sample set was obtained by data augmentation of multi-channel spike signal samples; The data augmentation method is time point masking, specifically: A binary mask matrix with the same shape as the multi-channel spike signal sample is preset and initialized as an all-one matrix; a probability of whether or not to mask is randomly generated for each element in the binary mask matrix, and the element is masked to obtain a new binary mask matrix; the new binary mask matrix is ​​multiplied element-wise with the spike signal sample to simulate the absence of spikes and obtain a new spike signal. And / or, the data augmentation method is a time period mask, specifically: A binary mask matrix with the same shape as the single-channel spike signal is preset and initialized as an all-one matrix; a probability of whether or not to mask is generated for any element in the binary mask matrix, and the element is masked to obtain a new binary mask matrix; the new binary mask matrix is ​​multiplied element-wise with each channel in the multi-channel spike signal sample to simulate the absence of spikes and obtain a new spike signal. And / or, the data enhancement method is channel masking, specifically: A binary mask matrix with the same shape as the single-time-point multi-channel spike signal is preset and initialized as an all-1 matrix. A probability of whether or not to mask is generated for each element in the binary mask matrix, and the element is masked to obtain a new binary mask matrix. The new binary mask matrix is ​​then multiplied element-wise with each single-time-point multi-channel spike signal in the multi-channel spike signal sample to simulate the absence of spikes and obtain a new spike signal.

2. The method for constructing a peak signal decoding model as described in claim 1, characterized in that, The spiking neural network has residual connections at the thinned convolutional units.

3. A method for decoding spike signals applied to invasive brain-computer interfaces, characterized in that, The spike signal decoding model constructed by the spike signal decoding model construction method as described in claim 1 or 2 is used to perform spike signal decoding.

4. An invasive brain-computer interface system, characterized in that, The pattern recognition module uses the spike signal decoding method as described in claim 3 for pattern recognition.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein when the computer program is run by a processor, it controls the device on which the storage medium is located to perform the steps of the method as claimed in claim 1 or 2, or the steps of the method as claimed in claim 3.

6. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method as described in claim 1 or 2, or the steps of the method as described in claim 3.

Images