Peak signal decoding model construction method applied to intrusive brain-computer interface

Abstract

The invention belongs to the related technical field of brain-computer interface spike signal decoding, and particularly relates to a spike signal decoding model construction method applied to an intrusive brain-computer interface. The feature extraction module is used for extracting peak interval time features of each channel signal in the multichannel peak signals through one-dimensional convolution in the feature extraction module to obtain a pulse feature matrix; the channel fusion unit is used for fusing the inter-channel feature data in the pulse feature matrix in different modes through n two-dimensional convolutions to correspondingly obtain n one-dimensional pulse feature maps; the time convolution unit is used for performing feature extraction on each one-dimensional pulse feature map through two-dimensional convolution to obtain n new one-dimensional pulse feature maps; and the refined convolution unit is used for performing feature interaction among all the new one-dimensional pulse feature maps for multiple times to obtain a plurality of one-dimensional pulse interaction feature maps. According to the method, the calculation complexity can be reduced while the precision is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field related to spike signal decoding of brain-computer interface, and more specifically, relates to a method for constructing a spike signal decoding model applied to an invasive brain-computer interface. Background Art

[0002] Brain computer interfaces (BCIs) enable direct communication between the brain and external devices by bypassing traditional neuromuscular pathways. Originally developed for patients with severe motor dysfunction, BCIs have now expanded their applications to brain decoding for healthy users, such as gaming, emotion recognition, and the military.

[0003] According to the location of electrode placement, brain-computer interfaces can be divided into three types: non-invasive, semi-invasive, and invasive. Non-invasive brain-computer interfaces are based on electroencephalogram (EEG) signals. They are widely used in brain-computer interfaces because of their ease of use and low acquisition cost. However, the low signal-to-noise ratio and variability of their signals limit their further application in complex scenarios. Semi-invasive brain-computer interfaces use electrocorticograms (ECoG) to record signals implanted in the skull but outside the brain tissue, making a certain trade-off between signal quality and surgical risks. Invasive brain-computer interfaces involve implanting electrodes or microelectrode arrays directly into brain tissue, providing signals with the highest spatiotemporal resolution and temporal accuracy, making high-precision control of prosthetics or communication devices possible. However, invasive brain-computer interfaces also face problems such as infection risk, scar tissue, and long-term stability caused by the body's immune response.

[0004] Artificial neural networks (ANNs) have achieved remarkable success in the field of BCI. For example, Lawhern et al. proposed a compact EEGNet that combines temporal convolution with spatial deep convolution, which has been proven effective in various EEG-based paradigms. Although ANNs have achieved remarkable results, the high computational cost limits their further clinical application. In mobile or implantable brain-computer interface applications, excessive power will reduce the life of the device and may impair human function due to heat. At the same time, the low signal-to-noise ratio, non-stationarity, and significant inter-subject variability of EEG signals limit their application in complex tasks. In contrast, intracortical spike signals are characterized by high temporal accuracy and high spatiotemporal resolution. In addition, models designed for EEG signals may not be optimal in processing spike data, which emphasizes the need to develop architectures based on spiking neural networks (SNNs) for effective decoding.

[0005] Spiking neural networks are considered to be the third generation of neural network models, with rich temporal dynamics. Neurons in spiking neural networks communicate through binary spikes, and a variety of coding schemes can be used. The computational process in spiking neural networks is event-driven and only responds to input peak events, thus ensuring energy efficiency. In recent years, it has gradually attracted attention in the field of brain-computer interfaces. Its event-driven computing method and bio-inspired model characteristics provide new ideas for decoding brain signals. Although studies have explored the application of spiking neural networks in brain-computer interfaces, most of them are focused on the field of non-invasive BCI based on EEG signals, while there are still relatively few studies on decoding invasive BCI based on spike signals. Calibrating decoders and decoding high-dimensional neural signals face the challenges of high time and high computational burden. Calibrating decoders requires users to participate in experiments for a long time to collect enough data to ensure that the decoder captures stable neural activity patterns. At the same time, it requires repeated adjustments and verifications, which affects convenience and user experience. Decoding high-dimensional neural signals requires processing a large amount of data, which contains complex spatiotemporal dependencies, such as the time interval of spike signals and the synergistic characteristics of spatial channels. Complex feature extraction leads to a significant increase in computational complexity, which not only increases the difficulty of real-time decoding, but also places higher demands on hardware performance and power consumption, becoming a deployment bottleneck, especially in embedded systems. Summary of the invention

[0006] In view of the above defects or improvement needs of the prior art, the present invention provides a method for constructing a spike signal decoding model for an invasive brain-computer interface, which aims to solve the problem of high computational burden when decoding high-dimensional neural signals.

[0007] To achieve the above object, according to one aspect of the present invention, a method for constructing a spike signal decoding model for an invasive brain-computer interface is provided, comprising:

[0008] A pulse neural network is constructed, comprising a spatiotemporal feature extraction module and a channel fusion module and a classification module having multiple layers of spatiotemporal convolution; the spatiotemporal feature extraction module is used to extract the spike interval time characteristics of each channel signal in the multi-channel spike signal through the one-dimensional convolution therein, and obtain a pulse feature matrix through the spike neurons; the channel fusion module comprises a channel fusion unit, a time convolution unit and a refinement convolution unit, the channel fusion unit is used to fuse the inter-channel feature data in the pulse feature matrix in different ways through n two-dimensional convolutions, and obtain n one-dimensional pulse feature graphs through the spike neurons; the time convolution unit is used to extract features from each one-dimensional pulse feature graph through two-dimensional convolution, and obtain n new one-dimensional pulse feature graphs through the spike neurons; the refinement convolution unit is used to perform feature interaction between all the new one-dimensional pulse feature graphs for multiple times without changing the shape of the feature graph, so as to obtain multiple one-dimensional pulse interaction feature graphs, which are used to obtain decoding information through the classification module;

[0009] A training sample set is constructed to train the pulse neural network to obtain a spike signal decoding model.

[0010] Furthermore, a reverse synaptic connection is provided at the refined convolution unit in the pulse neural network.

[0011] Furthermore, the spatiotemporal feature extraction module is also used to perform point multiplication between an attention weight vector and each channel feature in the extracted pulse feature matrix based on an attention mechanism to obtain a pulse feature matrix that has undergone a channel-level attention mechanism.

[0012] Furthermore, a residual connection is provided at the refined convolution unit in the pulse neural network.

[0013] Furthermore, the training sample set is obtained by performing data enhancement on multi-channel spike signal samples;

[0014] The data enhancement method is time point masking, specifically:

[0015] A binary mask matrix with the same shape as the multi-channel spike signal sample is preset and initialized to an all-1 matrix; a probability of masking or not is randomly generated for each element in the binary mask matrix to mask the element and obtain a new binary mask matrix; the new binary mask matrix is ​​element-by-element dot multiplication with the spike signal sample to simulate the absence of spikes and obtain a new spike signal;

[0016] And / or, the data enhancement method is time period mask, specifically:

[0017] A binary mask matrix with the same shape as the single-channel spike signal is preset and initialized to an all-1 matrix; a probability of masking or not is generated for any segment of elements in the binary mask matrix to mask the segment of elements and obtain a new binary mask matrix; the new binary mask matrix is ​​element-by-element dot multiplication is performed on each channel in the multi-channel spike signal sample to simulate the absence of spikes and obtain a new spike signal;

[0018] And / or, the data enhancement method is channel masking, specifically:

[0019] A binary mask matrix with the same shape as the multi-channel spike signal at a single time point is preset and initialized to an all-1 matrix; a probability of masking or not is generated for each element in the binary mask matrix to mask the element and obtain a new binary mask matrix; the new binary mask matrix is ​​element-by-element dot multiplication is performed with each single-time-point multi-channel spike signal in the multi-channel spike signal sample to simulate the absence of spikes and obtain a new spike signal.

[0020] According to another aspect of the present invention, a spike signal decoding method applied to an invasive brain-computer interface is provided, and spike signal decoding is performed using a spike signal decoding model constructed by the spike signal decoding model construction method as described above.

[0021] According to another aspect of the present invention, an invasive brain-computer interface system is provided, wherein the pattern recognition module uses the spike signal decoding method as described above to perform pattern recognition.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the steps of the method described above and / or the steps of the method described above.

[0023] According to another aspect of the present invention, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implement the steps of the method described above and / or the steps of the method described above.

[0024] In general, compared with the prior art, the technical solution conceived by the present invention has the following beneficial effects:

[0025] 1. The present invention proposes a spike signal decoding model construction method for invasive brain-computer interface, which adopts a spiking neural network with multi-layer spatiotemporal convolution. The network effectively solves the problem of high computational burden when decoding high-dimensional neural signals through multi-layer module design and event-driven characteristics of spiking neural network, which is specifically reflected in the following aspects: (1) Effective feature representation of spatiotemporal feature extraction module: The spatiotemporal feature extraction module of the network extracts the spike interval time features of each channel in the multi-channel spike signal through one-dimensional convolution. This feature representation directly captures the timing information of the pulse signal, and at the same time converts these features into a pulse feature matrix with the help of spike neurons. (2) Efficient computational design of the channel fusion module: First, the channel fusion unit uses multiple two-dimensional convolutions to extract fusion features between channels in different ways, and generates a one-dimensional pulse feature map through spike neurons. Then, the temporal convolution unit extracts the temporal features of each one-dimensional pulse feature map through two-dimensional convolution. Finally, the refinement convolution unit strengthens the information flow between one-dimensional pulse feature maps through multiple feature interactions without changing the shape of the feature map, while maintaining the sparsity of the calculation and avoiding the redundant overhead in the fully connected calculation. This event-driven computing architecture, combined with modular design, enables the channel fusion module to efficiently process high-dimensional neural signals, significantly reducing the computational burden and power consumption. (3) Phased feature extraction and information interaction: The network decomposes high-dimensional signal processing into multiple stages of feature extraction and fusion. Each stage uses convolution operations and spike neurons for local calculations rather than global processing. This layer-by-layer refinement of feature extraction reduces the computational complexity caused by direct mapping of high-dimensional signals and improves decoding efficiency. (4) Event-driven computing reduces power consumption and redundancy: The network's calculation is only triggered when a spike event occurs in the input signal (i.e., the pulse signal value is 1), and no calculation is performed when the signal value is 0, thereby significantly reducing unnecessary computational overhead. In traditional artificial neural networks, all time steps are involved in the calculation, regardless of whether they contain meaningful information; while spike neural networks only calculate time steps with a signal value of 1. In addition, this sparse computing feature propagates layer by layer, because the output of spike neurons is also a sparse signal, and subsequent layers are only activated when they receive a spike input. As the network depth increases, sparsity accumulates layer by layer, further reducing the overall computational workload and energy consumption, making it particularly suitable for processing high-dimensional, sparse spike signals. Therefore, the present invention effectively solves the problem of high computational burden when decoding high-dimensional neural signals.

[0026] 2. The present invention further proposes that a reverse synaptic connection is provided at the refinement convolution unit, so that the response of each neuron is more stable and orderly, avoiding instability caused by sudden noise or external disturbances, that is, realizing a local synaptic stabilization mechanism. This mechanism enables neurons to maintain a stable activation state for a long time, which is particularly critical for the time interval information unique to spike signals, because these time intervals carry most of the neural activity information. Specifically, by re-inputting the output of the refinement convolution layer into the layer, the previous accumulated membrane potential is utilized, so that the learning efficiency of the layer is further improved, which not only increases the deep interaction of features, but also enables the continuous optimization and dynamic adjustment of the membrane potential of the spike neurons. This design enables neurons to accumulate and update membrane potentials more efficiently when processing sparse spike signals, thereby improving the ability of neurons to respond to spike signals, while enhancing the network's retention and decoding accuracy of timing information. By optimizing the update of the membrane potential of the fourth layer of spike neurons, performance and energy efficiency are significantly improved. Experiments show that this reverse connection improves model performance and reduces energy consumption, and also effectively alleviates the calibration time problem required for decoding high-dimensional neural signals in brain-computer interface systems.

[0027] 3. The present invention further proposes a channel-level attention mechanism to dynamically adjust the activation weight of each channel according to the signal strength or information contribution of each channel. For important channels, higher weights are given to enhance their ability to express key information; for irrelevant or noisy channels, their weights are reduced, thereby effectively reducing the interference of noise on the network. Through this adaptive weighting method, the network can more accurately extract task-related features and improve the overall decoding performance.

[0028] 4. The present invention further proposes three data enhancement methods, so that the pulse neural network can efficiently and accurately decode the neural signals in the cortex. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a pulse neural network architecture diagram for an invasive brain-computer interface provided by an embodiment of the present invention;

[0030] Figure 2 It is a pulse neural network structure diagram applied to an invasive brain-computer interface provided by an embodiment of the present invention;

[0031] Figure 3 It is a schematic diagram of data enhancement provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0033] Embodiment 1

[0034] A spike signal decoding model construction method for invasive brain-computer interface, such as Figure 1 As shown, including:

[0035] A pulse neural network is constructed, comprising a spatiotemporal feature extraction module and a channel fusion module and a classification module having multiple layers of spatiotemporal convolution; the spatiotemporal feature extraction module is used to extract the spike interval time characteristics of each channel signal in the multi-channel spike signal through the one-dimensional convolution therein, and obtain a pulse feature matrix through the spike neurons; the channel fusion module comprises a channel fusion unit, a time convolution unit and a refinement convolution unit, the channel fusion unit is used to fuse the inter-channel feature data in the pulse feature matrix in different ways through n two-dimensional convolutions, and obtain n one-dimensional pulse feature graphs through the spike neurons; the time convolution unit is used to extract features from each one-dimensional pulse feature graph through two-dimensional convolution, and obtain n new one-dimensional pulse feature graphs through the spike neurons; the refinement convolution unit is used to perform feature interaction between all the new one-dimensional pulse feature graphs for multiple times without changing the shape of the feature graph, so as to obtain multiple one-dimensional pulse interaction feature graphs, which are used to obtain decoding information through the classification module;

[0036] A training sample set is constructed to train the pulse neural network to obtain a spike signal decoding model.

[0037] Regarding the spatiotemporal feature extraction module, in order to fully extract the spatiotemporal dynamic features of the spike data, this embodiment uses a one-dimensional convolution layer (the convolution kernel size is, for example, 64) in the spatiotemporal feature extraction module, aiming to capture the temporal features (time intervals) within each individual channel. The initial one-dimensional convolution in the pulse neural network promotes the comprehensive extraction of cross-channel information and ensures that the dependencies between channels are well represented. After this, as a preference, a channel-level attention mechanism is applied to assign appropriate weights to each channel and enhance relevant features. The output is then introduced with nonlinearity through a spike neuron (e.g., a PLIF neuron) so that the model can process discrete peak information.

[0038] Regarding the channel fusion module, the channel fusion module with multiple layers of spatiotemporal convolution is the core of enhancing the network's ability to capture cross-channel interactions in input data. This module includes three different units: channel fusion unit, temporal convolution unit, and refinement convolution unit. Multiple spatiotemporal convolution units are used to fully extract the spatiotemporal information of spike signals.

[0039] Among them, the channel fusion unit uses multiple two-dimensional convolutions with preset kernel sizes of (C, 1) to fuse channel data, where C is the number of channels of the multi-channel spike signal, and the other parameters of the convolution kernel are parameters to be learned, so as to obtain multiple feature maps, thereby promoting feature interaction between channels. After convolution, batch normalization and average pooling are applied to normalize the data to reduce the size of the feature map, and finally obtain multiple one-dimensional pulse feature maps.

[0040] It is then processed by two key convolutional units (temporal convolutional unit and refinement convolutional unit), each designed to capture and refine different features. The temporal convolutional unit uses a two-dimensional convolution with a kernel size of (1, K) to capture sequential features across the time dimension (i.e., further extract temporal features), allowing the network to extract relevant features along the time dimension, and obtain multiple new one-dimensional pulse feature maps after temporal feature extraction. The refinement convolutional layer uses a two-dimensional convolution with a kernel size of (1, 1) to further refine the mutual mapping between feature maps, enhancing the model's ability to focus on basic features without changing the structure (i.e., shape) of the feature map. In each unit, a spiking neuron (preferably, a PLIF neuron) is applied at the output to facilitate effective peak activity in the network. This combination of sequential feature extraction and refined mapping ensures that the network captures complex patterns while maintaining computational efficiency.

[0041] Regarding the classification module, for example, after the spike signal undergoes hierarchical feature extraction and convolution operations, the classification module aggregates the extracted features using global average pooling, which simplifies each feature map into a single value, thereby generating a fixed-dimensional feature vector aligned with the target class count, thereby simplifying the complexity of the model while retaining basic information. In order to alleviate overfitting, a dropout layer is introduced to encourage the model to rely on multiple paths through the network and enhance generalization. Finally, the processed features are passed through a fully connected layer for final classification.

[0042] As a preferred implementation, a reverse synaptic connection is provided at the refinement convolution unit in the above-mentioned pulse neural network.

[0043] Inspired by biological neuron connections, a local synaptic stabilization mechanism is added to the refined convolutional unit. The local synaptic stabilization mechanism improves the learning efficiency and network performance of neurons by optimizing the membrane potential of deep neurons (i.e., spiking neurons of the refined convolutional unit).

[0044] The local synaptic stabilization (LSS) mechanism introduces stable synaptic connections into the network, allowing the response of each neuron to be updated stably and orderly without being disturbed by sudden noise or external disturbances. Its main features are: (1) Temporal and spatial stability: The LSS mechanism can adjust the update rules of the membrane potential so that neurons can maintain a stable activation state for a long time. This is crucial for the time interval information in the spike signal, because the interval information in the spike signal carries most of the accuracy of neural activity. (2) Noise suppression: The LSS mechanism effectively suppresses the impact of noise on decoding by smoothing the response pattern of neurons, thereby enhancing the robustness of the model. Through smooth signal transmission, LSS ensures that the network can ignore meaningless noise and focus on important spike activity.

[0045] LSS can ensure that the timing information of the spike signal is not excessively disturbed or lost during the transmission process, thereby enhancing the decodability of the signal. By improving the stability within the network, the LSS mechanism enables the network to better adapt to various changes and noises in the experiment, thereby improving decoding accuracy.

[0046] As a preferred implementation, the spatiotemporal feature extraction module is also used to perform point multiplication of the attention weight vector and each channel feature extracted from the pulse feature matrix based on the attention mechanism to obtain the pulse feature matrix after the channel-level attention mechanism.

[0047] The channel-wise attention mechanism (CA) dynamically allocates weights to each channel, enabling the network to focus on neuronal activities with key information while suppressing irrelevant or redundant channel signals. Compared with the traditional global attention mechanism, channel-wise attention is more suitable for the sparse characteristics of spike signals, significantly improving the energy efficiency of the network while reducing noise interference.

[0048] The channel-level attention mechanism (CA) dynamically adjusts the importance of each channel, allowing the network to focus on information-rich signal channels and suppress unimportant or noisy channels. This mechanism optimizes the network in the following ways: Adaptive selection: Adjust the activation weights of the channels according to the signal strength or information contribution of each channel. The signals of important channels will receive higher weights to enhance the expressiveness of the signals, while the weights of irrelevant channels are suppressed to reduce the impact of noise. Save computation and energy: Since the attention mechanism effectively reduces the computation of irrelevant channels by dynamically adjusting the weights of the channels, it can significantly reduce the amount of computation and energy consumption.

[0049] By focusing on important channels, the CA mechanism reduces the need for irrelevant computations, reduces the amount of computation, and thus reduces power consumption. By dynamically weighting signal channels, the network can selectively activate channels relevant to the task, thereby avoiding useless computations and energy waste.

[0050] In a preferred embodiment, a local synaptic stabilization spiking neural network with channel-wise attention (LSS-CA-SNN) is designed to fully tap the information potential in spike signals, improve decoding efficiency and reduce energy consumption.

[0051] Combining two innovative mechanisms: The local synaptic stabilization mechanism optimizes the membrane potential update of deep neurons after the convolution layer by simulating the synaptic stabilization characteristics of biological neurons, making it more robust to respond to the dynamic changes of spike signals while improving the ability to extract spike interval features. The channel-level attention mechanism dynamically allocates weights to each channel, enabling the network to focus on neuronal activities with key information while suppressing irrelevant or redundant channel signals. Compared with the traditional global attention mechanism, channel-level attention is more suitable for the sparse characteristics of spike signals, significantly improving the energy efficiency of the network while reducing noise interference. Therefore, in a preferred implementation, the pulse neural network combines the above two innovations to achieve a dual improvement in accuracy and energy efficiency in the task of decoding intrusive spike signals, which has important theoretical significance and practical application value.

[0052] As a preferred implementation, a residual connection is provided at the refinement convolution unit in the above-mentioned pulse neural network.

[0053] In order to solve the problem of vanishing gradients in deep layers, a residual connection is introduced in the last layer of the spiking neural network to ensure the effective utilization of deep features and the effective propagation of gradients.

[0054] A more specific example of a spiking neural network can be seen in Figure 2 shown.

[0055] As a preferred implementation, the training sample set is obtained by performing data enhancement on multi-channel spike signal samples;

[0056] Among them, the data enhancement method is time point mask, specifically:

[0057] A binary mask matrix with the same shape as the multi-channel spike signal sample is preset and initialized to an all-1 matrix; a probability of masking or not is randomly generated for each element in the binary mask matrix to mask the element and obtain a new binary mask matrix; the new binary mask matrix is ​​element-by-element dot multiplication with the spike signal sample to simulate the absence of spikes and obtain a new spike signal;

[0058] And / or, the data augmentation method is time segment masking, specifically:

[0059] A binary mask matrix with the same shape as the single-channel spike signal is preset and initialized to an all-1 matrix; a probability of masking or not is generated for any segment of elements in the binary mask matrix to mask the segment of elements and obtain a new binary mask matrix; the new binary mask matrix is ​​element-by-element dot multiplication is performed on each channel in the multi-channel spike signal sample to simulate the absence of spikes and obtain a new spike signal;

[0060] And / or, the data augmentation method is channel masking, specifically:

[0061] A binary mask matrix with the same shape as the multi-channel spike signal at a single time point is preset and initialized to an all-1 matrix; a probability of masking or not is generated for each element in the binary mask matrix to mask the element and obtain a new binary mask matrix; the new binary mask matrix is ​​element-by-element dot multiplication is performed with each single-time-point multi-channel spike signal in the multi-channel spike signal sample to simulate the absence of spikes and obtain a new spike signal.

[0062] Consider the training data as {X i}∈B C×T , where B represents the binary domain {0, 1}, i represents the i-th training data, C represents the number of channels (corresponding to the number of neurons), and T represents the discrete time point. Figure 3 As shown, data augmentation is performed by randomly masking the time point dimension, time period dimension, and channel dimension to simulate the absence of spikes and increase the data pattern:

[0063] Time point mask: define a binary mask matrix M t ∈B C×T , initialized to a matrix of all 1s, and then passed a probability p t To mask each element, thus simulating the absence of spikes, M t It can be obtained by the following formula: (a random probability value is randomly generated for each point)

[0064]

[0065] The enhanced data x i ' can be expressed as X i'=X i ⊙M t , where ⊙ represents element-wise multiplication.

[0066] Time segment mask: define a binary mask matrix M s ∈B 1×T , through a probability p s Randomly select a time period [t s , t s +l] all channel masks in M s It can be obtained by the following formula:

[0067]

[0068] The enhanced data x i 'x can be obtained through the broadcast mechanism i '=X i ⊙M s .

[0069] Channel mask: Define a binary mask matrix M c ∈B C×1 , through a probability p c Randomly mask each channel, M c It can be obtained by the following formula:

[0070]

[0071] Enhanced data X i 'x can be obtained through the broadcast mechanism i '=X i ⊙M c .

[0072] Using the above-mentioned SpikeDrop (spike mask) data enhancement, the spiking neural network can efficiently and accurately decode the neural signals in the cortex.

[0073] Experimental verification shows that on the invasive spike data from two adult male rhesus monkeys, the decoding method based on the pulse neural network proposed in the present invention shows obvious advantages over the traditional artificial neural network in terms of classification performance and energy efficiency. In addition, the experiment also verifies that the SpikeDrop data enhancement strategy proposed in the present invention has good applicability to a variety of neural network architectures, which not only improves the generalization performance of the pulse neural network model, but also has a certain performance improvement on the traditional artificial neural network. The experimental results in Table 1 clearly show the comprehensive superiority of the decoding method based on the pulse neural network of the present invention in terms of performance and energy consumption.

[0074] Table 1: Classification accuracy and energy consumption of different methods on intrusive spike data

[0075]

[0076] In Table 1, the best result is bolded and the second best result is underlined. The accuracy and energy consumption data in microjoules (uJ) are included. The content in brackets shows the energy consumption ratio of different networks relative to the pulse neural network proposed in the present invention in the same scenario.

[0077] In summary, the present invention proposes a pulse neural network method for the decoding problem of spike signals in invasive brain-computer interfaces, which can reduce computational complexity. In addition, the method innovatively combines the local synaptic stabilization mechanism with the channel-level attention mechanism, and has made significant progress in improving decoding performance, energy efficiency and model stability. Specifically, the local synaptic stabilization mechanism enhances the ability of the pulse neural network to handle complex time dynamics by optimizing the membrane potential and spike activation of neurons, ensuring the decoding accuracy during long-term operation. At the same time, the channel-level attention mechanism focuses on extracting the most informative features in the spike signal and suppressing the noise interference of irrelevant channels, thereby significantly reducing computational redundancy and energy consumption. This method not only solves the contradiction between traditional artificial neural networks in high-precision decoding and low-power operation, but also enhances the model's ability to capture spike signal features, providing a new direction for the efficient development of the field of invasive brain-computer interfaces. The proposal of the present invention is expected to promote the wider application of brain-computer interface systems in medical rehabilitation, emotion recognition, and human-computer interaction in the future.

[0078] In general, the present invention has verified through experimental tests that the pulse neural network model of the present invention has significant and stable performance in the classification of invasive spike EEG signals, and has significantly reduced power consumption compared to traditional ANN models, and can be used as a decoding method for actual spike-based invasive brain-computer interfaces.

[0079] As a further illustration, the method of this embodiment also has the following advantages: (1) Improving the robustness of the model. In brain-computer interface (BCI), the signal is affected by noise, interference and changes in the experimental environment, resulting in poor robustness of the model and prone to misdecoding. LSS establishes a stable synaptic connection between neurons and adjusts the update rule of the membrane potential, making the network's response to different stimuli more stable. This mechanism enables the network to ignore some burst noise and focus on meaningful spike activities, reducing the occurrence of misdecoding. By stabilizing the activation pattern of neurons and reducing overfitting, LSS effectively improves the robustness of the network in an uncertain environment. (2) Energy-saving optimization. Traditional deep neural networks often require a lot of computing resources when processing high-dimensional spatiotemporal signals, resulting in high power consumption, which is particularly prominent in real-time BCI applications, especially in invasive brain-computer interfaces. The channel-level attention mechanism adjusts the weight of each neuron channel so that the network can focus on the signal activity of important channels and suppress the signals of unimportant channels, thereby reducing irrelevant calculations and energy consumption. This mechanism effectively reduces the network's computational workload and power consumption by adaptively and selectively activating the most important neurons, thereby achieving the goal of energy saving. (3) Enhance the extraction of time interval information of spike signals. Compared with spiking neural networks (SNNs), traditional neural networks may ignore the time interval information in spike signals, which is crucial for decoding accuracy. Time interval information carries more semantics. PLIF neurons model the time interval of spike signals and adaptively adjust the membrane potential of neurons to enable them to respond to subtle time changes. In addition, the spatiotemporal convolution layer can deeply explore the potential information carried by the time interval by performing multi-level extraction of the spatiotemporal features of spike signals. Through the combination of PLIF and spatiotemporal convolution, LSS-CA-SNN can decode spike signals more accurately, improve the sensitivity to timing features, and thus improve the overall decoding accuracy. (4) Improve the training stability of the model. SNNs and deep neural networks are prone to problems such as gradient disappearance and gradient explosion during training, resulting in unstable training, especially in deep neural networks. The local synaptic stabilization mechanism optimizes the update rules of neuronal membrane potentials, making the network training process smoother and reducing training instability caused by gradient fluctuations. At the same time, PLIF neurons adjust the membrane potential and activation function of neurons to make the gradient change of spike signals more stable during propagation, avoiding the problem of gradient disappearance or explosion. In this way, the model can maintain stable convergence during training.

[0080] Embodiment 2

[0081] A spike signal decoding method applied to an invasive brain-computer interface is characterized in that a spike signal decoding model constructed by the spike signal decoding model construction method as described above is used to perform spike signal decoding.

[0082] The relevant technical solutions are the same as above and will not be described again here.

[0083] Embodiment 3

[0084] An invasive brain-computer interface system, wherein the pattern recognition module uses the spike signal decoding method as described above to perform pattern recognition.

[0085] The relevant technical solutions are the same as above and will not be described in detail here.

[0086] Embodiment 4

[0087] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0088] Specifically, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0089] The relevant technical solutions are the same as above and will not be described again here.

[0090] Embodiment 5

[0091] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the method of the above embodiment of the present application.

[0092] The relevant technical solutions are the same as above and will not be described in detail here.

[0093] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a spike signal decoding model for an invasive brain-computer interface, characterized in that: include: A pulse neural network is constructed, comprising a spatiotemporal feature extraction module and a channel fusion module and a classification module having multiple layers of spatiotemporal convolution; the spatiotemporal feature extraction module is used to extract the spike interval time characteristics of each channel signal in the multi-channel spike signal through the one-dimensional convolution therein, and obtain a pulse feature matrix through the spike neurons; the channel fusion module comprises a channel fusion unit, a time convolution unit and a refinement convolution unit, the channel fusion unit is used to fuse the inter-channel feature data in the pulse feature matrix in different ways through n two-dimensional convolutions, and obtain n one-dimensional pulse feature graphs through the spike neurons; the time convolution unit is used to extract features from each one-dimensional pulse feature graph through two-dimensional convolution, and obtain n new one-dimensional pulse feature graphs through the spike neurons; the refinement convolution unit is used to perform feature interaction between all the new one-dimensional pulse feature graphs for multiple times without changing the shape of the feature graph, so as to obtain multiple one-dimensional pulse interaction feature graphs, which are used to obtain decoding information through the classification module; A training sample set is constructed to train the pulse neural network to obtain a spike signal decoding model.

2. The method for constructing a spike signal decoding model according to claim 1, wherein: In the pulse neural network, a reverse synaptic connection is arranged at the refined convolution unit.

3. The method for constructing a spike signal decoding model according to claim 2, wherein: The spatiotemporal feature extraction module is also used to perform point multiplication between an attention weight vector and each channel feature in the extracted pulse feature matrix based on an attention mechanism to obtain a pulse feature matrix that has undergone a channel-level attention mechanism.

4. The method for constructing a spike signal decoding model according to claim 1, wherein: In the pulse neural network, a residual connection is arranged at the refinement convolution unit.

5. The method for constructing a spike signal decoding model according to any one of claims 1 to 4, characterized in that: The training sample set is obtained by performing data enhancement on multi-channel spike signal samples; The data enhancement method is time point masking, specifically: A binary mask matrix with the same shape as the multi-channel spike signal sample is preset and initialized to an all-1 matrix; a probability of masking or not is randomly generated for each element in the binary mask matrix to mask the element and obtain a new binary mask matrix; the new binary mask matrix is ​​element-by-element dot multiplication with the spike signal sample to simulate the absence of spikes and obtain a new spike signal; And / or, the data enhancement method is time period mask, specifically: A binary mask matrix with the same shape as the single-channel spike signal is preset and initialized to an all-1 matrix; a probability of masking or not is generated for any segment of elements in the binary mask matrix to mask the segment of elements and obtain a new binary mask matrix; the new binary mask matrix is ​​element-by-element dot multiplication is performed on each channel in the multi-channel spike signal sample to simulate the absence of spikes and obtain a new spike signal; And / or, the data enhancement method is channel masking, specifically: A binary mask matrix with the same shape as the multi-channel spike signal at a single time point is preset and initialized to an all-1 matrix; a probability of masking or not is generated for each element in the binary mask matrix to mask the element and obtain a new binary mask matrix; the new binary mask matrix is ​​element-by-element dot multiplication is performed with each single-time-point multi-channel spike signal in the multi-channel spike signal sample to simulate the absence of spikes and obtain a new spike signal.

6. A spike signal decoding method for an invasive brain-computer interface, characterized in that: A spike signal decoding model constructed by the spike signal decoding model construction method according to any one of claims 1 to 5 is used to perform spike signal decoding.

7. An invasive brain-computer interface system, characterized in that: The pattern recognition module uses the peak signal decoding method as described in claim 6 to perform pattern recognition.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the steps of the method according to any one of claims 1 to 5 and / or the steps of the method according to claim 6.

9. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 and / or the steps of the method according to claim 6 are implemented.

Images