A brain signal adaptive length time segmentation decoding method and system based on pulse neurons

Abstract

The application discloses a brain signal adaptive length time segmentation decoding method and system based on a pulse neuron, and the method comprises the following steps: acquiring a multi-channel brain signal and constructing a time sequence representation; converting the brain signal into an event-driven representation; inputting the event-driven representation into a pulse neural network to generate an adaptive length time segmentation trigger signal in the time dimension, and performing time segmentation on the brain signal according to the trigger signal; inputting the segmented brain signal time segment into a decoding model to execute a downstream brain signal decoding task; and optimizing the model for generating the time segmentation trigger signal according to the performance feedback of the decoding task. The application realizes adaptive length segmentation of the time structure of the brain signal by utilizing the threshold triggering and state resetting mechanism of the pulse neuron to isolate the historical information at the time segmentation point, and establishes a joint optimization relationship between the time segmentation and the decoding task, thereby improving the stability and performance of the brain signal decoding.

Description

Technical Field

[0001] This invention belongs to the field of brain-computer interface and neural signal processing technology, and in particular relates to a brain signal adaptive length-time segmentation decoding method and system based on spiking neurons. Background Technology

[0002] Brain-computer interface (BCI) technology is a key technology that enables information exchange between human neural activity and external devices by collecting and analyzing brain signals. It has significant application value in fields such as neurological disease diagnosis, cognitive state assessment, rehabilitation training, and human-computer interaction. Brain signals typically exist in the form of multi-channel time series, such as electroencephalograms (EEGs), which have high temporal resolution, significant non-stationarity, and strong individual variability.

[0003] Brain signal decoding is a core task of brain-computer interface systems. Typical decoding processes in existing technologies usually include brain signal acquisition, time segmentation, feature modeling, and task optimization. Among these, time segmentation divides continuous brain signals into several time segments for subsequent modeling and learning; the segmentation method has a significant impact on the final decoding performance.

[0004] (a) Limitations of fixed-length time segmentation.

[0005] Among existing brain signal decoding methods, the most commonly used approach is fixed-length time segmentation, which involves pre-setting a uniform time window length and uniformly dividing the brain signal according to this window. However, brain signals exhibit significant non-stationary characteristics in the time dimension, and their statistical features change with time, cognitive state, and external stimuli. Furthermore, there are significant differences in the time scale of neural activity among different subjects and for different tasks.

[0006] In this situation, fixed-length time segmentation is difficult to adjust according to the actual neural activity rhythm and easily ignores individual and task differences. At the same time, fixed windows may cut off signals before the neural activity pattern has ended or merge multiple different patterns into the same time segment, thereby disrupting the intrinsic temporal structure of brain signals and causing large fluctuations in decoding performance among different subjects or tasks.

[0007] (II) Current status of research on adaptive length time segmentation methods.

[0008] To overcome the shortcomings of fixed segmentation methods, some existing technologies attempt to introduce adaptive time-length segmentation methods, dynamically determining segmentation points based on the characteristics of brain signals. Existing methods mainly include segmentation based on statistical properties and those based on recurrent neural networks.

[0009] Segmentation methods based on statistical characteristics typically determine segmentation points by detecting changes in statistical features such as brain signal amplitude, frequency, or correlation. These methods have a certain degree of interpretability, but the segmentation rules rely on manually set thresholds, which limits their versatility and makes it difficult to directly correlate them with the performance goals of downstream decoding tasks.

[0010] Methods based on recurrent neural networks (RNNs) utilize their temporal modeling capabilities to generate segmentation decisions and have strong learning abilities. However, they retain historical state information after generating segmentation points, making it difficult to effectively isolate irrelevant temporal information. Furthermore, in long-term series or discrete segmentation decision scenarios, it is difficult to achieve stable optimization through standard backpropagation.

[0011] (III) Application of spiking neural networks in time series modeling.

[0012] Spiking Neural Networks (SNNs) are a class of models inspired by biological neurons. They encode temporal information in the form of pulses and possess mechanisms such as membrane potential integration, threshold triggering, and state resetting, making them naturally suitable for time-series data modeling. In existing technologies, spiking neural networks have been applied to various time-varying signal processing tasks and have demonstrated good temporal sensitivity. However, current methods mainly focus on feature extraction or classification modeling, and have not yet systematically applied the resetting mechanism of spiking neurons to time segmentation decisions, nor have they addressed the issues of historical information isolation during segmentation and the co-optimization of segmentation and decoding objectives.

[0013] (iv) Gradient blocking problem in joint optimization of segmentation and decoding.

[0014] In the brain signal decoding process, time segmentation is typically a non-differentiable operation, and the generation of segmentation points corresponds to discrete decisions. This makes it difficult for the loss function of downstream tasks to backpropagate through the segmentation module, resulting in gradient blocking. In this case, the segmentation module often has to rely on manual rules or heuristic methods for optimization, which has a high computational cost and lacks systematicity, making it difficult to apply to complex tasks or large-scale data scenarios. Summary of the Invention

[0015] To address the problems of fixed-time segmentation methods in existing brain signal decoding processes being unable to adapt to the non-stationarity of brain signals, adaptive-length segmentation methods being unable to isolate irrelevant historical temporal features, and gradient blocking between segmentation and decoding modules, this invention provides a brain signal adaptive-length-time segmentation decoding method and system based on spiking neurons, which can improve the overall performance and stability of brain signal decoding.

[0016] An adaptive length-time segmentation decoding method for brain signals based on spiking neurons includes the following steps: (1) Obtain raw brain signals from multiple channels and construct time-series representations; (2) Convert raw brain signals into event-driven representations; (3) Input the event-driven representation into the spiking neural network to generate a segmentation scheme consisting of time-division trigger signals; (4) Based on the time segmentation trigger signal, generate time segmentation points of adaptive length in the time dimension to segment the brain signal; (5) Input the segmented brain signals into the decoding model to perform the brain signal decoding task; (6) Optimize the spiking neural network that generates the segmentation scheme based on the performance feedback of the decoding task.

[0017] In step (1), the time series representation of the brain signals is as follows: ; in, Represents the original brain signal matrix. The number of channels representing brain signals. This indicates the number of sampling points in the time dimension.

[0018] In step (2), the event-driven representation is generated by comparing the changes in brain signal amplitude at adjacent time sampling points. When the change amplitude exceeds a preset threshold, an event is generated, and each event is represented in the form of a triple: ; in, This represents the index of the brain signal channel corresponding to the event. Indicates the polarity of the signal change. It indicates the time and location of an event; multiple events are arranged in chronological order to form an event set, which serves as the time input for the spiking neural network.

[0019] In step (3), the spiking neural network is composed of multiple spiking neurons, and the state update of each spiking neuron at the discrete time step satisfies: ; in, Indicates time membrane potential, This represents the neuron's state at the previous moment. This represents the input current generated by the event-driven representation. It is a time constant; When the membrane potential exceeds a preset threshold, the spiking neuron outputs a pulse signal. Its output format is: ; in, Indicates the threshold. Represents a step function; when At that time, the corresponding moment is mapped to the time segmentation trigger point on the original time axis.

[0020] When the spiking neuron outputs a pulse at time t, the corresponding neuron state is reset as follows: ; in, The preset reset potential is used; through the reset operation, the accumulated state in the previous time period is cleared at the time segmentation trigger point, so that the segmentation decision of the subsequent time period is decoupled from the neural activity of the previous time period, thereby achieving explicit isolation of the historical temporal characteristics at the time segmentation point.

[0021] In step (5), each time segment of the segmented brain signal is input into the decoding model as an independent time unit for feature extraction and task prediction, making the prediction results of the decoding model sensitive to the time segmentation position, thereby binding the time segmentation with the decoding task at the model structure level.

[0022] In step (6), a random-greedy optimization strategy based on pseudo-labels is introduced. By comparing the task performance corresponding to the newly generated segmentation scheme with that of the current pseudo-label, it is determined whether to update the pseudo-label. Let the temporal segmentation pseudo-label in the Nth round of training be... The segmentation scheme generated by the spiking neural network in the current round is: Its update rules are as follows: ; in, Let be a random variable that follows a uniform distribution and takes values ​​from 0 to 1. For the probability of acceptance; The above-mentioned random-greedy strategy prioritizes the adoption of new segmentation schemes when they can improve task performance; and accepts new time segmentation schemes with a preset probability when they lead to a decrease in task performance. This maintains the ability to explore in the discrete-time segmentation decision space and avoids the segmentation strategy from converging to a local optimum.

[0023] An adaptive length-time segmentation decoding system for brain signals based on spiking neurons, comprising: The event conversion module is used to convert continuous brain signals into event-driven representations; The spiking neural network module is used to generate time-segmented trigger sequences in the time dimension based on event-driven representations. The time segmentation module is used to generate time segmentation points with unequal intervals on the original time axis based on the time segmentation trigger sequence; The decoding module is used to perform downstream decoding tasks with time segments as the basic input unit; The feedback update module receives the task performance evaluation output by the decoding module and feeds the evaluation back to the spiking neural network module to update the generation strategy of the time segmentation trigger sequence.

[0024] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention utilizes the reset mechanism of spiking neurons to automatically isolate irrelevant historical temporal features when generating each segmentation point, thereby improving the effectiveness and stability of brain signal time segmentation.

[0025] 2. This invention does not require manual setting of a fixed segmentation length. It can adaptively adjust the segmentation method according to the intrinsic temporal structure of brain signals, adapting to different subjects and different tasks.

[0026] 3. This invention introduces a pseudo-label optimization strategy based on task performance, which can still achieve collaborative optimization of the segmentation module and the brain signal decoding task even when the segmentation operation is non-differentiable.

[0027] 4. This invention can significantly improve the performance of various brain signal decoding tasks and has good versatility and interpretability. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of the overall process of an adaptive length-time segmentation decoding method for brain signals based on spiking neurons, according to an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram comparing the process of the present invention with that of existing methods.

[0031] Figure 3 This is a schematic diagram illustrating how the traditional loop model in this invention cannot clear irrelevant historical information.

[0032] Figure 4 This is a schematic diagram of the spiking neuron structure in this invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0035] like Figure 1 As shown, an adaptive length-time segmentation decoding method for brain signals based on spiking neurons includes the following steps: S01: Acquire raw brain signal data and form a multi-channel time series representation for subsequent processing.

[0036] In this embodiment, brain signals are acquired using a brain signal acquisition device, and these brain signals can be electroencephalogram (EEG) signals. The acquired brain signals are arranged in chronological order to form multi-channel time-series data, the mathematical representation of which is as follows: ; in, Represents the original brain signal matrix. The number of channels representing brain signals. This indicates the number of sampling points in the time dimension. Each row corresponds to one brain signal channel, and each column corresponds to one time sampling point.

[0037] In this embodiment, brain signals can undergo conventional preprocessing operations as needed after input, including but not limited to denoising, normalization, or resampling. However, the above preprocessing operations do not constitute a limitation on the technical solution of this invention.

[0038] The task labels corresponding to brain signals are represented as follows: ; in, F This indicates the dimension of the label features.

[0039] S02 converts the acquired raw brain signals into event-driven representations to highlight the changes in brain signals over time.

[0040] In this embodiment, events are generated by detecting changes in brain signal amplitude between adjacent time sampling points. An event is generated when the signal amplitude change of a certain channel between adjacent time sampling points exceeds a preset threshold. Each event is represented as a triple: ; in, This represents the index of the brain signal channels in which the event occurred; This indicates the polarity of the signal change, used to indicate whether the signal amplitude is rising or falling. This indicates the time and location of the event.

[0041] Multiple events arranged in chronological order constitute an event set. : ; When the change in brain signals at adjacent time points exceeds a preset threshold, a corresponding event is triggered.

[0042] To facilitate subsequent processing by the spiking neural network, in this embodiment, the event set is further converted into a voxelized event representation. Its definition is: ; in, The downsampling time interval; It is the Dirac function; For the indicator function; the event representation obtained from this satisfies: ; in, .

[0043] The second dimension corresponds to different polarities of change. This represents the duration of time on the event timeline. Through the above event-based representation construction process, continuous brain signals are transformed into a sparse, change-driven representation.

[0044] S03, input the obtained event representation into the spiking neural network to generate time-division trigger signals in the time dimension.

[0045] In this embodiment, a spiking neural network composed of multiple spiking neurons is used to process the event-based representation. For example... Figure 4 As shown, each spiking neuron uses a leaky integral firing model, and its membrane potential update process at discrete time steps is as follows: ; in, Indicates at time step The input current, Represents the time constant. This represents the membrane potential at the current time step. This represents the neuron state at the previous time step.

[0046] When the membrane potential exceeds a preset threshold, the spiking neuron outputs a pulse signal, the output of which is as follows: ; in, Indicates the threshold. This represents the step function. After the output pulse, the neuron's state is reset to: ; in, Indicates pulse output; when At this time, the neuron outputs a pulse and resets its state, thereby isolating the accumulated information from the previous time period, such as... Figure 3 As shown, this is a unique reset mechanism of spiking neural networks to achieve historical information isolation.

[0047] Through this mechanism, spiking neurons automatically clear accumulated information from previous time periods after outputting a pulse, ensuring that subsequent time steps are no longer affected by irrelevant historical information. (Pulse signal) The generation time is used as the trigger signal for time segmentation.

[0048] Representing events Inputting the signal into a spiking neural network yields a pulse output sequence: .

[0049] S04. Based on the pulse signal output by the spiking neural network, time segmentation points of adaptive length are generated in the time dimension, and the brain signal is segmented accordingly.

[0050] In this embodiment, the spiking neural network outputs pulse signals on the event time axis. When at a certain time step When a pulse signal is generated, the position corresponding to that time step is taken as a time-division trigger point. A set of time-division points is thus constructed, represented as: ; in, This indicates the mapping relationship from the event timeline to the original timeline. It represents the length of the original brain signal in the time dimension.

[0051] In this embodiment, the system is based on a set of time division points. The original brain signal obtained in step S01 is segmented into several time segments by dividing the intervals between adjacent segmentation points. Each time segment corresponds to a relatively independent subsequence of brain signals, used to represent a potential neural activity pattern.

[0052] like Figure 2As shown, the number and location of time segmentation points are adaptively determined by the spiking neural network without the need for a pre-set fixed time window length. This allows for dynamic adjustment of the segmentation method based on the intrinsic temporal structure of the brain signal, enabling modeling of individual heterogeneity—the most substantial change compared to existing methods. Therefore, when brain signals exhibit different rates of change or different neural activity patterns in different time periods, the time segmentation method can automatically adjust the segmentation interval, generating denser segmentation points in time regions of more dramatic changes and reducing the number of segmentation points in time regions of more gradual changes. Simultaneously, since each time segmentation point is triggered by the firing behavior of spiking neurons and accompanied by a reset of the neuron state, the feature information between adjacent time segments is effectively isolated at the segmentation point, thereby reducing the interference of irrelevant historical information on the modeling of subsequent time segments and further improving the stability and effectiveness of the segmented brain signal when used for downstream decoding tasks.

[0053] S05, the obtained segmented brain signals are input into the downstream decoding model to execute the corresponding brain signal decoding task.

[0054] In this embodiment, the brain signal within each time segment is treated as an independent processing unit and input into the decoding model for feature extraction and task prediction. The decoding model can be a convolutional neural network, a recurrent neural network, or other model structures suitable for brain signal decoding, and its specific form does not constitute a limitation of the present invention.

[0055] Let the segmented brain signals be represented as The corresponding task tag is represented as The result output by the decoding model is used to calculate the task loss function, which is expressed as: ; in, Represents the decoding model. This represents the loss function.

[0056] Simultaneously, the system calculates task performance metrics based on the decoding results to evaluate the degree of support the current time segmentation scheme provides for the decoding task, which are expressed as follows: ; In this embodiment, the task performance index can be classification accuracy, regression error, or other evaluation indexes, depending on the type of brain signal decoding task being performed.

[0057] S06. Based on the task performance indicators obtained in step S05, optimize the time segmentation model to generate a better segmentation scheme.

[0058] Since time segmentation is a discrete decision, it is difficult to optimize directly through backpropagation. Therefore, this embodiment introduces a stochastic-greedy optimization strategy based on pseudo-labels.

[0059] Let the time segment pseudo-label during the Nth round of training be . The segmentation scheme generated by the segmentation model (spiking neural network) in the current round is: .

[0060] In this embodiment, the decision to update the pseudo-labels is made by comparing the task performance corresponding to the newly generated segmentation scheme with that of the current pseudo-labels. The update rules are as follows: ; in, Let be a random variable that follows a uniform distribution and takes values ​​from 0 to 1. The probability of acceptance is defined as follows: ; Through the above-mentioned random-greedy strategy, when a new segmentation scheme can improve task performance, the system will prioritize accepting the scheme; when the new scheme causes a decrease in task performance, the system will still accept the scheme with a certain probability, thereby avoiding the segmentation model from getting trapped in local optima during the optimization process.

[0061] Through multiple rounds of iterative training, the time segmentation model gradually learns segmentation strategies that are more beneficial to downstream decoding tasks, achieving collaborative optimization between time segmentation and decoding tasks.

[0062] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A brain signal adaptive length-time segmentation decoding method based on spiking neurons, characterized in that, Includes the following steps: (1) Obtain raw brain signals from multiple channels and construct time-series representations; (2) Convert raw brain signals into event-driven representations; (3) Input the event-driven representation into the spiking neural network to generate a segmentation scheme consisting of time-division trigger signals; (4) Based on the time segmentation trigger signal, generate time segmentation points of adaptive length in the time dimension to segment the brain signal; (5) Input the segmented brain signals into the decoding model to perform the brain signal decoding task; (6) Optimize the spiking neural network that generates the segmentation scheme based on the performance feedback of the decoding task.

2. The brain signal adaptive length-time segmentation decoding method based on spiking neurons according to claim 1, characterized in that, In step (1), the time series representation of the brain signals is as follows: ； in, Represents the original brain signal matrix. The number of channels representing brain signals. This indicates the number of sampling points in the time dimension.

3. The brain signal adaptive length-time segmentation decoding method based on spiking neurons according to claim 1, characterized in that, In step (2), the event-driven representation is generated by comparing the changes in brain signal amplitude at adjacent time sampling points. When the change amplitude exceeds a preset threshold, an event is generated, and each event is represented in the form of a triple: ； in, This represents the index of the brain signal channel corresponding to the event. Indicates the polarity of the signal change. It indicates the time and location of an event; multiple events are arranged in chronological order to form an event set, which serves as the time input for the spiking neural network.

4. The brain signal adaptive length-time segmentation decoding method based on spiking neurons according to claim 1, characterized in that, In step (3), the spiking neural network is composed of multiple spiking neurons, and the state update of each spiking neuron at the discrete time step satisfies: ； in, Indicates time membrane potential, This represents the neuron's state at the previous moment. This represents the input current generated by the event-driven representation. It is a time constant; When the membrane potential exceeds a preset threshold, the spiking neuron outputs a pulse signal. Its output format is: ； in, Indicates the threshold. Represents a step function; when At that time, the corresponding moment is mapped to the time segmentation trigger point on the original time axis.

5. The brain signal adaptive length-time segmentation decoding method based on spiking neurons according to claim 4, characterized in that, When the spiking neuron outputs a pulse at time t, the corresponding neuron state is reset as follows: ； in, The preset reset potential is used; through the reset operation, the accumulated state in the previous time period is cleared at the time segmentation trigger point, so that the segmentation decision of the subsequent time period is decoupled from the neural activity of the previous time period, thereby achieving explicit isolation of the historical temporal characteristics at the time segmentation point.

6. The brain signal adaptive length-time segmentation decoding method based on spiking neurons according to claim 1, characterized in that, In step (5), each time segment of the segmented brain signal is input into the decoding model as an independent time unit for feature extraction and task prediction, making the prediction results of the decoding model sensitive to the time segmentation position, thereby binding the time segmentation with the decoding task at the model structure level.

7. The brain signal adaptive length-time segmentation decoding method based on spiking neurons according to claim 1, characterized in that, In step (6), a random-greedy optimization strategy based on pseudo-labels is introduced. By comparing the performance of the newly generated segmentation scheme with the task corresponding to the current pseudo-label, it is determined whether to update the pseudo-label. Let the time segment pseudo-label during the Nth round of training be . The segmentation scheme generated by the spiking neural network in the current round is: Its update rules are as follows: ； in, Let be a random variable that follows a uniform distribution and takes values ​​from 0 to 1. For the probability of acceptance; The above-mentioned random-greedy strategy prioritizes the adoption of new segmentation schemes when they can improve task performance; and accepts new time segmentation schemes with a preset probability when they lead to a decrease in task performance. This maintains the ability to explore in the discrete-time segmentation decision space and avoids the segmentation strategy from converging to a local optimum.

8. A brain signal adaptive length-time segmentation decoding system based on spiking neurons, characterized in that, include: The event conversion module is used to convert continuous brain signals into event-driven representations; The spiking neural network module is used to generate time-segmented trigger sequences in the time dimension based on event-driven representations. The time segmentation module is used to generate time segmentation points with unequal intervals on the original time axis based on the time segmentation trigger sequence; The decoding module is used to perform downstream decoding tasks with time segments as the basic input unit; The feedback update module receives the task performance evaluation output by the decoding module and feeds the evaluation back to the spiking neural network module to update the generation strategy of the time segmentation trigger sequence.

Images