A method for extracting task-related neural signals for invasive brain-computer interface

Abstract

The present invention discloses a method for extracting neural signals related to invasive brain-computer interface tasks, comprising the following steps: (1) constructing an encoder h to convert the original neural signal x into a feature space to obtain a neural representation z = h(x), and using an affine function f to decode z to obtain a first predicted speed y1 = f(z) of a target speed y; (2) constructing a decoder g to convert the neural representation z into a neural signal x r = g(z); (3) The generated neural signal x r Resend it to the encoder h and the affine function f to obtain the second predicted speed y2=f(h(x r )); (4) Construct a priori generative neural network m, and learn the priori neural representation z from the target speed y p =m(y); (5) Calculate the loss function and simultaneously train the encoder, decoder, and prior generative neural network; (6) Input the original signal to be processed into the trained encoder and decoder to obtain the generated neural signal. Utilizing the present invention, decoding performance and generation performance can be significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of invasive action potential brain signal analysis, and in particular to a method for extracting task-related neural signals of an invasive brain-computer interface. Background Art

[0002] Understanding how the brain encodes and decodes task information is a fundamental goal of neuroscience and neuroengineering. However, neural signals exhibit significant variability, with repeated experiments yielding significantly different neural activity. This variability can be caused by numerous factors, including intrinsic noise in the neural system, neuroplasticity, instability in signal recordings, and the simultaneous encoding of multiple task parameters by neurons.

[0003] The significant variability of neural signals has a significant impact on both the study of neural mechanisms and the application of brain-machine interfaces (BMIs). Because the true task-related signals are unknown, current research relies on analyzing raw, noisy signals, potentially leading to unreliable conclusions. Therefore, extracting the cleanest possible task-related signals is of great significance to the fields of neuroscience and neural engineering.

[0004] For example, Chinese patent document CN103584851A discloses a multi-channel neuronal signal acquisition, control, and transmission device. In response to the requirements of neuroelectrophysiology for neuronal information analysis, an FPGA-based multi-channel parallel acquisition and synchronous stimulation device is designed. The device is suitable for multi-channel neuronal signal extraction from animals and provides a variety of programmable stimulation waveforms.

[0005] Chinese patent document with publication number CN111984118A discloses a method for decoding electromyographic signals from electroencephalographic signals based on a complex recurrent neural network, including: constructing electroencephalographic signal data and electromyographic signal data, and preprocessing them separately; performing short-time Fourier transform on the preprocessed results; inputting the transformed results into a complex recurrent neural network for training and testing; and performing an inverse short-time Fourier transform on the test results to obtain a decoding result.

[0006] There are some methods that can also extract task-related neural signals, such as pi-VAE, PSID, VAE, LFADS, etc., but they can only extract a small part of the neural signals. Currently, there is no method that focuses on maximizing the extraction of task-related signals. Summary of the Invention

[0007] The present invention provides a method for extracting task-related neural signals for an invasive brain-computer interface, which can maximize the extraction of task-related signals. At the same time, compared with other methods, both decoding performance and generation performance are significantly improved.

[0008] The technical solutions of the present invention are as follows:

[0009] A method for extracting neural signals related to invasive brain-computer interface tasks, comprising the following steps:

[0010] (1) Construct an encoder h to convert the original neural signal x into the feature space, obtain the neural representation z = h(x), and use the affine function f to decode z to obtain the first predicted speed y1 = f(z) of the target speed y;

[0011] (2) Construct a decoder g to convert the neural representation z into a neural signal x r =g(z);

[0012] (3) The generated neural signal x r Resend it to the encoder h and the affine function f to obtain the second predicted speed y2=f(h(x r ));

[0013] (4) Construct a priori generative neural network m, and learn the priori neural representation z from the target speed y p =m(y);

[0014] (5) Calculate the loss function and train the encoder, decoder, and prior generative neural network simultaneously;

[0015] (6) The original signal to be processed is input into the trained encoder and decoder to obtain the generated neural signal.

[0016] Furthermore, in the present invention, the encoder, decoder and priori generative neural network all adopt multi-layer perceptrons.

[0017] In step (5), back propagation is used in an end-to-end manner to train the encoder, decoder and prior generative neural network.

[0018] The specific process of calculating the loss function is:

[0019] (5-1) Calculate the loss function of the first predicted speed y1 and the target speed y

[0020] (5-2) Calculate and generate neural signal x r The loss function of the original neural signal x

[0021] (5-5) Calculate the loss function of the second predicted speed y2 and the target speed y

[0022] (5-6) Calculate the prior neural representation z p KL divergence loss function with the neural representation z

[0023] (5-5) The weighted combination of the four loss functions gives the final loss The formula is:

[0024]

[0025] In step (5-1), the loss function of the first predicted speed y1 and the target speed y is calculated The mean square error loss function is used, and the formula is:

[0026]

[0027] Where i represents the i-th sample and N represents the total number of samples.

[0028] In step (5-2), the neural signal x is calculated and generated r The loss function of the original neural signal x The Poisson negative log-likelihood loss function is used, and the formula is:

[0029]

[0030] In step (5-3), the loss function of the second predicted speed y2 and the target speed y is calculated The mean square error loss function is also used, and the formula is:

[0031]

[0032] In step (5-4), the prior neural representation z is calculated p KL divergence loss function with the neural representation z The formula is:

[0033]

[0034] Among them, q, p represent z and z respectively. p distribution.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. The method of the present invention is improved on the basis of the variational autoencoder. By decoding the neural representation z and reconstructing (generating) the original signal and then decoding it again, it can maximize the extraction of task-related signals, which is beneficial to the subsequent research on neural mechanisms and the application of neural engineering.

[0037] 2. Compared with other methods, the method of the present invention significantly improves the decoding performance and generation performance of extracted neural signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1Schematic diagram of an animal experiment on motion signal decoding according to an embodiment of the present invention;

[0039] Figure 2 This is an architectural diagram of a method for extracting task-related neural signals of an invasive brain-computer interface according to the present invention;

[0040] Figure 3 The figure is a comparison chart of the average decoding and generation performance of the embodiment of the present invention and other methods on two days of test data. DETAILED DESCRIPTION

[0041] The present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be noted that the following examples are intended to facilitate understanding of the present invention and do not have any limiting effect on the present invention.

[0042] The present invention uses data from a monkey collected in the document “Li, H., Hao, Y., Zhang, S., Wang, Y., Chen, W., & Zheng, X. (2017). Prior Knowledge of Target Direction and Intended Movement Selection Improves Indirect Reaching Movement Decoding. Behavioural Neurology, 2017.”

[0043] In the experiment, an adult male rhesus monkey was trained to use its arms to control a large joystick to complete the task of avoiding obstacles and moving the cursor to the target point. The monkey was fixed in a chair with a monitor placed 50 cm in front of the chair. The monkey was trained to use its arms to control the joystick to move the cursor on a two-dimensional plane from the initial position (small circle) to avoid obstacles (rectangular bars) to the target position (large circle). At this time, the system automatically gave the monkey water as a reward. Figure 1 shown.

[0044] The experiment used a 96-channel microelectrode array (10×10 matrix, 4.2×4.2 mm) implanted in the dorsal premotor cortex (PMd) of monkeys. Neural signals acquired by the array were transmitted to the Cerebus system. The analog waveforms of the neural signals were amplified and then bandpass filtered using a Butterworth filter (0.3 Hz to 7.5 kHz). They were then digitized (16-bit resolution and 30 kHz sampling rate) and Butterworth high-pass filtered (250 Hz). Finally, action potential pulse signals were acquired using a threshold method (minus 4.5 times the baseline root mean square).

[0045] In the present invention, two days of data are used for testing, MUA (multiunit activity) data is used, and the pulse signal is binned with a time window of 100ms.

[0046] The method framework diagram of the present invention is as follows Figure 2 As shown, a method for extracting neural signals related to invasive brain-computer interface tasks includes the following steps:

[0047] Step S1: Construct an encoder h to convert the original neural signal x into the feature space to obtain the neural representation z = h(x), and use the affine function f to decode it. z The first predicted speed y1=f(z) of the target speed y is obtained.

[0048] Step S2: Construct a decoder g to convert the neural representation z into a neural signal x r =g(z).

[0049] Step S3: Send the generated neural signal back to the encoder h and the affine function f to obtain the second predicted speed y2=f(h(x r )).

[0050] Step S4: construct a priori generative neural network m to learn the priori neural representation z from the target speed y p =m(y).

[0051] Step S5, calculate the loss function and use an end-to-end backpropagation training model (encoder + decoder + prior generative neural network).

[0052] S501, calculate the loss function of the predicted speed y1 and the target speed y

[0053]

[0054] Where i represents the i-th sample and N represents the total number of samples.

[0055] S502, calculate and generate neural signal x r The loss function of the original neural signal x

[0056]

[0057] S503, calculate the loss function of the predicted speed y2 and the target speed y

[0058]

[0059] S504, calculate the prior neural representation z pKL divergence loss function with the neural representation z

[0060]

[0061] Among them, q, p represent z and z respectively. p distribution.

[0062] S505, the weighted combination of the four loss functions is used to obtain the final loss The formula is as follows:

[0063]

[0064] In the present invention, the encoder, decoder and priori generative neural network can all adopt multi-layer perceptrons.

[0065] To verify the feasibility of the proposed method for extracting task-related signals from motor areas, the model was tested in a monkey obstacle avoidance paradigm. In the experiment, an invasive electrode array was used to capture signals from the dorsal premotor cortex (PMd) of the monkey brain, collecting them as multi-channel discrete pulse trains.

[0066] This method uses two days of data to decode the extracted task-related signals using an artificial neural network (ANN). The prediction results are calculated using the coefficient of determination R. 2 To measure the decoding effect. The similarity between the generated neural signal and the original neural signal is also an important indicator, and the determination coefficient R is also used. 2 To measure.

[0067] The experimental results are as follows Figure 3 As shown in the figure, the experimental results show that the decoding and generation capabilities of the neural signals extracted by the method of the present invention are significantly better than those of VAE, LFADS, pi-VAE, PSID and other methods.

[0068] By using the present invention, the extraction of task-related neural signals can be maximized, which is beneficial to the subsequent research on neural mechanisms and the application of neural engineering. The decoding and generation performance is significantly better than other methods.

[0069] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for extracting neural signals related to invasive brain-computer interface tasks, characterized in that: The steps include: (1) Construct an encoder h to convert the original neural signal x into the feature space, obtain the neural representation z = h(x), and use the affine function f to decode z to obtain the first predicted speed y1 = f(z) of the target speed y; (2) Construct a decoder g to convert the neural representation z into a neural signal x r =g(z); (3) The generated neural signal x r Resend it to the encoder h and the affine function f to obtain the second predicted speed y2=f(h(x r )); (4) Construct a priori generative neural network m, and learn the priori neural representation z from the target speed y p =m(y); (5) Calculate the loss function and train the encoder, decoder, and prior generative neural network at the same time; the specific process of calculating the loss function is: (5-1) Calculate the loss function of the first predicted speed y1 and the target speed y The mean square error loss function is used, and the formula is: Where i represents the i-th sample and N represents the total number of samples; (5-2) Calculate and generate neural signal x r The loss function of the original neural signal x The Poisson negative log-likelihood loss function is used, and the formula is: (5-3) Calculate the loss function of the second predicted speed y2 and the target speed y The mean square error loss function is used, and the formula is: (5-4) Calculate the prior neural representation z p KL divergence loss function with the neural representation z The formula is: Among them, q, p represent z and z respectively. p distribution of (5-5) The weighted combination of the four loss functions gives the final loss (6) The original signal to be processed is input into the trained encoder and decoder to obtain the generated neural signal.

2. The method for extracting neural signals related to invasive brain-computer interface tasks according to claim 1, characterized in that: The encoder, decoder and prior generation neural network all use multi-layer perceptrons.

3. The method for extracting neural signals related to invasive brain-computer interface tasks according to claim 1, characterized in that: In step (5), back propagation is used in an end-to-end manner to train the encoder, decoder and prior generative neural network.

Images