Brain-computer interface closed-loop nerve regulation method, storage medium, system and program product

Abstract

The invention provides a brain-computer interface closed-loop nerve regulation and control method, a storage medium, a system and a program product, and the method comprises the following steps: circularly executing the following steps according to a first period until an end condition is met: controlling transcranial magnetic stimulation equipment to run for a first time length according to a set stimulation parameter to stimulate a testee; collecting an electroencephalogram signal sequence of the testee in the target brain area; the electroencephalogram signal sequence is processed, a preset index is obtained, and the preset index is used for reflecting the active condition of the target brain area; and processing the preset index by using a deep reinforcement learning network to obtain a stimulation parameter of the next period. According to the method, the brain state is adjusted according to accurate individual characteristics in a self-adaptive closed-loop mode, and a more accurate and effective nerve regulation and control scheme can be provided.

Description

Technical Field

[0001] The present disclosure generally relates to the field of neural regulation technology, and more specifically, to a brain-computer interface closed-loop neural regulation method, storage medium, system and program product. Background Art

[0002] Neuromodulation technology is a technology that precisely regulates brain neural activity through physical or chemical means (such as electrical stimulation, magnetic stimulation, drugs, etc.). It is used to treat neurological diseases, improve neural function or study brain mechanisms. It has broad application prospects in the fields of medical neuroscience and computational neurology. After decades of traditional physical therapy practice, neuroscientists and doctors have gradually realized that studying individual differences in environmental, biological and socio-psychological factors is crucial to disease treatment. This concept is summarized as "precision medicine." When exploring new treatments, taking these individual differences into full consideration can help achieve personalized regulation tailored to the specific characteristics of the patient.

[0003] However, it is particularly important to apply the concept of "precision medicine" to emerging treatment technologies, such as transcranial magnetic stimulation (TMS). As a safe and non-invasive means of neuromodulation, TMS has achieved remarkable results in clinical treatment. By combining TMS with electroencephalogram (EEG) technology, the activity of electrical neuron groups in the brain can be monitored with the assistance of scalp electrodes, so that precise TMS stimulation can be implemented in target areas of sustained neuronal oscillations under specific brain states. This process is called brain state-dependent stimulation. This method of precisely regulating brain activity lays the foundation for the realization of "personalized TMS" treatment.

[0004] At present, "personalized TMS" is still in its infancy. On the one hand, an open-loop control strategy is usually adopted to first detect the brain state of the subject, and then determine the TMS stimulation parameters with reference to the detected brain state, and then stimulate according to the stimulation parameters. Since the ongoing neuronal activity in the stimulated brain area is not known, the precise time targeting of the brain state is limited. On the other hand, when determining the TMS stimulation parameters with reference to the detected brain state, a fixed state-parameter rule is used, that is, a certain brain state corresponds to a certain set of stimulation parameters, and the precise detection of the brain state itself is difficult. At the same time, the fixed rule is obtained by artificial analysis and summary, and its accuracy is limited, which results in insufficient accuracy of the determined stimulation parameters.

[0005] Therefore, how to improve the stimulation accuracy of "personalized TMS" is an urgent problem to be solved in applying the concept of "precision medicine" to neuromodulation technology. Summary of the invention

[0006] The present disclosure provides a brain-computer interface closed-loop neural regulation method, storage medium, system and program product, which are used to solve at least one of the above-mentioned problems.

[0007] According to one aspect of the present disclosure, a brain-computer interface closed-loop neural regulation method is provided, comprising: executing the following steps in a first cycle loop until an end condition is met: controlling a transcranial magnetic stimulation device to run for a first duration according to set stimulation parameters to stimulate a subject; collecting an electroencephalogram signal sequence of the subject in a target brain area; processing the electroencephalogram signal sequence to obtain a preset indicator, wherein the preset indicator is used to reflect the activity of the target brain area; using a deep reinforcement learning network to process the preset indicator to obtain stimulation parameters for the next cycle.

[0008] Optionally, collecting the EEG signal sequence of the subject in the target brain area includes: collecting the EEG signal sequence of multiple channels of the subject, wherein the multiple signal collection points corresponding to the multiple channels include multiple points of the target brain area, the target brain area includes a default mode network, and the multiple points of the target brain area include at least one of the following: the midpoint of the vertex, the midpoint of the occipital, the central vertex, the left vertex, and the right vertex.

[0009] Optionally, the processing of the EEG signal sequence to obtain a preset indicator includes: performing compression preprocessing on the EEG signal sequence to obtain a compressed EEG signal sequence; and processing the compressed EEG signal sequence to obtain a preset indicator.

[0010] Optionally, the preset indicator includes at least one of the following: average power spectrum density, frequency band phase synchronization index, and phase locking value.

[0011] Optionally, the compression preprocessing of the EEG signal sequence to obtain a compressed EEG signal sequence includes: for the EEG signal sequence of each channel of the multiple channels, sampling according to a preset sampling frequency to obtain multiple signal subsequences; for each signal subsequence of each channel, retaining signals less than a preset frequency, and compressing signals greater than or equal to the preset frequency to obtain each compressed signal subsequence of each channel; for each compressed signal subsequence of each channel, determining a correlation measure between the compressed signal subsequence and a compressed signal subsequence of the same time sequence in an adjacent channel; and from all compressed signal subsequences, removing compressed signal subsequences whose correlation measures are less than a correlation threshold to obtain the compressed EEG signal sequence.

[0012] Optionally, before the step of looping the following steps according to the first cycle until the end condition is met, the step further includes: looping the following steps according to the second cycle until the end duration is reached, wherein the duration of the second cycle is less than the duration of the first cycle, and the end duration is of the same order of magnitude as the duration of the first cycle: controlling the transcranial magnetic stimulation device to run for a second duration according to the set stimulation parameters to stimulate the subject; collecting the electroencephalogram signal sequence of the subject in the target brain area within a third duration, wherein the third duration is greater than the second duration; processing the electroencephalogram signal sequence to obtain the preset index; using the deep reinforcement learning network to process the preset index to obtain the stimulation parameters of the next cycle, wherein, in the process of looping the above steps according to the second cycle, the deep reinforcement learning network is initialized and trained using the experience replay mechanism, and the experience data used for training includes the experience data collected in the process of looping the above steps according to the second cycle.

[0013] Optionally, the experience data collected when the subject previously received transcranial magnetic stimulation is recorded as historical experience data, wherein the experience data used for the training also includes the historical experience data; or wherein the deep reinforcement learning network on which the initialization training is based is a deep reinforcement learning network trained using the historical experience data.

[0014] Optionally, the brain-computer interface closed-loop neural regulation method also includes: comparing the preset indicator with a preset value range to obtain regulation evaluation information, wherein the preset value range is obtained by counting the values ​​of the preset indicator of the target population, and the regulation evaluation information is used to represent the evaluation of the neural regulation effect.

[0015] Optionally, during the initialization training of the deep reinforcement learning network, an immediate reward for deep reinforcement learning is determined based on the preset indicator and a preset value range, wherein the preset value range is obtained by statistically analyzing the values ​​of the preset indicator of the target population.

[0016] According to another aspect of the present disclosure, a brain-computer interface closed-loop neural regulation device is provided, including: a first cycle unit, configured to cyclically run the following units according to a first cycle until an end condition is met; a first control unit, configured to control the transcranial magnetic stimulation device to run for a first duration according to set stimulation parameters to stimulate a subject; a first acquisition unit, configured to acquire an electroencephalogram signal sequence of the subject in a target brain area; a first processing unit, configured to process the electroencephalogram signal sequence to obtain a preset indicator, wherein the preset indicator is used to reflect the activity of the target brain area; and a first regulation unit, configured to use a deep reinforcement learning network to process the preset indicator to obtain stimulation parameters for the next cycle.

[0017] Optionally, the first acquisition unit is further configured to collect electroencephalogram signal sequences of multiple channels of the subject, wherein the multiple signal acquisition points corresponding to the multiple channels include multiple points of the target brain area, the target brain area includes a default mode network, and the multiple points of the target brain area include at least one of the following: the midpoint of the vertex, the midpoint of the occipital, the central vertex, the left vertex, and the right vertex.

[0018] Optionally, the first processing unit is further configured to: perform compression preprocessing on the electroencephalogram signal sequence to obtain a compressed electroencephalogram signal sequence; and process the compressed electroencephalogram signal sequence to obtain a preset indicator.

[0019] Optionally, the preset indicator includes at least one of the following: average power spectrum density, frequency band phase synchronization index, and phase locking value.

[0020] Optionally, the first processing unit is further configured to: for the EEG signal sequence of each channel of the multiple channels, sample according to a preset sampling frequency to obtain multiple signal subsequences; for each signal subsequence of each channel, retain signals less than a preset frequency, and compress signals greater than or equal to the preset frequency to obtain each compressed signal subsequence of each channel; for each compressed signal subsequence of each channel, determine a correlation measure between the compressed signal subsequence and a compressed signal subsequence of the same time sequence in an adjacent channel; from all compressed signal subsequences, remove compressed signal subsequences whose correlation measures are less than a correlation threshold to obtain the compressed EEG signal sequence.

[0021] Optionally, it also includes a second loop unit, which is configured to run the following units in a second cycle before running the first loop unit until the end duration is reached, wherein the duration of the second cycle is less than the duration of the first cycle, and the end duration is of the same order of magnitude as the duration of the first cycle; a second control unit, which is configured to control the transcranial magnetic stimulation device to run for a second duration according to the set stimulation parameters to stimulate the subject; a second acquisition unit, which is configured to collect the electroencephalogram signal sequence of the subject in the target brain area within a third duration, wherein the third duration is greater than the second duration; a second processing unit, which is configured to process the electroencephalogram signal sequence to obtain the preset indicator; a second control unit, which is configured to use the deep reinforcement learning network to process the preset indicator to obtain the stimulation parameters of the next cycle, wherein, in the process of executing the above steps in a cycle according to the second cycle, the deep reinforcement learning network is initialized and trained using the experience replay mechanism, and the experience data used for training includes the experience data collected in the process of executing the above steps in a cycle according to the second cycle.

[0022] Optionally, the experience data collected when the subject previously received transcranial magnetic stimulation is recorded as historical experience data, wherein the experience data used for the training also includes the historical experience data; or wherein the deep reinforcement learning network on which the initialization training is based is a deep reinforcement learning network trained using the historical experience data.

[0023] Optionally, the brain-computer interface closed-loop neural regulation device also includes an evaluation unit, which is configured to compare the preset indicator with a preset value range to obtain regulation evaluation information, wherein the preset value range is obtained by counting the values ​​of the preset indicator of the target population, and the regulation evaluation information is used to represent the evaluation of the neural regulation effect.

[0024] Optionally, during the initialization training of the deep reinforcement learning network, an immediate reward for deep reinforcement learning is determined based on the preset indicator and a preset value range, wherein the preset value range is obtained by statistically analyzing the values ​​of the preset indicator of the target population.

[0025] According to another aspect of the present disclosure, a computer-readable storage medium storing instructions is provided, wherein, when the instructions are executed by at least one computing device, the at least one computing device is prompted to execute the brain-computer interface closed-loop neural regulation method as described above.

[0026] According to another aspect of the present disclosure, a system is provided, comprising at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, prompt the at least one computing device to execute the brain-computer interface closed-loop neural regulation method as described above.

[0027] Optionally, the at least one computing device includes: a first computing device, used to control the operation of the transcranial magnetic stimulation device and the electroencephalogram signal acquisition device; a second computing device, used to process the electroencephalogram signal sequence to obtain the preset indicators; and a third computing device, used to use the deep reinforcement learning network to process the preset indicators to obtain the stimulation parameters for the next cycle.

[0028] According to another aspect of the present disclosure, a computer program product is provided, comprising instructions, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to execute the brain-computer interface closed-loop neural regulation method as described above.

[0029] According to the brain-computer interface closed-loop neural regulation method, storage medium, system and program product of the exemplary embodiment of the present disclosure, by first controlling the transcranial magnetic stimulation device to run for a first period of time according to the set stimulation parameters, then collecting the subject's electroencephalogram signal sequence and analyzing and processing it, and then updating the stimulation parameters according to the analysis results (i.e., preset indicators), and repeating this cycle, it is possible to adjust the stimulation parameters in time in response to changes in the subject's brain state, thereby achieving closed-loop neural regulation, which helps to improve the accuracy of brain state time targeting. When updating the stimulation parameters, by analyzing and processing the collected EEG signal sequence, preset indicators with higher information concentration can be extracted to reflect the brain state, thereby improving the efficiency of determining new stimulation parameters; at the same time, by using a deep reinforcement learning network to process preset indicators to determine new stimulation parameters, compared with the fixed state-parameter rules in related technologies, there is no need to give a clear brain state, and implicit data associations can be mined. Compared with other machine learning networks, the experience replay mechanism can be used to accumulate and learn experience, and implicit data associations that are more in line with the current subject can be mined, thereby more efficiently determining the stimulation parameters suitable for the current subject, and improving the accuracy of the determined stimulation parameters. In general, the present disclosure adjusts the brain state according to precise individual characteristics in an adaptive closed-loop manner, which helps to provide a more accurate and effective neural regulation scheme (i.e., stimulation parameters).

[0030] Additional aspects and / or advantages of the present general inventive concept will be set forth in part in the following description and in part will be apparent from the description or may be learned through practice of the present general inventive concept. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] These and / or other aspects and advantages of the present disclosure will become clear and more easily understood from the following description of the embodiments in conjunction with the accompanying drawings.

[0032] Figure 1 is a flow chart showing a closed-loop neural regulation method of a brain-computer interface according to an exemplary embodiment of the present disclosure.

[0033] Figure 2 is a schematic diagram showing the control process of the brain-computer interface closed-loop neural control method according to an exemplary embodiment of the present disclosure.

[0034] Figure 3 is a block diagram showing a brain-computer interface closed-loop neural regulation device according to an exemplary embodiment of the present disclosure.

[0035] Figure 4 is a schematic diagram showing a brain-computer interface closed-loop neural regulation system according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the embodiments of the present invention as defined by the claims and their equivalents. Various specific details are included to assist in understanding, but these details are considered to be exemplary only. Therefore, one of ordinary skill in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. In addition, descriptions of well-known functions and structures are omitted for clarity and brevity.

[0037] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three types of parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three types of parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0038] Refer to the following Figures 1 to 4 The brain-computer interface closed-loop neural regulation method, storage medium, system and program product according to exemplary embodiments of the present disclosure are described in detail.

[0039] Figure 1 is a flow chart showing a closed-loop neural control method for a brain-computer interface according to an exemplary embodiment of the present disclosure. The closed-loop neural control method for a brain-computer interface according to an exemplary embodiment of the present disclosure can be implemented in a computing device with sufficient computing power.

[0040] Reference Figure 1 , step S101 to step S104 are executed cyclically according to the first cycle until the end condition is met. The end condition will be introduced together with step S104, and will not be expanded here.

[0041] In step S101, the transcranial magnetic stimulation device is controlled to operate for a first time period according to set stimulation parameters to stimulate the subject.

[0042] As an example, the stimulation parameters include stimulation intensity and / or stimulation frequency. Regarding stimulation intensity, a conventional motor evoked potential (MEP) may be performed on the subject to determine the stimulation response of the subject and determine the initial stimulation intensity motor threshold (MT). In subsequent execution, the stimulation intensity is maintained within a safe range (for example, including but not limited to 80% to 120%) of the initial stimulation intensity motor threshold (MT).

[0043] As an example, the first duration ranges from 3 minutes to 10 minutes, including but not limited to 4 minutes, 5 minutes, and 7 minutes, which is long enough to ensure reliable effectiveness of neural regulation, and short enough to facilitate timely updating of stimulation parameters, thereby helping to achieve precise personalized neural regulation.

[0044] In step S102, the EEG signal sequence of the subject in the target brain area is collected.

[0045] Step S101 and step S102 can realize the TMS-EEG joint control process, for example, a computing device can be used to control the transcranial magnetic stimulation device and the electroencephalogram signal acquisition device in real time to achieve stimulation and signal acquisition. As an example, the computing device can turn off the transcranial magnetic stimulation device when the electroencephalogram signal acquisition device is enabled through a rotation control mechanism, and enable the electroencephalogram signal acquisition device during the interval period when the transcranial magnetic stimulation device performs stimulation, so as to achieve the effect of real-time regulation. It should be understood that "real-time" does not mean precise synchronization in the strict sense, but means that it is carried out continuously at the set frequency, so that intervals within the design range and reasonable delays due to technical limitations are allowed. The same applies to the "real-time" that appears in the following text, and no longer explained one by one. In addition, the "real-time regulation" here focuses on emphasizing the closed-loop regulation of transcranial magnetic stimulation, which can collect electroencephalogram signals in time during the stimulation process and analyze and update the stimulation parameters.

[0046] As an example, the acquisition frequency of the EEG signal acquisition device can be fixed at 500 Hz to 2000 Hz, and a wired data interface can be used to provide data to a computing device for executing step S103 to analyze the EEG signal sequence, and the data interface can use the TCP / IP Socket communication protocol.

[0047] Optionally, step S102 includes: collecting electroencephalogram signal sequences of multiple channels of the subject, wherein the multiple signal collection points corresponding to the multiple channels include multiple points in the target brain area, the target brain area includes a default mode network, and the multiple points in the target brain area include at least one of the following: the midpoint of the parietal (Pz), the midpoint of the occipital (POz), the central vertex (CPz), the left vertex (P3), and the right vertex (P4). The multiple points in the target brain area that serve as signal collection points can also be used as stimulation points of the transcranial magnetic stimulation device to achieve multi-point transcranial magnetic stimulation, thereby transforming the transcranial magnetic stimulation to multi-node brain stimulation at the default mode network level of the brain.

[0048] The default mode network (DMN) in the brain is a key social function regulation system that plays an important role in the pathogenesis of a variety of neuropsychiatric diseases. Studies have shown that DMN, as a highly integrated platform, is responsible for processing complex information related to social cognition and emotional processes. DMN mainly includes the following brain regions: posterior cingulate cortex (PCC), anterior cingulate cortex (ACC), precuneus cortex (Precuneus), medial prefrontal cortex (mPFC), dorsolateral prefrontal cortex (DLPFC), temporo-parietal junction (TPJ), etc. These brain regions are interconnected through complex neural networks and jointly maintain the default state function of the brain.

[0049] As an example, the EEG signal acquisition device includes an EEG cap, which covers the monitoring range of the default mode network of the two regions of PCC and Precuneus, and is based on the standard 10-20 system electrode placement method specified by the International Electroencephalography Society. PCC includes two points, Pz and POz, and Precuneus includes three points, CPz, P3, and P4. Of course, other points in the target brain area may also be used, and the present disclosure does not limit this.

[0050] In step S103, the electroencephalogram signal sequence is processed to obtain a preset index.

[0051] The preset indicators are used to reflect the activity of the target brain area and can achieve indirect and quantitative representation of the brain state.

[0052] Optionally, the preset indicator includes at least one of the following: average power spectral density (Power Spectral Density, PSD for short), frequency band phase synchronization index (Phase Synchronization Index, PSI for short), and phase locking value (Phase Locking Value, PLV for short).

[0053] Specifically, the average power spectral density is the average value of the power spectral density of the EEG signal at a specific frequency k. The processed signal sequence can be divided into several segments according to the time window, and the power spectral density of the signal in each time window at a specific frequency k is calculated, and then the average value of all windows is calculated. The formula can be expressed as: .

[0054] In this formula, P m (k) is the power spectral density of the mth time window, M is the number of time windows, and the length of each time window is, for example, no more than 2s. The power spectral density in each time window is calculated using a standard discrete Fourier transform (DFT) or fast Fourier transform (FFT) calculation method. As an example, for a case where the number of channels of an EEG signal sequence is multiple, since the average power spectral density can be calculated for a single channel, the average power spectral density of each channel can be obtained, and the statistical value of the average power spectral density of each channel can be further calculated, and the present disclosure does not limit this.

[0055] The frequency band phase synchronization index and phase lock value are both used to measure the consistency of frequency phases in different measurement areas. The phase angle can be obtained by using Hilbert Transform, and then calculated according to the respective formulas of the frequency band phase synchronization index and phase lock value. It should be understood that since these two indicators are for calculating the phase consistency of two signals, when the number of channels of the EEG signal sequence is two, the calculation can be performed directly. When the number of channels is greater than two, the signals of multiple channels can be paired first, and then the indicators of the paired channel signals can be calculated. Finally, the indicators of multiple channel pairs can be summarized, such as but not limited to calculating statistical values ​​such as the mean, mode, and median as the final indicator.

[0056] Optionally, step S103 includes: compressing and preprocessing the EEG signal sequence to obtain a compressed EEG signal sequence; processing the compressed EEG signal sequence to obtain a preset indicator. By first performing real-time compression and preprocessing on the acquired EEG signal sequence, the amount of data can be reduced, and the calculation speed when processing to obtain the preset indicator can be improved, thereby improving the real-time processing capability, and facilitating the rapid implementation of real-time processing after the signal is acquired, so as to determine the brain state required to trigger transcranial magnetic stimulation (indicated by the preset indicator), and fully guarantee the response rate of closed-loop neural regulation. It should be understood that the calculation time for processing to obtain the preset indicator is extremely short and almost negligible.

[0057] Regarding compression preprocessing, optionally, in step S103, the operation of compressing and preprocessing the EEG signal sequence to obtain a compressed EEG signal sequence includes: for the EEG signal sequence of each channel in multiple channels, sampling according to a preset sampling frequency to obtain multiple signal subsequences; for each signal subsequence of each channel, retaining signals less than the preset frequency, and compressing signals greater than or equal to the preset frequency, to obtain each compressed signal subsequence of each channel. By sampling the signal sequence of each channel, the signal sequence can be split to obtain multiple signal subsequences, providing a stable and reliable data basis for signal compression. Specifically for each signal subsequence, by retaining the required low-frequency signal (i.e., signal less than the preset frequency) and compressing the high-frequency signal, detailed key information can be selectively retained, reducing the amount of data of other information, thereby improving the information density of the compressed data. As an example, the preset frequency is 10Hz to retain signals below the β wave (i.e., α wave, β wave, δ wave, θ wave) less than 10Hz. Of course, the preset frequency can also be set to other frequencies as needed, and the present disclosure is not limited to this.

[0058] As an example, in order to obtain sufficient data to provide a reliable basis for neuromodulation decisions, the collected EEG signal sequence may be required to have a sufficient duration, for example, including but not limited to 1s to 2s. In actual implementation, the signal acquisition may be stopped when the duration represented by the collected EEG signal sequence is long enough, and step S103 may be performed on the collected signal. Alternatively, compression preprocessing may be performed while the signal is acquired until the duration represented by the final signal sequence is long enough. The present disclosure does not impose any restrictions on this.

[0059] Further optionally, the operation of performing compression preprocessing on the EEG signal sequence in step S103 to obtain a compressed EEG signal sequence also includes: determining, for each compressed signal subsequence of each channel, a correlation measure between the compressed signal subsequence and a compressed signal subsequence of the same time sequence in an adjacent channel; and removing compressed signal subsequences whose correlation measures are less than a correlation threshold from all compressed signal subsequences to obtain a compressed EEG signal sequence. By further calculating the correlation measure between compressed signal subsequences of adjacent channels of the same time sequence and removing compressed signal subsequences with insufficient correlation, invalid data can be further reduced, information density can be increased, and real-time processing capabilities can be further improved.

[0060] As an example, the correlation measure includes, but is not limited to, covariance (Covariance, commonly represented by the symbol Cov).

[0061] As an example, since this embodiment will delete the compressed signal subsequences with insufficient correlation, the duration represented by the compressed EEG signal sequence obtained in the end may be shortened compared to the original EEG signal sequence. Therefore, for each channel, the EEG signal sequence can be sampled, compressed and removed according to the preset sampling frequency while collecting the EEG signal sequence. That is, each time a number of EEG signals corresponding to the preset sampling frequency are collected (for example, f / 10 EEG signals, where f is the preset sampling frequency), these signals are determined as a signal subsequence and compressed to obtain compressed signal subsequences of multiple channels in the current time sequence, and then the correlation measure of adjacent channels is calculated, and then the compressed signal subsequences with insufficient correlation are removed. At the same time, the duration represented by all the compressed signal subsequences currently obtained is continuously counted. If the duration is long enough, for example, including but not limited to reaching the duration threshold of 1s or 2s, it is considered that the amount of data is sufficient as a decision basis for neural regulation, and the collection of EEG signals is stopped. Of course, it is also possible to first collect an EEG signal sequence of a certain length and perform compression preprocessing. If the compressed signal sequence is not long enough, continue to collect EEG signals. Other achievable collection methods can also be used, as long as it can ensure that the final compressed EEG signal sequence has a sufficient length. The present disclosure does not limit the specific collection and preprocessing methods.

[0062] As an example, the method of real-time acquisition, compression and processing of EEG signals is to obtain the original signal data of the EEG in real time, including the timestamp of each channel (Ts, accurate to 1ms) and the original electrical signal of each timestamp (unit uV), and convert the original signal data into a triplet T(n) (multiple triplets can be obtained for each channel), and push it into the compressed processing queue (Compressed Process Queue, referred to as CPQ), that is, the compressed EEG signal sequence. Each triplet contains: 1. Compression value (Cpv); 2. Variance value (Var); 3. Timestamp (Ts). The specific steps include the following.

[0063] 1) Use the Median median and algorithm to simplify the compression value of each f / 10 (where f is the sampling frequency) sampled electrical signal (i.e., a signal subsequence) received from each channel, that is, compress the data from 10Hz to 1kHz, and use the median of these data (i.e., the middle value after sorting by size) as the compression value to reduce the data dimension. At the same time, retain the signals below the β wave (α wave, β wave, δ wave, θ wave) with a frequency less than 10Hz, and add the variance value of these f / 10 sampled electrical signals to reflect the degree of data change. Also, add the middle point of the timestamp of these f / 10 sampled electrical signals to ensure the time correlation of the signal, and finally obtain a tuple T(n), where n is the compression sequence number.

[0064] 2) Calculate the covariance Cov between the currently compressed f / 10 signals and the time-consistent f / 10 signals in the adjacent channel as a correlation measure. If the correlation measure is less than a correlation threshold (for example, including but not limited to 0.1), discard the T(n); otherwise, put the T(n) into the CPQ queue.

[0065] 3) When the compressed pre-processed data in the CPQ queue reaches the minimum processing time threshold (e.g., 1s to 2s), data processing begins, and the power spectrum density of the α band, β band, etc. of each channel in the window period, as well as the phase synchronization index or phase locking value between each channel in the α band, β band, etc. are calculated. After the processing is completed, the average preset index of each channel is obtained, the processed data in the CPQ queue is cleared, and the calculated preset index result is given to the calculation device for executing step S104 in real time.

[0066] In step S104, the preset indicators are processed using a deep reinforcement learning network to obtain stimulation parameters for the next cycle.

[0067] This step uses a deep reinforcement learning network to make neural control decisions. Deep reinforcement learning is a technology that combines deep learning and reinforcement learning to learn the optimal strategy in a given environment.

[0068] As an example, the deep reinforcement learning network is the DQN (Deep Q-Network) network. The core of DQN is to use a deep neural network to approximate the Q function, that is, the state-action value function. The Q function represents the expected return that can be obtained by taking a specific action in a specific state. DQN contains the following connotations.

[0069] Q-Learning Basics: DQN is based on the Q-Learning algorithm, which is a model-free reinforcement learning algorithm that aims to learn an optimal strategy to maximize the cumulative return.

[0070] Neural network approximation: DQN uses a deep neural network to approximate the Q function. The deep neural network receives the state (in this disclosure, the state is specifically a preset indicator) as input and outputs the Q value of each possible action (in this disclosure, the action is specifically a stimulus parameter) .

[0071] Experience replay: In order to reduce the correlation between data and avoid overfitting, DQN uses an experience replay mechanism to replay the experience of the agent interacting with the environment (including the state s before executing the action). t , Action performed a t , immediate reward r, state s after executing the action t+1) is stored in a replay memory and then randomly sampled for training. In one epoch learning, several continuously differentiable Q functions can be specified and updated iteratively according to the set network weight steps.

[0072] Target Q network: To improve the stability of training, DQN uses two neural networks: the main Q network and the target Q network. The main Q network is used to predict Q values, while the target Q network is used to generate target Q values ​​(the output value of the target Q network needs to be processed in a certain way before the target Q value is obtained). The weights of the target Q network are regularly updated to the weights of the current main Q network to follow the main Q network.

[0073] The target Q value can be calculated according to the following formula: .

[0074] In this formula, Q new (s t , a t ) is the target Q value, α is the learning rate, Q(s t , a t ) is the target Q network calculated in state s t Take action a t Q value, r is the immediate reward, γ is the discount factor, max a Q(s t+1 ,a) is in the next state s t+1 The maximum Q value (calculated using the target Q network) of all possible actions a under

[0075] Learning strategy: The learning strategy of DQN includes selection strategy, reward strategy and shutdown strategy. The selection strategy determines how the agent chooses an action in a given state, such as selecting the action with the largest predicted Q value. The reward strategy is a reward under given conditions, that is, a way to determine the immediate reward. The shutdown strategy is to stop making decisions when the corresponding conditions are met. In the present disclosure, it means no longer updating the stimulation parameters and stopping neural regulation. The conditions met here are the end conditions of executing steps S101 to S104 in the first cycle as recorded above. As an example, the end condition includes: 1. After the cycle is executed for a preset time, the change rate of the preset indicator of the most recent preset round is less than the preset change rate. The preset time can guarantee the basic neural regulation time, thereby meeting the effectiveness standard. The preset change rate indicates that the subject's state tends to be stable, indicating that continuing transcranial magnetic stimulation may be difficult to cause further changes in the subject's state, so the regulation can be terminated. For example, the condition may be that after 5 minutes, the rate of change of the PSD values ​​in the last two rounds is less than 0.1%, and the rate of change of the PSI / PLV values ​​in the last two rounds is less than 0.1%. 2. From the time the subject is subjected to transcranial magnetic stimulation, the total duration of the experience reaches the preset regulation duration (for example, including but not limited to 20 minutes, 30 minutes). That is, regardless of whether the subject's state can continue to change, the regulation ends when a sufficiently long time has passed throughout the entire process. This can reduce the physical and mental consumption of the subject caused by long-term regulation, and can reduce the subject's physical load. When either of these two conditions is met, the cycle can be ended. The combination of these two end conditions can take into account both the effect of neural regulation and the subject's physical feelings, and achieve more reasonable neural regulation.

[0076] The training process of DQN includes the following steps.

[0077] 1) Initialize the weights of the main Q network and the target Q network.

[0078] 2) Interact with the environment, collect experience and store it in replay memory.

[0079] 3) Randomly sample a batch of experiences from the replay memory.

[0080] 4) Use the main Q-network to predict the Q-values ​​of these experiences.

[0081] 5) Using the target Q network, calculate the target Q value according to the above formula.

[0082] 6) Calculate the loss based on the predicted Q value and the corresponding target Q value, and use the gradient descent method to update the weights of the main Q network.

[0083] 7) Regularly update the weights of the target Q network to make it close to the weights of the main Q network.

[0084] It should be noted that, regarding the stimulation parameters of transcranial magnetic stimulation, high stimulation frequency (for example, including but not limited to greater than or equal to 1 Hz) and high stimulation intensity (for example, including but not limited to 100% to 120% of the initial stimulation intensity motor threshold MT) can achieve the inhibition of the activity level of the target brain area, and low stimulation frequency (for example, including but not limited to less than 1 Hz) and low stimulation intensity (for example, including but not limited to 80% to 100% of the initial stimulation intensity motor threshold MT) can achieve the stimulation of the activity level of the target brain area.

[0085] In addition, regarding the relationship between the preset indicators and the active state of the default mode network (DMN), taking the three preset indicators listed in the previous article, namely the average power spectral density, the frequency band phase synchronization index, and the phase locking value, as an example, an increase in any of these three preset indicators can indicate that the DMN is active. At this time, the subject may be in a state of self-reflection or rest. Otherwise, it means that the DMN is inhibited by external tasks or high-frequency and high-intensity transcranial magnetic stimulation, and this correlation can be gradually refined through continuous learning of the deep reinforcement learning network.

[0086] As an example, looking at steps S101 to S104, the duration of the first cycle includes the first duration, the duration consumed for collecting the EEG signal sequence, the duration consumed for processing to obtain the preset indicators and the stimulation parameters of the next cycle. The second item here is determined by the duration of the signal sequence required to process the preset indicators, such as 1s or 2s as an example in the previous text, and because there is a possibility of removing some data, the actual collected signal duration will be greater than or equal to this duration. The processing of the third item is completed almost instantly by the computing device. Therefore, it can be approximately considered that the duration of the first cycle is equal to the sum of the first duration and the duration of the collected EEG signal sequence, and because the duration of the second item may fluctuate slightly, the duration of the first cycle is not precisely fixed. The value range of the first duration is, for example, 3min to 10min, and specific examples include but are not limited to 4min, 5min, and 7min.

[0087] The above-mentioned steps S101 to S104 represent a formal neural regulation process. In some embodiments, optionally, before executing steps S101 to S104 in a first cycle loop, the brain-computer interface closed-loop neural regulation method according to the exemplary embodiment of the present disclosure also includes: executing the following steps in a second cycle loop until the end duration is reached, wherein the duration of the second cycle is less than the duration of the first cycle, and the end duration is of the same order of magnitude as the duration of the first cycle: controlling the transcranial magnetic stimulation device to run for a second duration according to the set stimulation parameters to stimulate the subject; collecting the EEG signal sequence of the subject in the target brain area within a third duration, wherein the third duration is greater than the second duration; processing the EEG signal sequence to obtain preset indicators; using a deep reinforcement learning network to process the preset indicators to obtain the stimulation parameters of the next cycle, wherein, in the process of executing the above steps in a second cycle loop, the deep reinforcement learning network is initialized and trained using an experience replay mechanism, and the experience data used for training includes the experience data collected in the process of executing the above steps in a second cycle loop. The first cycle is the duration of a regulation cycle in the formal neural regulation. By using a period of time of the same order of magnitude as the first cycle (i.e., a period of time equal to the end length) for high-frequency cyclic regulation (i.e., cyclically regulating with a shorter second cycle) before the formal neural regulation, a large amount of experience data can be collected in a short time, and based on these experience data, the initialization training of the deep reinforcement learning network is performed for the current subject, which helps to further improve the stimulation accuracy of personalized transcranial magnetic stimulation. It should be understood that the stimulation parameters finally obtained in this stage are used as the stimulation parameters when step S101 is first executed. It should also be understood that for the embodiment of first determining the initial stimulation intensity motion threshold of the subject introduced above, the initialization training introduced here should be performed after the determination is completed. It should also be understood that for the second condition in the end condition of the cyclic execution of steps S101 to S104 introduced above, that is, the total time experienced since the subject is subjected to transcranial magnetic stimulation reaches the preset regulation time, the total time should include the end time introduced here, that is, the initialization training stage needs to be included in the timing range. For example, if the preset control time is 30 minutes and the end time is 5 minutes, the time for looping step S101 to step S104 should not exceed 25 minutes.

[0088] As an example, in the initialization training stage, the duration of the second cycle includes the second duration, the third duration, and the duration consumed by processing to obtain the preset index and the stimulation parameters of the next cycle. Since the processing of the last item is completed almost instantly by the computing device, it can be approximately considered that the duration of the second cycle is equal to the sum of the second duration and the third duration. The value range of the second duration is, for example, 3s to 10s, including but not limited to 4s, 5s, and 7s, and the value range of the third duration is, for example, 5s to 15s, including but not limited to 8s, 10s, and 13s, and the second duration is less than the third duration to collect more EEG signals for deep reinforcement learning network processing and improve training efficiency. The value range of the end duration is, for example, 3min to 10min, including but not limited to 5min and 8min. It should be understood that these operation steps are similar to the operations of steps S101 to S104, and some steps are different in parameters. Therefore, the specific operation method can refer to the introduction of steps S101 to S104 in the previous text, and will not be repeated here.

[0089] Regarding the initialization training, optionally, the experience data collected when the subject previously received transcranial magnetic stimulation is recorded as historical experience data. The experience data used for the initialization training also includes historical experience data; or the deep reinforcement learning network on which the initialization training is based is a deep reinforcement learning network trained using historical experience data. It should be understood that the closed-loop neuroregulation method of the brain-computer interface according to the exemplary embodiment of the present disclosure is used to perform a complete neuroregulation for the subject, but the same subject may need to receive multiple neuroregulations. Therefore, the historical experience data accumulated in the previous neuroregulation can be used as a reference for the subsequent neuroregulation of the same subject. By adding historical experience data to the experience data used for the initial training, or directly using the deep reinforcement learning network previously trained and used by the same subject and continuing training, the accuracy of the deep reinforcement learning network can be improved with the help of historical experience.

[0090] Optionally, the brain-computer interface closed-loop neural regulation method according to the exemplary embodiment of the present disclosure further includes: comparing the preset indicator with the preset value range to obtain regulation evaluation information, wherein the preset value range is obtained by counting the values ​​of the preset indicators of the target population, and the regulation evaluation information is used to represent the evaluation of the neural regulation effect. By selecting the target population to perform value statistics of the preset indicators, a cross-subject database can be constructed to obtain a general health standard, and provide a reference for evaluating the brain state of the subject and the neural regulation effect.

[0091] As an example, the target population is a group of people who do not need neuromodulation through transcranial magnetic stimulation. In other words, if transcranial magnetic stimulation is regarded as a treatment method for treating certain diseases, then the target population is a group of people who do not have the disease and can be used as a treatment target to provide a reference for evaluating the treatment effect. In addition, the target population can further include people of different age groups, such as children, teenagers, adults, and the elderly, and the gender ratio in the target population can be reasonably controlled according to statistical needs, and the present disclosure does not limit this.

[0092] As an example, regarding the timing of the evaluation, it can be evaluated centrally after the entire neural regulation is completed, or it can be evaluated at the end of each first cycle, or it can be evaluated at other reasonable times, and the present disclosure does not limit this. When evaluating, the time can be used as the horizontal axis and the preset indicator can be used as the vertical axis to draw a change curve of the preset indicator being evaluated, and a reference line can be drawn in the change curve according to the preset value range to intuitively display the change of the preset indicator relative to the preset value range.

[0093] Optionally, in the process of initializing the training of the deep reinforcement learning network, the instant reward of the deep reinforcement learning is determined according to the preset index and the preset value range, wherein the preset value range is obtained by counting the values ​​of the preset index of the target population. By determining the instant reward according to the preset index and the preset value range, specifically, the closer the preset index is to the preset value range, the larger the instant reward is, and the maximum is reached when the preset index enters the preset value range, which can guide the deep reinforcement learning network to learn how to adjust the stimulation parameters so that the preset index is as close as possible to or even enters the preset value range, providing a clear basis for the decision-making of the deep reinforcement learning network. In addition, such an instant reward can also determine reasonable stimulation parameters, so that when the preset index is too high, it is adjusted down, and when the preset index is too low, it is adjusted up, which can achieve two-way regulation of the state of the target brain area, for example, when the preset index indicates that the target brain area is too active, the appropriate inhibition of the target brain area is achieved by enhancing the stimulation parameters, and when the preset index indicates that the target brain area is not active enough, the appropriate activation of the target brain area is achieved by weakening the stimulation parameters, which helps to achieve a reasonable neural regulation effect.

[0094] Figure 2 is a schematic diagram showing the control process of the brain-computer interface closed-loop neural control method according to an exemplary embodiment of the present disclosure.

[0095] Reference Figure 2 The whole process includes the initialization stage and the formal control stage. In the initialization stage, the cyclic high-frequency stimulation of the end time (each cycle includes the second-time transcranial magnetic stimulation, the third-time electroencephalogram signal collection, and the short-time to negligible signal analysis) is used. Figure 2This analysis part is not shown in the figure) to complete the initial training of the deep reinforcement learning network for the subjects and determine the initial stimulation scheme. In the following formal regulation phase, the first duration of transcranial magnetic stimulation, short-term EEG signal collection, and short-term to negligible signal analysis are performed in a loop ( Figure 2 This analysis part is also not shown in the figure) until the end condition is met.

[0096] Figure 3 is a block diagram showing a brain-computer interface closed-loop neural regulation device according to an exemplary embodiment of the present disclosure.

[0097] Reference Figure 3 The brain-computer interface closed-loop neural regulation device 300 includes a first circulation unit 301, a first control unit 302, a first acquisition unit 303, a first processing unit 304, and a first regulation unit 305.

[0098] The first cycle unit 301 may cyclically operate the first control unit 302 , the first acquisition unit 303 , the first processing unit 304 , and the first regulation unit 305 according to the first period until an end condition is met.

[0099] The first control unit 302 can control the transcranial magnetic stimulation device to run for a first duration according to the set stimulation parameters to stimulate the subject.

[0100] The first acquisition unit 303 may acquire an EEG signal sequence of the subject in the target brain area.

[0101] Optionally, the first acquisition unit 303 can also collect EEG signal sequences of multiple channels of the subject, wherein the multiple signal acquisition points corresponding to the multiple channels include multiple points of the target brain area, the target brain area includes a default mode network, and the multiple points of the target brain area include at least one of the following: the midpoint of the vertex, the midpoint of the occipital, the central vertex, the left vertex, and the right vertex.

[0102] The first processing unit 304 may process the EEG signal sequence to obtain a preset index, wherein the preset index is used to reflect the activity of the target brain region.

[0103] Optionally, the first processing unit 304 may further: perform compression preprocessing on the electroencephalogram signal sequence to obtain a compressed electroencephalogram signal sequence; and process the compressed electroencephalogram signal sequence to obtain a preset indicator.

[0104] Optionally, the first processing unit 304 may also: for the EEG signal sequence of each channel in the multiple channels, sample at a preset sampling frequency to obtain multiple signal subsequences; for each signal subsequence of each channel, retain the signals less than the preset frequency, and compress the signals greater than or equal to the preset frequency to obtain each compressed signal subsequence of each channel; for each compressed signal subsequence of each channel, determine the correlation measure between the compressed signal subsequence and the compressed signal subsequence of the same time sequence in an adjacent channel; from all compressed signal subsequences, remove the compressed signal subsequences whose correlation measures are less than a correlation threshold to obtain a compressed EEG signal sequence.

[0105] The first control unit 305 can use a deep reinforcement learning network to process the preset indicators to obtain the stimulation parameters of the next cycle.

[0106] Optionally, the preset indicator includes at least one of the following: average power spectrum density, frequency band phase synchronization index, and phase locking value.

[0107] Optionally, the brain-computer interface closed-loop neural regulation device 300 also includes a second cycle unit, a second control unit, a second acquisition unit, a second processing unit, and a second control unit (all not shown in the figure). The second cycle unit may, before running the first cycle unit, cyclically run the following units according to the second cycle until the end duration is reached, wherein the duration of the second cycle is less than the duration of the first cycle, and the end duration is of the same order of magnitude as the duration of the first cycle; the second control unit may control the transcranial magnetic stimulation device to run the second duration according to the set stimulation parameters to stimulate the subject; the second acquisition unit may collect the EEG signal sequence of the subject in the target brain area within the third duration, wherein the third duration is greater than the second duration; the second processing unit may process the EEG signal sequence to obtain a preset indicator; the second control unit may use a deep reinforcement learning network to process the preset indicator to obtain the stimulation parameters of the next cycle, wherein, in the process of cyclically executing the above steps according to the second cycle, the deep reinforcement learning network is initialized and trained using the experience playback mechanism, and the experience data used for training includes the experience data collected in the process of cyclically executing the above steps according to the second cycle.

[0108] It should be understood that there are similar parts in the operations performed by the second circulation unit and the first circulation unit 301, and such similar operation parts can refer to the first circulation unit 301; similarly, the second control unit, the second acquisition unit, the second processing unit, and the second regulation unit respectively have similar operations with the first control unit 302, the first acquisition unit 303, the first processing unit 304, and the first regulation unit 305, which can be referred to and will not be described one by one. In addition, the second processing unit and the first processing unit 304 can be the same unit or different units, and the present disclosure does not limit this, and the second regulation unit and the first regulation unit 305 can also be the same unit or different units, and the present disclosure does not limit this either.

[0109] Optionally, the experience data collected when the subject previously received transcranial magnetic stimulation is recorded as historical experience data, wherein the experience data used for training also includes historical experience data; or wherein the deep reinforcement learning network based on which the initialization training is based is a deep reinforcement learning network trained using historical experience data.

[0110] Optionally, the brain-computer interface closed-loop neural regulation device 300 also includes an evaluation unit (not shown in the figure), which can compare the preset indicators with the preset value range to obtain regulation evaluation information, wherein the preset value range is obtained by counting the values ​​of the preset indicators of the target population, and the regulation evaluation information is used to represent the evaluation of the neural regulation effect.

[0111] Optionally, during the initialization training of the deep reinforcement learning network, the immediate reward of the deep reinforcement learning is determined according to a preset indicator and a preset value range, wherein the preset value range is obtained by statistically analyzing the values ​​of the preset indicators of the target population.

[0112] The above has been referred to Figures 1 to 3 A brain-computer interface closed-loop neural regulation method and apparatus according to exemplary embodiments of the present disclosure are described.

[0113] Figure 3 Each unit in the brain-computer interface closed-loop neural control device shown can be configured as software, hardware, firmware, or any combination of the above items to perform specific functions. For example, each unit may correspond to a dedicated integrated circuit, or to a pure software code, or to a module that combines software and hardware. In addition, one or more functions implemented by each unit may also be uniformly executed by components in a physical entity device (e.g., a processor, a client, or a server, etc.).

[0114] In addition, refer to Figure 1The described brain-computer interface closed-loop neural regulation method can be implemented by a program (or instruction) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions can be provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to execute the brain-computer interface closed-loop neural regulation method according to the present disclosure.

[0115] The computer program in the computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. It should be noted that the computer program can also be used to perform additional steps in addition to the above steps or perform more specific processing when performing the above steps. The contents of these additional steps and further processing have been described in reference to Figure 1 It is mentioned in the description of the related method, so it will not be repeated here to avoid repetition.

[0116] It should be noted that each unit in the brain-computer interface closed-loop neural regulation device according to the exemplary embodiment of the present disclosure can completely rely on the operation of the computer program to realize the corresponding function, that is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (for example, lib library) to realize the corresponding function.

[0117] on the other hand, Figure 3 The various units shown may also be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented by software, firmware, middleware, or microcode, the program code or code segment for performing the corresponding operation may be stored in a computer-readable medium such as a storage medium, so that the processor may perform the corresponding operation by reading and running the corresponding program code or code segment.

[0118] For example, an exemplary embodiment of the present disclosure may also be implemented as a computing device, which includes a storage component and a processor, wherein a set of computer executable instructions is stored in the storage component. When the set of computer executable instructions is executed by the processor, the brain-computer interface closed-loop neural regulation method according to the exemplary embodiment of the present disclosure is executed.

[0119] Specifically, the computing device can be deployed in a server or client, or can be deployed on a node device in a distributed network environment. In addition, the computing device can be a PC, a tablet device, a personal digital assistant, a smart phone, a web application, or other device capable of executing the above instruction set.

[0120] Here, the computing device is not necessarily a single computing device, but may also be any device or circuit collection that can execute the above instructions (or instruction sets) individually or jointly. The computing device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0121] In a computing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0122] According to the exemplary embodiment of the present disclosure, some operations described in the brain-computer interface closed-loop neural regulation method can be implemented by software, some operations can be implemented by hardware, and in addition, these operations can also be implemented by a combination of software and hardware.

[0123] The processor may execute instructions or codes stored in one of the storage components, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0124] The storage component may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor, etc. In addition, the storage component may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The storage component and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the storage component.

[0125] In addition, the computing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the computing device may be connected to each other via a bus and / or a network.

[0126] The brain-computer interface closed-loop neural regulation method according to the exemplary embodiment of the present disclosure can be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams can be equally integrated into a single logical device or operated according to non-exact boundaries.

[0127] Therefore, refer to Figure 1 The described brain-computer interface closed-loop neural regulation method can be implemented by a system including at least one computing device and at least one storage device storing instructions.

[0128] According to an exemplary embodiment of the present disclosure, at least one computing device is a computing device for executing a brain-computer interface closed-loop neural control method according to an exemplary embodiment of the present disclosure, and a computer executable instruction set is stored in the storage device. When the computer executable instruction set is executed by the at least one computing device, the execution reference Figure 1 The described brain-computer interface closed-loop neural control method.

[0129] Figure 4 is a schematic diagram showing a brain-computer interface closed-loop neural regulation system according to an exemplary embodiment of the present disclosure.

[0130] Optionally, refer to Figure 4 The at least one computing device in the system includes: a first computing device 41, which is used to control the operation of the transcranial magnetic stimulation device 42 and the electroencephalogram signal acquisition device 43, and can be used to execute Figure 1 Step S101 in the embodiment can specifically turn on, turn off, and perform parameter control on the transcranial magnetic stimulation device 42 and the electroencephalogram signal acquisition device 43; the second computing device 44 is used to process the electroencephalogram signal sequence acquired by the electroencephalogram signal acquisition device 43 to obtain a preset index, which can be used to perform Figure 1 Steps S102 and S103 in the above method can specifically receive the EEG signal collected by the EEG signal collection device 43 in real time and perform synchronous analysis; the third computing device 45 is used to use the deep reinforcement learning network to process the preset index obtained by the second computing device 44 to obtain the stimulation parameters of the next cycle, which can be used to perform Figure 1 Step S104 in the process and determining whether the end condition is met may specifically determine the initialized stimulation parameters and continue to generate iterative stimulation parameters, and may also determine whether to continue regulation or stop regulation, and transmit relevant instructions to the first computing device 41 to achieve closed-loop neural regulation.

[0131] Optionally, refer to Figure 4 , the system also includes a transcranial magnetic stimulation device 42 and an electroencephalogram signal acquisition device 43 connected between the first computing device 41 and the second computing device 44 to provide a completely connected system. As an example, the transcranial magnetic stimulation device 42 and the electroencephalogram signal acquisition device 43 can be worn by the subject and fixed in an immovable position. As an example, the transcranial magnetic stimulation device 42 uses a fixed coil, the stimulation frequency range is 0.01Hz to 50Hz, the stimulation intensity is 1.5Tesla to 6Tesla, and both parameters are continuously adjustable; the number of channels of the electroencephalogram signal acquisition device 43 is greater than 8, and covers five key channels of the PCC brain area (Pz, POz) and the Precuneus brain area (CPz, P3, P4).

[0132] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, comprising instructions, which, when executed by at least one computing device, prompt the at least one computing device to execute the brain-computer interface closed-loop neural regulation method according to the present disclosure.

[0133] The above describes various exemplary embodiments of the present disclosure, and it should be understood that the above description is only exemplary and not exhaustive, and the present disclosure is not limited to the disclosed exemplary embodiments. Without departing from the scope and spirit of the present disclosure, many modifications and changes are obvious to those of ordinary skill in the art. Therefore, the scope of protection of the present disclosure should be based on the scope of the claims.

Claims

1. A closed-loop neural control method for a brain-computer interface, characterized in that: include: The following steps are executed in a loop according to the first cycle until the end condition is met: Controlling the transcranial magnetic stimulation device to operate for a first duration according to the set stimulation parameters to stimulate the subject; Collecting an electroencephalogram signal sequence of the subject in the target brain area; Processing the electroencephalogram signal sequence to obtain a preset index, wherein the preset index is used to reflect the activity of the target brain area; The preset indicators are processed using a deep reinforcement learning network to obtain stimulation parameters for the next cycle.

2. The closed-loop neural control method of the brain-computer interface according to claim 1, characterized in that: The collecting of the electroencephalogram signal sequence of the subject in the target brain area includes: Collect electroencephalogram signal sequences of multiple channels of the subject, wherein the multiple signal collection points corresponding to the multiple channels include multiple points of the target brain area, the target brain area includes a default mode network, and the multiple points of the target brain area include at least one of the following: the midpoint of the vertex, the midpoint of the occipital, the central vertex, the left vertex, and the right vertex.

3. The closed-loop neural control method of the brain-computer interface as claimed in claim 2, characterized in that: The step of processing the electroencephalogram signal sequence to obtain a preset indicator includes: Performing compression preprocessing on the electroencephalogram signal sequence to obtain a compressed electroencephalogram signal sequence; The compressed electroencephalogram signal sequence is processed to obtain a preset index, wherein the preset index includes at least one of the following: average power spectrum density, frequency band phase synchronization index, and phase lock value.

4. The closed-loop neural control method of the brain-computer interface as claimed in claim 3, characterized in that: The step of performing compression preprocessing on the electroencephalogram signal sequence to obtain a compressed electroencephalogram signal sequence includes: For each electroencephalogram signal sequence of the multiple channels, sampling is performed at a preset sampling frequency to obtain multiple signal subsequences; For each signal subsequence of each channel, retain signals with a frequency less than a preset frequency, and compress signals with a frequency greater than or equal to the preset frequency, to obtain each compressed signal subsequence of each channel; For each compressed signal subsequence of each channel, determining a correlation measure between the compressed signal subsequence and a compressed signal subsequence of the same time sequence in an adjacent channel; From all compressed signal subsequences, compressed signal subsequences whose correlation metrics are less than a correlation threshold are removed to obtain the compressed electroencephalogram signal sequence.

5. The closed-loop neural control method of a brain-computer interface according to any one of claims 1 to 4, characterized in that: The following steps are executed in a loop according to the first cycle until the end condition is met, further comprising: The following steps are performed in a loop according to the second cycle until the end duration is reached, wherein the duration of the second cycle is shorter than the duration of the first cycle, and the end duration has the same order of magnitude as the duration of the first cycle: Controlling the transcranial magnetic stimulation device to operate for a second time period according to the set stimulation parameters to stimulate the subject; Collecting an electroencephalogram signal sequence of the subject in the target brain area within a third time period, wherein the third time period is greater than the second time period; Processing the electroencephalogram signal sequence to obtain the preset index; Using the deep reinforcement learning network to process the preset indicators, obtaining the stimulation parameters for the next cycle, In which, in the process of executing the above steps in a loop according to the second cycle, the deep reinforcement learning network is initialized and trained using an experience replay mechanism, and the experience data used for training includes the experience data collected in the process of executing the above steps in a loop according to the second cycle.

6. The closed-loop neural control method of the brain-computer interface as claimed in claim 5, characterized in that: The experience data collected from the subject during previous transcranial magnetic stimulation is recorded as historical experience data. Wherein, the experience data used in the training also includes the historical experience data; or Among them, the deep reinforcement learning network on which the initialization training is based is a deep reinforcement learning network trained using the historical experience data.

7. The closed-loop neural control method of the brain-computer interface as claimed in claim 5, characterized in that: The brain-computer interface closed-loop neural regulation method further includes: comparing the preset indicator with a preset value range to obtain regulation evaluation information, wherein the preset value range is obtained by counting the values ​​of the preset indicator of the target population, and the regulation evaluation information is used to represent the evaluation of the neural regulation effect; and / or During the initialization training of the deep reinforcement learning network, the immediate reward of deep reinforcement learning is determined according to the preset indicator and the preset value range, wherein the preset value range is obtained by counting the values ​​of the preset indicator of the target population.

8. A computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by at least one computing device, the at least one computing device is prompted to execute the brain-computer interface closed-loop neural control method as described in any one of claims 1 to 7.

9. A system comprising at least one computing device and at least one storage device storing instructions, characterized in that: When the instructions are executed by the at least one computing device, they prompt the at least one computing device to execute the brain-computer interface closed-loop neural regulation method as described in any one of claims 1 to 7.

10. A computer program product comprising instructions, characterized in that When the instructions are executed by at least one computing device, they prompt the at least one computing device to execute the brain-computer interface closed-loop neural regulation method as described in any one of claims 1 to 7.

Images