A wearable brain-magnetic brain-computer interface system

Abstract

This specification discloses a wearable EEG brain-computer interface system that can perform high-precision source imaging of EEG signals, improve the spatial resolution of the measurement space decoding method, and achieve accurate identification and classification of EEG signals. The system includes a wearable EEG acquisition device, a signal analysis and processing system, and a multimodal stimulation presentation device. The wearable EEG acquisition device is used to acquire EEG signals generated by the subject's brain nerve activity and send the EEG signals to the signal analysis and processing system; the signal analysis and processing system is used to pre-process, image, and decode the EEG signals to generate decoding information, and send the decoding information to the multimodal stimulation presentation device; the multimodal stimulation presentation device is used to present stimulation information that induces brain nerve activity and present the decoding information generated by the signal analysis and processing system.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface technology, and in particular to a wearable magnetic brain-computer interface system. Background Art

[0002] The Brain Computer Interface (BCI) system is based on a direct information exchange channel that is independent of the human peripheral nervous system and tissues, and can connect the human brain and computers or other electronic devices. It is widely used in medical equipment, sports and entertainment, industrial processes, aerospace and other fields. The measurement of brain nerve activity signals is the basis of BCI, and the accurate detection of brain nerve activities with high complexity and dynamic characteristics is the prerequisite for achieving efficient human-computer interaction. The wearable magnetoencephalography (wMEG) measurement method based on wearable measuring devices can cover the entire brain non-invasively, has the unique advantage of being wearable and ready to use, and can also be flexibly used in mobile applications. The development of non-invasive, instant, and high-precision brain-computer interface technology has brought new opportunities.

[0003] In wMEG-based BCI systems, some related technologies remain unresolved, such as the low sensitivity of wearable EEG signal detection, the inaccurate spatial resolution of wMEG-based brain functional imaging, the slightly insufficient encoding and decoding accuracy and efficiency, and the magnetic shielding environment required for wMEG applications, which also hinders practical application and large-scale promotion. Currently, no mature and complete brain-computer interface system solution and usage method based on wearable EEG has been proposed. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a wearable EEG brain-computer interface system that can perform high-precision source imaging of EEG signals, improve the spatial resolution of the measurement space decoding method, and achieve accurate identification and classification of EEG signals.

[0005] The embodiment of this specification provides a wearable brain-magnetic brain-computer interface system, which includes a wearable brain-magnetic acquisition device, a signal analysis and processing system, and a multimodal stimulation presentation device;

[0006] The wearable brain magnetic acquisition device is used to collect brain magnetic signals generated by the subject's brain nerve activity and send the brain magnetic signals to the signal analysis and processing system;

[0007] The signal analysis and processing system is used to preprocess, image and decode the magnetic brain signals to generate decoded information, and send the decoded information to the multimodal stimulation presentation device;

[0008] The multimodal stimulation presentation device is used to present stimulation information that induces brain nerve activity and present decoding information generated by the signal analysis and processing system.

[0009] Optionally, the wearable brain magnetic field acquisition device includes a plurality of brain magnetic field sensors and supporting devices for acquiring magnetic signals generated by brain nerve activities;

[0010] Each of the magnetoencephalogram (MEG) sensors is configured to provide at least one signal channel, and a plurality of the MEG sensors collect multi-channel signals to form the MEG signal;

[0011] The supporting devices include a digital signal processing acquisition card, front-end / back-end amplifiers and a signal transmission controller.

[0012] Optionally, the signal analysis and processing system includes a preprocessing module, a global feature extraction module, a detail feature extraction module and a decoding module;

[0013] The preprocessing module is used to preprocess the electroencephalogram signal to generate an electroencephalogram measurement signal;

[0014] The global feature extraction module is used to extract the main rhythm feature signal from the magnetoencephalography measurement signal, and determine the global feature information of the magnetoencephalography measurement signal in the measurement space based on the main rhythm feature signal;

[0015] The detail feature extraction module is used to convert the MEG measurement signal into a MEG source signal, and extract the time series signal corresponding to the MEG source signal in the key area as the detail feature information of the MEG measurement signal in the source space;

[0016] The decoding module is used to extract the global feature information and the implicit feature information of the detailed feature information and fuse them to generate fused feature information, and classify and identify the brain magnetic signal based on the fused feature information.

[0017] Optionally, the method in which the preprocessing module preprocesses the electroencephalogram signal to generate an electroencephalogram measurement signal includes:

[0018] Decomposing the magnetic brain signal to generate a plurality of corresponding decomposition signals, and performing signal analysis on the plurality of decomposition signals to determine a plurality of analytical components corresponding to the plurality of decomposition signals;

[0019] Identifying and classifying the plurality of analytical components, and determining brain magnetic source components and artifact source components among the plurality of analytical components;

[0020] The artifact source components are eliminated, the brain magnetism source components are retained, and the inverse operation of signal analysis is performed to restore the multiple brain magnetism source components into pure decomposition signals, and the pure decomposition signals restored by the inverse operation are reconstructed to generate the brain magnetism measurement signal.

[0021] Optionally, the detail feature extraction module includes a source imaging unit, and the source imaging unit is used to convert the brain magnetometry measurement signal into a brain magnetometry source signal in a brain function imaging source space;

[0022] The method for the source imaging unit to map the electroencephalogram measurement signal into the electroencephalogram source signal includes:

[0023] Generating positions of a plurality of brain magnetic sensors in the wearable brain magnetic acquisition device on a surface of a brain structure;

[0024] Performing forward problem modeling to determine a conduction matrix between the source space and the measurement space;

[0025] An inverse problem is solved to determine the neuron activation position and activation range corresponding to the magnetoencephalography measurement signal in the source space.

[0026] Optionally, the method in which the source imaging unit generates the positions of the plurality of brain magnetic sensors in the wearable brain magnetic collection device on the surface of the brain structure includes:

[0027] determining a brain structural model, and determining a plurality of reference points in the brain structural model;

[0028] Adjusting the wearable EEG acquisition device so that positions of the plurality of EEG sensors in the wearable EEG acquisition device are consistent with positions of the plurality of reference points in the brain structure model;

[0029] Wherein, a plurality of reference points are determined in the brain structure model, including:

[0030] determining a plurality of first landmarks on the brain structure model;

[0031] generating a plurality of first marking lines based on the plurality of first marking points, and determining a plurality of first reference points in the first marking lines;

[0032] Selecting a plurality of second marking points from the plurality of first reference points, generating a plurality of second marking lines based on the plurality of second marking points, and determining a plurality of second reference points in the second marking lines;

[0033] The plurality of reference points include a plurality of first reference points and a plurality of second reference points.

[0034] Optionally, the source imaging unit performs forward problem modeling to determine a conduction matrix between the source space and the measurement space, including:

[0035] Determining a brain magnetism mathematical equation, and determining the magnetic potential at the location of the brain magnetism sensor based on the brain magnetism mathematical equation;

[0036] determining a magnetic flux corresponding to the coverage area of the brain magnetic sensor according to the magnetic potential;

[0037] Performing grid discretization processing on the source space to determine a discretization equation corresponding to the magnetic flux;

[0038] The conduction matrix is determined based on the discretized equation.

[0039] Optionally, the source imaging unit solves an inverse problem to determine the neuron activation position and activation range corresponding to the magnetoencephalography measurement signal in the source space, including:

[0040] Constructing an activity state equation and an activity measurement equation of a population of intracranial neurons, and performing spatial domain solutions based on the activity state equation and the activity measurement equation;

[0041] Determine the state prediction set estimate at the current moment according to the activity state equation, and determine the state output compatible set estimate at the next moment according to the activity measurement equation;

[0042] The state prediction set estimation and the state output compatible set estimation are updated in time and space to perform a time series fusion solution.

[0043] Optionally, the brain magnetic sensor uses a small and extremely weak magnetic sensor such as an atomic magnetometer.

[0044] Optionally, the system further includes a magnetic shielding device for providing a necessary shielding environment for brain magnetic field acquisition.

[0045] As can be seen from the above, the wearable brain-magnetic brain-computer interface system provided in the embodiments of this specification has the following beneficial technical effects:

[0046] (1) The wearable brain-magnetic brain-computer interface system can collect the brain magnetic signals of the subject in a whole-brain, non-invasive, comfortable and real-time manner, and pre-process, image and decode the collected brain magnetic signals, thereby achieving high-efficiency and high-precision recognition and classification of brain magnetic signals, and promoting the development of non-invasive, instant, efficient and high-precision brain-computer interface systems.

[0047] (2) In the wearable EEG brain-computer interface system, the signal analysis and processing system generates EEG measurement signals with fewer artifacts and higher quality by pre-processing the EEG signals, extracts the global feature information of the EEG measurement signals in the measurement space, extracts the detail feature information of the EEG measurement signals in the source space, and achieves classification and identification of the EEG signals by fusing the global feature information with the detail feature information. This approach will, on the one hand, reduce the artifacts of the EEG signals measured in the measurement space and purify the EEG signals, and on the other hand, fuse the global feature information of the measurement space with the detail feature information of the brain function imaging source space, thereby overcoming the non-real and non-global limitations of the brain function imaging source space decoding method, improving the spatial resolution of the measurement space decoding method, and achieving high-precision identification and classification of EEG signals.

[0048] (3) The preprocessing module of the signal analysis and processing system in the wearable EEG brain-computer interface system performs artifact removal preprocessing on the EEG signal. By performing intrinsic mode signal decomposition on the EEG signal to determine multiple analytical components, the multiple analytical components are classified and identified to eliminate the artifact source components in the multiple analytical components, and then the signal is restored and reconstructed to generate a wearable EEG signal that does not contain artifact components. This method can remove the complex artifact effects in the wearable EEG brain-computer interface system with high quality, thereby ensuring the accuracy and robustness of the subsequent classification and identification of the EEG signal.

[0049] (4) In the wearable EEG brain-computer interface system, the detail feature extraction module performs brain function source imaging mapping on the EEG measurement signal to generate an EEG source signal. The positions of the wearable EEG acquisition sensors are generated through the brain structure space to ensure the accurate correspondence between the positions of multiple EEG sensors and the brain structure space. On this basis, forward problem modeling and inverse problem solving are performed to achieve high spatiotemporal resolution brain function imaging for the EEG measurement signal, providing high spatiotemporal resolution brain function activity signals for brain-computer interface decoding.

[0050] (5) In the wearable EEG brain-computer interface system, the detail feature extraction module performs brain function source imaging mapping on the EEG measurement signal, and directly determines the positions of multiple EEG sensors in the wearable EEG acquisition device on the surface of the brain structure, which can avoid the problems of traditional registration with many operation links, low operation efficiency and greater errors to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0052] Figure 1A schematic structural diagram of a wearable magnetic brain-computer interface system provided by one or more optional embodiments of this specification is shown;

[0053] Figure 2 A schematic structural diagram of a signal analysis and processing system in a wearable magnetic brain-computer interface system provided by one or more optional embodiments of this specification is shown;

[0054] Figure 3 A schematic diagram showing a method for preprocessing, imaging, and decoding brain magnetic signals by a signal analysis and processing system in a wearable brain magnetic brain-computer interface system provided by one or more optional embodiments of this specification is shown;

[0055] Figure 4 A schematic diagram showing a method for preprocessing brain magnetic signals by a preprocessing module in a wearable brain magnetic brain-computer interface system provided by one or more optional embodiments of this specification is shown;

[0056] Figure 5 A schematic diagram showing a method for mapping the magnetoencephalogram measurement signal into the magnetoencephalogram source signal by a source imaging unit in a wearable magnetoencephalogram brain-computer interface system provided by one or more optional embodiments of this specification is shown;

[0057] Figure 6 A schematic diagram of a method for determining the positions of multiple brain magnetic sensors on the surface of a brain structure in a wearable brain magnetic brain-computer interface system provided by one or more optional embodiments of this specification is shown;

[0058] Figure 7 A schematic diagram of a method for performing forward problem modeling in a wearable magnetic brain-computer interface system provided by one or more optional embodiments of this specification is shown;

[0059] Figure 8 A schematic diagram of a method for solving an inverse problem in a wearable magnetic brain-computer interface system provided by one or more optional embodiments of this specification is shown. DETAILED DESCRIPTION

[0060] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] The Brain Computer Interface (BCI) system is based on a direct information interaction channel that does not rely on the human peripheral nervous system and tissues. It can connect the human brain and computers or other electronic devices. It is widely used in medical equipment, cultural and sports entertainment, industrial processes, aerospace and other fields.

[0062] The measurement of brain neural activity signals is the foundation of BCI, and the precise detection of brain neural activities with high complexity and dynamic characteristics is the prerequisite for achieving efficient human-computer interaction. Currently, the most commonly used methods for measuring brain neural activity signals in BCI include implantable EEG measurement and scalp EEG measurement. The implantable EEG measurement method has advantages in obtaining the quality of local neural signals, but has not yet been overcome in terms of global signal acquisition, invasiveness, and long-term safety of devices in the body. In order to obtain higher-quality signals, the scalp EEG measurement method requires the application of conductive glue or saline before use, which restricts its efficiency. Compared with the above two measurement methods, the wearable magnetoencephalography (wMEG) measurement method based on wearable measurement devices can cover the entire brain non-invasively, has the unique advantage of being wearable and ready to use, and can also be flexibly used in mobile applications. The development of non-invasive, instant, and high-precision brain-computer interface technology has brought new opportunities.

[0063] In wMEG-based BCI systems, some related technologies remain unresolved. For example, there is a lack of dedicated brain magnetic signal detection equipment for wearable brain-computer interfaces (BCIs). The spatial resolution of brain functional imaging based solely on wMEG without individualized brain structure priors is still at the centimeter level, which cannot meet the needs of accurately locating and monitoring brain region functions and weak activity states. The encoding and decoding technology based solely on wMEG data cannot obtain a high amount of information and signal quality because the number of extracranial acquisition channels is far lower than the number of intracranial neurons. The encoding and decoding accuracy and efficiency are slightly insufficient, limiting practical use. The shielding environment is designed using a combination of active and passive shielding, but the design of large-scale weak shielding environments under the high cost-effectiveness requirements of BCI lacks a precise calculation method for complex coupled magnetic fields, hindering practical application and large-scale promotion. Currently, no mature and complete brain-computer interface system solution and usage method based on wearable BCI has been proposed.

[0064] In response to the above problems, the purpose of the embodiments of this specification is to propose a wearable magnetoencephalogram (MEG) brain-computer interface system that fuses the global feature information of the measurement space with the detailed feature information of the brain function imaging source space. This approach can overcome the non-real and non-global limitations of the brain function imaging source space decoding method, improve the spatial resolution of the measurement space decoding method, and achieve high-precision recognition and classification of MEG signals.

[0065] Based on the above objectives, the embodiments of this specification provide a wearable brain-magnetic brain-computer interface system.

[0066] like Figure 1As shown, one or more optional embodiments of this specification provide a wearable magnetoencephalography brain-computer interface system 100, which includes a wearable magnetoencephalography acquisition device 102, a signal analysis and processing system 104 and a multimodal stimulation presentation device 106.

[0067] The wearable EEG acquisition device 102 is used to acquire EEG signals generated by the subject's brain nerve activities and send the EEG signals to the signal analysis and processing system 104 .

[0068] In an optional embodiment, the wearable EEG acquisition device 102 includes multiple EEG sensors and supporting components for acquiring magnetic signals generated by brain neural activity. Each EEG sensor provides at least one signal channel, typically one to three channels. The multi-channel signals acquired by the multiple EEG sensors constitute the EEG signal.

[0069] The supporting components include a digital signal processing acquisition card, front-end / back-end amplifiers, and a signal transmission controller. The digital signal processing acquisition card is used to collect magnetic signals acquired by the brain magnetoencephalography sensor. After the front-end / back-end amplifiers process the magnetic signals, the signal transmission controller transmits the magnetic signals to the signal analysis and processing system 104.

[0070] The brain magnetic sensor can be a small, low-magnetic field sensor, such as a high-precision, miniaturized atomic magnetometer. The atomic magnetometer can utilize rapid active magnetic compensation and magnetic noise suppression methods based on intelligent computing to acquire magnetic signals with high precision and sensitivity, while effectively suppressing noise in the magnetic signals.

[0071] The signal analysis and processing system 104 is used to pre-process, image, and decode the MEG signals to generate decoded information, and send the decoded information to the multimodal stimulation presentation device 106 .

[0072] The multimodal stimulation presentation device 106 is used to present stimulation information that induces brain nerve activity and present decoding information generated by the signal analysis and processing system.

[0073] In some optional embodiments, the wearable magnetic brain-computer interface system 100 further includes a magnetic shielding device. The magnetic shielding device is used to shield environmental magnetic fields unrelated to brain neural activity (such as the Earth's magnetic field or interfering magnetic signals from other electronic devices), thereby ensuring the operating stability of the wearable magnetic brain-computer interface system 100 and the effectiveness and accuracy of the brain magnetic signals collected by the wearable magnetic brain acquisition device 102.

[0074] The wearable brain-magnetic brain-computer interface system can collect the subject's brain magnetic signals in a whole-brain, non-invasive, comfortable and real-time manner, and pre-process, image and decode the collected brain magnetic signals, thereby achieving high-efficiency and high-precision recognition and classification of brain magnetic signals, and promoting the development of non-invasive, instant, efficient and high-precision brain-computer interface systems.

[0075] like Figure 2 As shown, in a wearable magnetic brain-computer interface system provided in one or more optional embodiments of this specification, the signal analysis and processing system 104 includes a preprocessing module 1040, a global feature extraction module 1042, a detail feature extraction module 1044 and a decoding module 1046.

[0076] The preprocessing module 1040 is used to preprocess the EEG signal to generate an EEG measurement signal;

[0077] The global feature extraction module 1042 is configured to extract a main rhythm feature signal from the magnetoencephalography measurement signal, and determine global feature information of the magnetoencephalography measurement signal in a measurement space based on the main rhythm feature signal;

[0078] The detail feature extraction module 1044 is used to convert the MEG measurement signal into a MEG source signal, and extract the time series signal corresponding to the MEG source signal in the key area as the detail feature information of the MEG measurement signal in the source space;

[0079] The decoding module 1046 is used to extract the global feature information and the implicit feature information of the detailed feature information and fuse them to generate fused feature information, and classify and identify the electroencephalographic signal based on the fused feature information.

[0080] like Figure 3 As shown, as an optional implementation, the method in which the signal analysis and processing system 104 preprocesses, images, and decodes the electroencephalographic signal to generate decoding information may include the following steps:

[0081] S301: Preprocess the magnetoencephalogram signal to generate a magnetoencephalogram measurement signal.

[0082] S302: Extracting a main rhythm characteristic signal from the magnetoencephalography measurement signal, and determining global characteristic information of the magnetoencephalography measurement signal in a measurement space based on the main rhythm characteristic signal.

[0083] S303: Mapping the magnetoencephalogram measurement signal into a magnetoencephalogram source signal, and extracting the time series signal corresponding to the magnetoencephalogram source signal in the key area as the detailed feature information of the magnetoencephalogram measurement signal in the source space.

[0084] S304: Extracting the global feature information and implicit feature information of the detailed feature information and fusing them to generate fused feature information, and classifying and identifying the magnetic brain signal based on the fused feature information.

[0085] In the wearable EEG brain-computer interface system, the signal analysis and processing system generates a higher-quality EEG measurement signal with fewer artifacts by preprocessing the EEG signal, extracts the global feature information of the EEG measurement signal in the measurement space, extracts the detailed feature information of the EEG measurement signal in the source space, and achieves classification and identification of the EEG signal by fusing the global feature information with the detailed feature information. This approach will, on the one hand, reduce artifacts and purify the EEG signal measured in the measurement space, and on the other hand, fuse the global feature information of the measurement space with the detailed feature information of the brain functional imaging source space, thereby overcoming the non-realistic and non-global limitations of the brain functional imaging source space decoding method, improving the spatial resolution of the measurement space decoding method, and achieving high-precision identification and classification of EEG signals.

[0086] like Figure 4 As shown, in a wearable EEG brain-computer interface system 100 provided in one or more optional embodiments of this specification, the method in which the preprocessing module 1040 preprocesses the EEG signal to generate an EEG measurement signal includes:

[0087] S401: Decomposing the magnetic brain signal to generate a plurality of corresponding decomposition signals, and performing signal analysis on the plurality of decomposition signals to determine a plurality of analytical components corresponding to the plurality of decomposition signals.

[0088] S402: Identify and classify the plurality of analyzed components, and determine brain magnetic source components and artifact source components among the plurality of analyzed components.

[0089] S403: Eliminate the artifact source components, retain the EEG source components, and perform the inverse operation of signal analysis to restore the multiple EEG source components into pure decomposition signals, and reconstruct the pure decomposition signals restored by the inverse operation to generate the EEG measurement signal.

[0090] Specifically, the method for the preprocessing module 1040 to preprocess the electroencephalographic signal may include the following steps:

[0091] Step (1): Determine the input signal. The input signal is the multi-channel EEG signal containing complex artifacts. The number of signal channels of the EEG signal is N.

[0092] Step (2): Signal decomposition. Decompose the N-channel EEG signals to generate corresponding decomposed signals.

[0093] An optional signal decomposition method is to use empirical mode decomposition on the N-channel EEG signal to generate a corresponding Intrinsic Mode Function (IMF) signal as the decomposition signal.

[0094] The method for performing signal decomposition on the EEG signal may also be to perform time-frequency decomposition on the N-channel EEG signal using wavelet transform or the like, and obtain corresponding time-frequency decomposition signals as the decomposed signals.

[0095] Assuming that each channel magnetic signal is decomposed into K IMF signals / time-frequency decomposition signals, a set of IMF / time-frequency decomposition signals is obtained, which contains N*K signals.

[0096] Step (3): Intrinsic mode unmixing. Use blind source separation algorithms such as canonical correlation analysis (CCA) and independent component analysis (ICA) to analyze the components of the signal determined in step (2) and obtain N*K analyzed components.

[0097] Step (4): Component screening. Automatically extract features and classify the multiple analyzed components using methods such as entropy, spectrum, machine learning, and deep learning. Determine multiple EMG source components and multiple artifact source components from the N*K analyzed components. The number of EMG source components is denoted as P, and the number of artifact source components is denoted as Q. The relationship between the two can be expressed as P + Q = N*K.

[0098] Step (5): Artifact removal: Set the columns of the unmixing matrix corresponding to the Q artifact source components to 0, and then use inverse blind source separation to generate the pure IMF / time-frequency decomposition signal corresponding to each signal channel.

[0099] Step (6): Signal reconstruction and output. The pure IMF / time-frequency decomposition signal corresponding to each channel is reconstructed to obtain an N-dimensional artifact-free EEG signal. This achieves high-quality removal of complex artifacts in the wearable EEG brain-computer interface.

[0100] The preprocessing module of the signal analysis and processing system in the wearable EEG brain-computer interface system performs artifact removal preprocessing on the EEG signal. This preprocessing module performs intrinsic mode signal decomposition on the EEG signal to determine multiple analytical components, classifies and identifies the multiple analytical components, and thereby removes artifact source components from the multiple analytical components. Signal restoration and reconstruction are then performed to generate a wearable EEG signal that does not contain artifact components. This method can achieve high-quality removal of complex artifact effects in the wearable EEG brain-computer interface system, thereby ensuring the accuracy and robustness of subsequent EEG signal classification and identification.

[0101] In a wearable magnetoencephalogram (MEB) brain-computer interface system 100 provided in one or more optional embodiments of this specification, the detail feature extraction module 1044 includes a source imaging unit, which is used to map the MEB measurement signal into a MEB source signal in a brain functional imaging source space.

[0102] like Figure 5 As shown, the method in which the source imaging unit maps the electroencephalogram measurement signal to the electroencephalogram source signal includes:

[0103] S501: Determine the positions of multiple brain magnetic sensors in the wearable brain magnetic acquisition device on the surface of the brain structure.

[0104] like Figure 6 As shown, as a specific embodiment, determining the positions of multiple brain magnetic sensors in the wearable brain magnetic collection device on the surface of the brain structure may include the following steps:

[0105] S601: Determine a brain structure model, and determine a plurality of reference points in the brain structure model.

[0106] The brain structure model can be constructed based on the MR / CT scan structure. The brain structure model includes a scalp surface grid structure.

[0107] S602: Adjust the wearable brain magnetoelasticity acquisition device so that the positions of the multiple brain magnetoelasticity sensors in the wearable brain magnetoelasticity acquisition device are consistent with the positions of the multiple reference points in the brain structure model.

[0108] Wherein, a plurality of reference points are determined in the brain structure model, including:

[0109] determining a plurality of first landmarks on the brain structure model;

[0110] generating a plurality of first marking lines based on the plurality of first marking points, and determining a plurality of first reference points in the first marking lines;

[0111] Selecting a plurality of second marking points from the plurality of first reference points, generating a plurality of second marking lines based on the plurality of second marking points, and determining a plurality of second reference points in the second marking lines;

[0112] The plurality of reference points include a plurality of first reference points and a plurality of second reference points.

[0113] The following is an example of setting sensor positions in the magnetoencephalography device based on a 10-20 system (19-channel signal). The steps of determining a plurality of reference points in the brain structure model include:

[0114] Step (a): Determine a plurality of first landmark points on the brain structure model.

[0115] Based on a large number of head structures and the locations of corresponding landmarks for the nasion (N), left ear (L), right ear (R), and occipital process (I), a database of brain structures and their corresponding landmarks is established. An image recognition algorithm is then used to identify and determine multiple first landmarks in the brain structure model. The first landmarks include the nasion (N), left ear (L), right ear (R), and occipital process (I) landmarks.

[0116] Step (b): Generate multiple first marking lines based on the multiple first marking points, and determine multiple first reference points in the first marking lines.

[0117] Using the geodesic method, the shortest curve between the nasion (N) and the occipital process (I) is generated on the brain structure surface mesh as the first longitudinal landmark line NI. The shortest curve between the left ear (L) and the right ear (R) is generated as the first transverse landmark line LR. The landmark lines are composed of points from the head surface structure data of the brain structure model.

[0118] A plurality of first reference points are respectively determined on the first marking line NI and the first marking line LR.

[0119] The intersection point X between the first marking line NI and the first marking line LR may be taken as a first reference point, denoted as a center reference point Cz.

[0120] A plurality of first reference points are respectively determined on the first marking line NI and the first marking line LR.

[0121] The first reference points determined on the first marker line NI include: FPz, Fz, Cz, Pz, and Oz. From the nasion N to the occipital process I, the first reference points include: FPz, Fz, (Cz), Pz, and Oz, in order. Specifically, assuming that the length of NI is d1, the points along the curve NI that are 10*d1 and 30%*d1 away from point N are FPz and Fz, respectively. The points along the curve NI that are 10*d1 and 30%*d1 away from point I are Oz and Pz, respectively.

[0122] The first reference points determined on the first marking line LR include T3, C3, (Cz), C4, and T4. From the left ear L to the right ear R, the first reference points include T3, C3, (Cz), C4, and T4 in order. Specifically, assuming the length of LR is d2, the points along the curve LR that are 10*d2 and 30%*d2 away from point L are T3 and C3, respectively. The points along the curve LR that are 10*d2 and 30%*d2 away from point L are T4 and C4, respectively.

[0123] Step (c): Select multiple second marking points from the multiple first reference points, generate multiple second marking lines based on the multiple second marking points, and determine multiple second reference points in the second marking lines.

[0124] Select N, I and T3 as the second marker points. Based on the header table structure data and the positions of the second marker points N, I and T3, use the geodesic method to calculate the shortest curve FOT3 passing through FPz, Oz and T3, and record it as the second marker line FOT3.

[0125] FPz, Oz and T4 are selected as the second marker points. Based on the positions of the second marker points FPz, Oz and T4, the geodesic method is used to calculate the shortest curve FOT4 passing through FPz, Oz and T4, which is recorded as the second marker line FOT4.

[0126] A plurality of second reference points are respectively determined on the second marking line FOT3 and the second marking line FOT4.

[0127] The second reference points determined on the second marking line FOT3 include Fp1, F7, T5, and O1. Specifically, assuming that the length of FOT3 is d3, the points along the curve that are 10%*d3 and 30%*d3 away from the point FPz are Fp1 and F7, and the points along the curve that are 10%*d3 and 30%*d3 away from the point Oz are T5 and O1.

[0128] The second reference points determined on the second marking line FOT4 include Fp2, F8, T6, and O2. Specifically, assuming that the length of FOT4 is d4, the points along the curve that are 10%*d3 and 30%*d3 away from the point FPz are Fp2 and F8, and the points along the curve that are 10%*d3 and 30%*d3 away from the point Oz are T6 and O2.

[0129] In some optional implementation schemes, after determining the plurality of second reference points, a plurality of supplementary reference points may be further determined.

[0130] Supplementary reference points include F3, P3, F4, and P4. For F3 and P3, based on the header table structure data and the positions of Fp1, C3, and O1, the shortest curve FCP1 passing through these three points is calculated. The point on FCP1 that is equidistant from Fp1 and C3 is set as F3, and the point on FCP1 that is equidistant from O1 and C3 is set as P3. Based on the header table structure data and the positions of Fp2, C4, and O2, a curve FCP2 passing through these three points is calculated. The point on FCP2 that is equidistant from Fp2 and C4 is set as F4, and the point on FCP2 that is equidistant from O2 and C4 is set as P4.

[0131] In some related technologies, mapping MEG signals in the measurement space to source signals in the source space presupposes co-registration between the MEG signal measurement space and the brain structure model space. This requires that the measurement electrode positions be matched to the scalp surface defined in the brain structure model space through some transformation operation. Key registration parameters are typically obtained by measuring some "common" landmarks during the acquisition of the brain structure model data. Most commonly, corresponding electrodes and / or some fiducial markers are placed on the skin. These electrodes and / or fiducial markers require pre-scanning measurements using equipment such as a 3D digitizer. Based on this information, transformation parameters can be determined, and then the MEG signal measurement space model and the brain structure model are registered based on these transformation parameters. This approach is cumbersome, requires a large amount of data processing, and is prone to introducing data errors, which can affect the accuracy of model registration. In contrast, in the embodiments of the present application, the positions of multiple MEG sensors in the wearable MEG acquisition device are determined directly on the brain structure surface, avoiding the problems of traditional registration, which involves multiple steps, low efficiency, and, to a certain extent, greater errors.

[0132] S502: Perform forward problem modeling to determine a conduction matrix between the source space and the measurement space.

[0133] like Figure 7 As shown, as a specific implementation, the method of performing forward problem modeling and determining the conduction matrix between the source space and the measurement space may include the following steps:

[0134] S701: Determine a brain magnetism mathematical equation, and determine the magnetic potential at the location of the brain magnetism sensor based on the brain magnetism mathematical equation.

[0135] Based on the quasi-static Maxwell equations and the Coulomb gauge, The mathematical equation for obtaining brain magnetism is:

[0136]

[0137] in, A is the magnetic potential, B is the magnetic induction intensity; μ is the magnetic permeability constant, j p is the initial current density, σ is the conductivity, and u is the potential.

[0138] S702: Determine the magnetic flux corresponding to the coverage area of the brain magnetoelectric sensor according to the magnetic potential.

[0139] The solution of the above mathematical equation can be expressed as:

[0140]

[0141] Where x represents the position of the MEG sensor and y represents the position of the source.

[0142] Let F be the area covered by the brain magnetic sensor, and The magnetic flux flowing through Y can be calculated by the following formula:

[0143]

[0144] The magnetic flux Ψ measured by the brain magnetic sensor consists of two parts: one is the initial current j p The primary magnetic flux Ψ p , which can be directly obtained from the distance between the source and the sensor electrode; the second part is obtained from the induced current The secondary magnetic flux Ψ generated sec .

[0145] Since the conductivity of different head tissues can only be treated as piecewise constant functions, and the geometric shapes of different tissues are complex, especially brain tissue, the secondary magnetic flux Ψ sec It needs to be obtained through numerical calculation.

[0146] The mathematical equation for the secondary magnetic flux is described as follows:

[0147] definition: Then the secondary magnetic flux Ψ sec It can be expressed as:

[0148]

[0149] The electric potential satisfies the following equation:

[0150]

[0151] S703: Performing grid discretization processing on the source space to determine a discretization equation corresponding to the magnetic flux.

[0152] The measurement domain is discretized using tetrahedral and hexahedral meshes, while the source domain is discretized using surface triangular meshes. The measurement domain is discretized using adaptive meshing. First, an initial fine mesh is created based on image pixels. Second, the mesh is coarsened based on point, edge, surface, and conductivity information. Finally, the stored information for points and volumes is reorganized, and mesh information of the same scale is recorded for subsequent assembly of the overall stiffness matrix to speed up the calculation.

[0153] The discretized measurement and the applied shape function Determine the discretization equation:

[0154]

[0155]

[0156] Where v = [v1, v2, ..., v P ] is the potential of the grid vertices in the measurement domain, and P is the total number of grid vertices;

[0157] Ψ sec =[Ψ sec1 ,Ψ sec2 ,…,Ψ secM ] is the secondary magnetic flux corresponding to the brain magnetic sensor, M is the total number of magnetometers;

[0158] X=[X1,X2,…,X N ] represents N intracranial discharge sources.

[0159] S704: Determine the conduction matrix based on the discretized equation.

[0160] According to the discretization equation:

[0161] Sv=Ψ sec

[0162] Kv=GX

[0163] The simplified conduction matrix is:

[0164] L=SK -1 G

[0165] in, is the secondary magnetic flux matrix, is the finite element stiffness matrix, is the source model matrix and satisfies:

[0166]

[0167]

[0168]

[0169] Among them, K is a large symmetric sparse matrix, K -1 The solution can be obtained by directly solving K T x=S T Obtain, that is, x=(K T ) - 1 S T =K -1 S T .

[0170] Large sparse matrices can be solved using the preconditioned conjugate gradient method, and multi-grid preconditioning can be solved using algorithms such as the conjugate gradient method.

[0171] S503: Solve the inverse problem to determine the neuron activation position and activation range corresponding to the magnetoencephalography measurement signal in the source space.

[0172] like Figure 8 As shown, as a specific embodiment, the method of solving the inverse problem and determining the neuron activation position and activation range corresponding to the magnetoencephalography measurement signal in the source space may include the following steps:

[0173] S801: Constructing an activity state equation and an activity measurement equation of a group of intracranial neurons, and performing spatial domain solutions based on the activity state equation and the activity measurement equation.

[0174] Among them, the state equation of intracranial neural source activity is constructed, including:

[0175] The state equation is constructed based on models such as Jansen's Neural Mass Model, which is commonly used to simulate the discharge of intracranial population neurons:

[0176] x k+1 =f(x k , w k )

[0177] in is the state vector of intracranial neural activity, is the process disturbance ( represents n-dimensional Euclidean space). The initial state value x0, the process disturbance w k is unknown, but belongs to a specific bounded set, that is, x0∈X0, w k ∈W k , where X0, W k is a known set.

[0178] Construct an equation for measuring intracranial neurogenic activity, including:

[0179] Based on the forward problem model and the determined conduction matrix, the measurement equation is constructed:

[0180] y k =g(x k , v k )=Lx k +v k

[0181] Wherein, L is the conductive matrix; v k It represents the error caused by uncertain factors such as conduction matrix calculation error and measurement noise during measurement, and the error is unknown but belongs to a specific bounded set, which is abbreviated as v k ∈V k , where V k It is a findable set.

[0182] S802: Determine a state prediction set estimate at the current moment according to the activity state equation, and determine a state output compatible set estimate at the next moment according to the activity measurement equation.

[0183] Solve by following the steps below:

[0184] Assume that the exact state set X can be obtained at time k k An approximate estimate of and measurement data y k , then the exact state set X can be obtained at time k+1 k+1 An approximate estimate of satisfy The corresponding process can be described as follows:

[0185] State prediction set estimation:

[0186] Known Obtain an approximate estimate of the state prediction set Make

[0187]

[0188] Where S represents the approximate estimation set.

[0189] State output compatible set estimation:

[0190] The measurement data y at time k+1 is known k+1 , find the state output compatible set X yk+1 An estimate of Make It is worth noting that the calculation of this step is subject to the constraint of boundary sparseness, that is,

[0191]

[0192] st(y k+1 -LX yk+1 )∈V k

[0193] Among them, V is the sparse matrix V obtained according to the position distribution of dipoles in the source model, and α is the penalty parameter.

[0194] S803: Performing a time series fusion solution by performing a state spatiotemporal fusion update on the state prediction set estimate and the state output compatible set estimate.

[0195] Find the intersection An approximate outsourcing set of Its satisfaction and That is an estimate at time k+1, thus completing the solution to the inverse problem.

[0196] Through the above solution, not only the accuracy of the solution can be improved, but the solution is a set containing the actual state, which can provide richer information than traditional state estimation.

[0197] In the wearable MEG BCI system, the detail feature extraction module performs brain function source imaging mapping on the MEG measurement signals to generate MEG source signals. The wearable MEG acquisition sensor positions are generated from the brain structure space, ensuring an accurate correspondence between the multiple MEG sensor positions and the brain structure space. Based on this, forward problem modeling and inverse problem solving are performed. This enables high-temporal-resolution brain function imaging of the MEG measurement signals, providing high-temporal-resolution brain function activity signals for BCI decoding.

[0198] It should be noted that the methods of one or more embodiments of this specification can be performed by a single device, such as a computer or server. The methods of this embodiment can also be applied in a distributed scenario, where multiple devices cooperate to perform the method. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the methods of one or more embodiments of this specification, and the multiple devices will interact with each other to complete the method.

[0199] It should be noted that the foregoing description of this specification is based on specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0200] For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing one or more embodiments of this specification, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0201] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). The storage medium can also include a combination of the above-mentioned types of memory.

[0202] The systems described in the above embodiments can be implemented by computer chips or physical devices, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0203] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0204] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0205] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0206] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0207] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0208] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present specification as described above, which are not provided in detail for the sake of simplicity.

[0209] Although the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0210] The one or more embodiments of this specification are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included within the scope of protection of this disclosure.

Claims

1. A wearable brain-magnetic brain-computer interface system, characterized in that: The system includes a wearable brain magnetic acquisition device, a signal analysis and processing system, and a multimodal stimulation presentation device; The wearable brain magnetic acquisition device is used to collect brain magnetic signals generated by the subject's brain nerve activity and send the brain magnetic signals to the signal analysis and processing system; The signal analysis and processing system is used to pre-process, image and decode the magnetic brain signals to generate decoded information, and send the decoded information to the multimodal stimulation presentation device; The multimodal stimulation presentation device is used to present stimulation information that induces brain nerve activity and present decoding information generated by the signal analysis and processing system; The signal analysis and processing system includes a preprocessing module, a global feature extraction module, a detail feature extraction module and a decoding module; The preprocessing module is used to preprocess the electroencephalogram signal to generate an electroencephalogram measurement signal; The global feature extraction module is used to extract the main rhythm feature signal from the magnetoencephalography measurement signal, and determine the global feature information of the magnetoencephalography measurement signal in the measurement space based on the main rhythm feature signal; The detail feature extraction module is used to convert the MEG measurement signal into a MEG source signal, and extract the time series signal corresponding to the MEG source signal in the key area as the detail feature information of the MEG measurement signal in the source space; The decoding module is used to extract the implicit feature information of the global feature information and the detailed feature information and fuse them to generate fused feature information, and classify and identify the electroencephalographic signal based on the fused feature information; The detail feature extraction module includes a source imaging unit, which is used to map the magnetoencephalography measurement signal into a magnetoencephalography source signal in a brain function imaging source space; The method for the source imaging unit to map the electroencephalogram measurement signal into the electroencephalogram source signal includes: Determining positions of a plurality of brain magnetic sensors in the wearable brain magnetic collection device on the surface of the brain structure; Performing forward problem modeling to determine a conduction matrix between the source space and the measurement space; Solving an inverse problem to determine the neuron activation position and activation range corresponding to the magnetoencephalography signal in the source space; The method for the source imaging unit to determine the positions of the plurality of brain magnetic sensors in the wearable brain magnetic collection device on the surface of the brain structure includes: determining a brain structural model, and determining a plurality of reference points in the brain structural model; Adjusting the wearable EEG acquisition device so that positions of the plurality of EEG sensors in the wearable EEG acquisition device are consistent with positions of the plurality of reference points in the brain structure model; Wherein, a plurality of reference points are determined in the brain structure model, including: determining a plurality of first landmarks on the brain structure model; generating a plurality of first marking lines based on the plurality of first marking points, and determining a plurality of first reference points in the first marking lines; Selecting a plurality of second marking points from the plurality of first reference points, generating a plurality of second marking lines based on the plurality of second marking points, and determining a plurality of second reference points in the second marking lines; The plurality of reference points include a plurality of first reference points and a plurality of second reference points.

2. The system according to claim 1, wherein: The wearable brain magnetic field acquisition device includes multiple brain magnetic field sensors and supporting devices for acquiring magnetic signals generated by brain nerve activities; Each of the magnetoencephalogram (MEG) sensors is used to provide at least one signal channel, and a plurality of the MEG sensors collect multi-channel signals to form the MEG signal; The supporting devices include a digital signal processing acquisition card, front-end / back-end amplifiers and a signal transmission controller.

3. The system according to claim 1, wherein: The method for the preprocessing module to preprocess the brain magnetoencephalogram signal to generate a brain magnetoencephalogram measurement signal includes: Decomposing the magnetic brain signal to generate a plurality of corresponding decomposition signals, and performing signal analysis on the plurality of decomposition signals to determine a plurality of analytical components corresponding to the plurality of decomposition signals; Identifying and classifying the plurality of analytical components, and determining brain magnetic source components and artifact source components among the plurality of analytical components; The artifact source components are eliminated, the brain magnetism source components are retained, and the inverse operation of signal analysis is performed to restore the multiple brain magnetism source components into pure decomposition signals, and the pure decomposition signals restored by the inverse operation are reconstructed to generate the brain magnetism measurement signal.

4. The system according to claim 1, wherein: The method of performing forward problem modeling by the source imaging unit to determine the conduction matrix between the source space and the measurement space includes: Determining a brain magnetism mathematical equation, and determining the magnetic potential at the location of the brain magnetism sensor based on the brain magnetism mathematical equation; determining a magnetic flux corresponding to the coverage area of the brain magnetic sensor according to the magnetic potential; Performing grid discretization processing on the source space to determine a discretization equation corresponding to the magnetic flux; The conduction matrix is determined based on the discretized equation.

5. The system according to claim 1, wherein: The source imaging unit solves an inverse problem to determine the neuron activation position and activation range corresponding to the magnetoencephalography measurement signal in the source space, including: Constructing an activity state equation and an activity measurement equation of a population of intracranial neurons, and performing spatial domain solutions based on the activity state equation and the activity measurement equation; Determine the state prediction set estimate at the current moment according to the activity state equation, and determine the state output compatible set estimate at the next moment according to the activity measurement equation; The state prediction set estimation and the state output compatible set estimation are updated in time and space to perform a time series fusion solution.

6. The system according to claim 2, wherein: The brain magnetic sensor adopts a small and extremely weak magnetic sensor such as an atomic magnetometer.

7. The system according to claim 1, wherein: The system also includes a magnetic shielding device for providing a necessary shielding environment for brain magnetic field acquisition.

Images