Electromagnetic photoacoustic synchronization integrated brain function imaging method and device

Abstract

The invention provides an electromagnetic photoacoustic synchronous integrated brain function imaging method and device, and relates to the field of multi-mode brain function imaging research. According to the invention, electroencephalogram, magnetoencephalogram, functional near infrared spectrum and functional ultrasonic imaging technologies are fused, important advantage complementation is formed in signal specificity and temporal-spatial resolution, and neural activity can be described from the angle of human cranial nerve electrophysiology and neural metabolism can be observed from hemodynamic information at the same time. And electrophysiology and hemodynamic activities of the brain can be synchronously recorded.

Description

Technical Field

[0001] The present invention relates to the research field of multimodal brain functional imaging, and specifically to an electromagnetic, photoacoustic, synchronous integrated brain functional imaging method and device that integrates electroencephalogram, magnetoencephalogram, functional near-infrared spectroscopy, and functional ultrasound to synchronously record the electrophysiological and hemodynamic activities of the brain and study neurovascular coupling and metabolic activities. Background Art

[0002] Multimodal brain functional imaging uses complementary signal specificity to analyze human brain neural activity from multiple angles. It is a cutting-edge brain imaging technology that has attracted much attention in recent years. Given that the working state of each brain region in the human brain changes dynamically and rapidly at any time, synchronous multimodal brain functional imaging can capture brain activities that are highly sporadic and random, and is a key foundation for fields such as cognitive neuroscience, brain pathology, and brain-computer interface. The neurovascular coupling effect describes the interconnection between brain neuroelectrophysiological activity and hemodynamic changes. It is one of the most important mechanisms for maintaining brain activity and is related to a variety of brain diseases such as hypertension, stroke, and Alzheimer's disease. The synchronous measurement and analysis of neuroelectrophysiological signals and hemodynamic signals is an important cornerstone for in-depth research on the neurovascular coupling effect, which helps to have a more comprehensive understanding of the laws of brain neural activity, and helps to promote the development of cognitive neuroscience and improve the level of diagnosis and treatment of brain diseases.

[0003] MEG and EEG are non-invasive brain electrophysiological measurement techniques that can directly record brain electrophysiological activity by measuring the magnetic field or electric field generated by brain neuronal currents on the scalp with millisecond time resolution. Although both MEG and EEG can directly measure neural activity in the brain, their sensitivity to neuronal currents varies in different directions and depths. Therefore, synchronously recording and integrating MEG and EEG signals can more accurately portray brain neural activity. In addition, compared with EEG source imaging, MEG source imaging is less affected by the distortion caused by the difference in conductivity of different tissues in the skull, and can more accurately locate brain activity. However, the core part of traditional MEG, the superconducting quantum interference device (SQUID), needs to work in a large cryogenic cooling system, which increases the difficulty of combining MEG with other brain functional imaging modalities. In recent years, the rapid development of optically pumped magnetometers (OPMs) has provided a very effective alternative to SQUID MEG technology. OPM can measure extremely weak magnetic fields in room temperature without the need for a cryogenic cooling system, greatly reducing equipment procurement and maintenance costs, and can achieve more flexible detector array arrangements. Compared to SQUID MEG detectors, OPM MEG detectors can be placed closer to the scalp, significantly improving the signal-to-noise ratio by reducing the distance to intracranial cortical sources. Flexible detector arrangement improves the compatibility of OPM MEG with other imaging modalities. Specifically, OPM detectors can be staggered with other modality sensors on a hat or customized helmets for individual subject head shape and sensor arrangement requirements.

[0004] Functional near-infrared spectroscopy is a non-ionizing, safe, low-cost, portable, and wearable oxygenated hemodynamic neuroimaging technology. Utilizing the neurovascular coupling effect, functional near-infrared spectroscopy indirectly reflects the brain's neural activity by measuring the brain's functional hemodynamic response (such as changes in the concentrations of oxygenated hemoglobin and deoxygenated hemoglobin). Functional near-infrared spectroscopy uses near-infrared light to illuminate the scalp, places the detector a few centimeters away from the light source, detects changes in the intensity of the scalp's emitted light, and calculates the brain's hemoglobin concentration based on different models. At present, magnetoencephalography and electroencephalography have been combined with functional near-infrared spectroscopy to study neurovascular coupling effects, such as exercise-induced neurovascular coupling and the habituation effect of neurovascular coupling, and promote physiological research (such as in the fields of sleep and normal aging) and pathological research (such as in the fields of Alzheimer's disease, hypertension, and stroke) on neurovascular coupling effects.

[0005] Functional ultrasound imaging uses ultrasound to monitor changes in cerebral blood volume and blood flow velocity in real time. It is an indirect neuroimaging method of neuronal activity at high spatiotemporal resolution. In the past decade, the development of new concepts such as ultrafast ultrasound has increased Doppler sensitivity by several orders of magnitude, paving the way for ultrasound functional neuroimaging. Functional ultrasound imaging can image awake and freely moving animals and clinical neuroimaging of human neonates, and can be used for functional connectivity mapping of brain connectomics. 3D ultrafast ultrasound imaging has great potential in 3D mapping of human stiffness, tissue movement and flow, providing new prospects for reducing invasive diagnosis and clinical application of ultrasound. Multimodal brain functional imaging technology has developed rapidly in the past three decades, and disease diagnosis assisted by multimodal data analysis has achieved certain success in clinical practice. However, most of these clinical practices rely on patients to go to multiple departments and multiple imaging instruments. There is still a lack of a four-modality synchronous integrated brain functional imaging device that integrates electroencephalography, magnetoencephalography, functional near-infrared spectroscopy and functional ultrasound imaging in the world. Summary of the invention

[0006] The purpose of this invention is to propose a non-invasive electromagnetic photoacoustic synchronous integrated brain function imaging method and device, which integrates electroencephalogram, magnetoencephalogram, functional near-infrared spectroscopy and functional ultrasound imaging technology, forming important complementary advantages in signal specificity and spatiotemporal resolution, and can simultaneously characterize neural activity from the perspective of human brain neuroelectrophysiology and observe neural metabolism from hemodynamic information. This device is a new type of scientific instrument urgently needed in the field of brain science research and brain disease diagnosis and treatment, which will help explore the neural principles of brain cognition and the diagnosis and treatment of major brain diseases.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] An electromagnetic-photoacoustic synchronous integrated brain function imaging method comprises the following steps:

[0009] 1) Head model construction steps: Based on the subject's MRI or CT head structure image, a head model including scalp, skull, cerebrospinal fluid, and cortical gray matter is constructed;

[0010] 2) Four-modal data acquisition steps: collecting the subject's whole-brain EEG signals, brain magnetic signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals;

[0011] 3) Brain activity signal extraction step: extract brain activity signals from the preprocessed EEG signals, MEG signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals, respectively, to obtain four types of signal data: event-related electric and magnetic fields, event-induced blood oxygen response, and blood flow response;

[0012] 4) Brain activity image data processing steps;

[0013] The head model is used to construct the source space of brain neural activity, and the transfer matrix from the source space to the EEG and MEG sensor array space is calculated in combination with the relative position of the subject's head and the EEG and MEG sensor arrays; the noise covariance matrix of the MEG signal is calculated based on the MEG data collected without the subject, and the minimum norm estimation is used in combination with the transfer matrix to perform joint EEG-MEG signal tracing to obtain the cortical gray matter neural intensity matrix E, where the rows of E represent the time series of neural activity at a certain cortical gray matter grid, and the columns represent different cortical gray matter grids;

[0014] Project the event-induced blood oxygen response onto the cortical gray matter of the head model and interpolate to obtain the cortical gray matter blood oxygen activity matrix H, where the rows of H represent the time series of blood oxygen activity at a certain cortical gray matter grid, and the columns represent different cortical gray matter grids;

[0015] Project the event-induced blood flow response onto the gray matter of the head model to obtain the blood flow activity matrix B, where the rows of B represent the time series of blood flow activity at a certain cortical gray matter grid, and the columns represent different cortical gray matter grids;

[0016] 5) Four-modal image fusion step: The cortical gray matter neural intensity matrix E, the cortical gray matter blood oxygen activity matrix H and the blood flow activity matrix B are fused on the head model to obtain the four-modal imaging result.

[0017] Preferably, in step 1), a head tissue segmentation algorithm is used to construct a head model including the scalp, skull, cerebrospinal fluid, and cortical gray matter.

[0018] Preferably, the head tissue segmentation algorithm adopts one or more of a grayscale gradient method, a grayscale threshold method, and a region segmentation method.

[0019] Preferably, in step 2), the collected EEG signals, MEG signals, near-infrared blood oxygen signals and ultrasonic blood flow signals are preprocessed for noise removal and signal correction.

[0020] Preferably, the step of preprocessing the EEG signal in step 2) includes:

[0021] In the time domain, the band-stop filter removes the power frequency and the band-pass filter extracts the required frequency components;

[0022] The independent component analysis method is used to remove the eye movement, electrocardiogram and myoelectric noise of the probe accessories, and the head movement EEG signal is corrected.

[0023] Preferably, the step of preprocessing the electroencephalographic signal in step 2) includes:

[0024] Perform spatial separation on the collected brain magnetic signals to remove environmental interference signals other than brain nerve activity;

[0025] In the time domain, the band-stop filter removes the power frequency and the band-pass filter extracts the required frequency components;

[0026] The independent component analysis method was used to remove the eye movement, electrocardiogram and myoelectric noise of the probe accessories, and the head movement magnetoencephalogram signal was corrected.

[0027] Preferably, the step of preprocessing the near-infrared blood oxygen signal in step 2) includes:

[0028] The relative change expression of oxygenated hemoglobin and deoxygenated hemoglobin concentrations obtained by using the modified Beer-Lambert Law is used to convert the optical intensity change into the relative change of oxygenated hemoglobin and deoxygenated hemoglobin concentrations.

[0029] The relative change data is corrected for head motion, respiratory and heartbeat artifacts are removed, and bandpass filtering is performed to extract the required frequency components.

[0030] Preferably, the step of preprocessing the ultrasonic blood flow signal in step 2) includes:

[0031] Using a transcranial echo correction algorithm to correct the distortion of the received ultrasound blood flow signal;

[0032] Using a spatiotemporal filtering algorithm based on singular value decomposition, the ultrasound image is filtered in the time and space domains to remove the influence of motion artifacts on image reconstruction, obtain cerebral cortical blood flow velocity and blood volume images, and reconstruct the ultrasound image.

[0033] Preferably, in step 3), a generalized linear model is used to extract brain activity signals from the preprocessed electroencephalogram signals, magnetic brain signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals.

[0034] Preferably, in step 5), the Laplacian pyramid and the wavelet pyramid are used to fuse the cortical gray matter nerve intensity matrix E, the cortical gray matter blood oxygen activity matrix H and the blood flow activity matrix B on the head model.

[0035] Preferably, the coupling coefficient calculation and characterization step is also included: the coupling coefficient C between any two modal data is calculated according to the following three formulas and the minimum norm estimation method: 1 , C 2 and D, according to the calculated coupling coefficient C 1 , C 2 and D plot the coupling coefficient on the head model;

[0036] E=C 1 H T +Noise

[0037] E=C 2 B T +Noise

[0038] H=DB T +Noise

[0039] Among them, T represents matrix transpose, and Noise represents noise.

[0040] An electromagnetic, photoacoustic and synchronous integrated brain function imaging device, comprising:

[0041] A magnetic field shielding device is used to shield the Earth's magnetic field and external magnetic field interference, and reduce the residual magnetic field, magnetic field gradient and magnetic noise in the imaging area;

[0042] The modal data processing module includes an electroencephalogram module, a magnetoencephalogram module, a near infrared module and an ultrasound module; wherein,

[0043] The EEG module is used to collect whole-brain EEG signals and provide EEG traceability imaging;

[0044] The magnetoencephalography module is used to collect magnetoencephalography signals of the whole brain and provide traceability imaging of brain magnetism;

[0045] The near-infrared module uses a multi-channel functional near-infrared spectroscopy system to measure changes in whole-brain blood oxygen and provide a cerebral cortex activation map;

[0046] Ultrasound module, using a functional ultrasound imaging system to measure cerebral cortical blood flow velocity and blood volume images;

[0047] The four-modal synchronization module is used to synchronize the modal signals of electroencephalogram, magnetoencephalogram, functional near-infrared spectroscopy and functional ultrasound through clock signals, clock feedback signals and homologous multi-frequency synchronization signals, and to compensate and correct the clocks of each channel according to the clock feedback;

[0048] The data acquisition module uses a four-modal sensor array to collect the subject's entire brain electroencephalogram signals, brain magnetic signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals.

[0049] Preferably, the magnetic field shielding device includes a multi-layer unequal spacing passive shielding device, a multi-order fingerprint coil active shielding device and a supporting mechanical device; the multi-layer unequal spacing passive shielding device is a semi-open 4-6 layers of unequal spacing Permalloy shielding tube, one end of the shielding tube is open, and the open end is integrated with a non-magnetic sliding bed; the multi-order fingerprint coil active shielding device includes a magnetic field compensation coil and a high-precision low-noise current source, and adopts an adaptive dynamic compensation algorithm.

[0050] Preferably, the EEG module includes EEG electrodes and an EEG electronic module, and the EEG electronic module includes a processing unit for electrostatic protection, multi-stage filtering, multi-stage amplification, analog-to-digital conversion and a microcontroller; wherein the multi-stage amplification uses an amplifier with a high common mode rejection ratio to perform high- and low-pass filtering and notching on the collected signals to filter out power frequency, DC and high-frequency noise, and a three-stage amplification circuit is used to amplify the EEG signal to the millivolt level.

[0051] Preferably, the magnetoencephalogram module includes an atomic magnetometer and a magnetoencephalogram electronic control and acquisition system; wherein the magnetoencephalogram electronic control and acquisition system includes physiological electrodes, a driver, an air chamber temperature control circuit, a laser closed-loop control circuit, an acquisition circuit, a physiological electrical acquisition amplifier and a system main controller; the system main controller adopts a field programmable gate array (FPGA) architecture to coordinate timing synchronization and data protocols between multiple channels.

[0052] Preferably, the near-infrared module includes a near-infrared photoelectrode, a near-infrared transmitting module, a near-infrared receiving module and a main control circuit; wherein the near-infrared transmitting module uses a frequency division multiplexing form of current to transmit near-infrared light to the brain area to be tested; the near-infrared receiving module collects the brain output light, uses FPGA to perform Fourier transform to obtain the amplitude of each frequency-coded near-infrared light, realizes frequency demodulation, pairs the near-infrared transmitting and receiving modules according to the frequency coding method, and identifies the imaging channel; the main control circuit includes a light source driving module that controls the near-infrared transmitting module, an analog acquisition circuit that collects the voltage signal of the near-infrared receiving module, a power supply module, a control module, and a communication module responsible for data aggregation and uploading.

[0053] Preferably, the ultrasound module adopts a functional ultrasound imaging system, including an ultrasound emission control and acquisition module, an ultrasound transducer array and a functional ultrasound signal processing module; the ultrasound transducer array includes a transducer array with a center frequency of 2MHz and a transducer array of 6MHz, which are used to obtain the highest signal-to-noise ratio and resolution near the cerebral cortex; the functional ultrasound signal processing module adopts a pre-corrected ultrasound emission sequence, a transcranial echo correction algorithm, a singular value-based space-time filtering algorithm and a generalized linear model.

[0054] Preferably, the quad-modal synchronization module utilizes the phase-locked loop of the FPGA to generate two homologous synchronization signals, high frequency and low frequency, with strict phase and fixed frequency, wherein the high frequency signal is connected to the magnetoencephalography module and the electroencephalography module, and the low frequency signal is connected to the near-infrared module and the ultrasound module, and these two homologous dual synchronization signals are used to align the data of each modality.

[0055] The electromagnetic photoacoustic synchronous integrated brain function imaging method and device described in the present invention integrates electroencephalography, magnetoencephalography, functional near-infrared spectroscopy and functional ultrasound imaging technology, realizes synchronous recording of electrophysiological and hemodynamic activities of the brain, and has the following advantages:

[0056] 1. This device has important complementary advantages in signal specificity, temporal resolution and spatial resolution, and can simultaneously depict neural activity from the perspective of neuroelectrophysiology and observe neural metabolism from hemodynamic information. Specifically, magnetoencephalography and electroencephalography are neuromagnetic imaging and neuroelectric imaging, respectively, and are sensitive to neuroelectrophysiological signals. Functional near-infrared spectroscopy is blood oxygen imaging, which is sensitive to changes in the concentration of oxygenated hemoglobin and deoxyhemoglobin in the brain. Functional ultrasound is blood flow imaging, which is sensitive to cerebral blood volume and cerebral blood flow. The signal specificity of the above four modalities is different, and brain activity is observed from the two aspects of neuroelectrophysiological activity and hemodynamics, respectively, and the signal specificity complements each other. Magnetoencephalography and electroencephalography have millisecond temporal resolution, and spatial resolution is at the millimeter and centimeter level, respectively, and are generally used to observe neural activity in the cerebral cortex. Both functional near-infrared spectroscopy and functional ultrasound have sub-second temporal resolution. The former has a spatial resolution of centimeters, and the imaging area can cover the entire brain, and the imaging depth is limited to the cerebral cortex. The latter has a sub-millimeter spatial resolution, and the imaging area covers part of the brain area, and the imaging depth is deeper. Therefore, the present invention complements the four sub-modalities of electromagnetic, photoacoustic and synchronous integrated brain function imaging in terms of signal specificity, spatiotemporal resolution and imaging area.

[0057] 2. The simultaneous measurement and analysis of neuroelectrophysiological signals and hemodynamic signals is a necessary way to deeply study the neurovascular coupling effect, which helps to have a more comprehensive understanding of the laws of brain neural activity and is a necessary foundation for promoting the development of cognitive neuroscience and the diagnosis of brain diseases.

[0058] 3. Currently, there is no non-invasive synchronous integrated brain function imaging system that includes electromagnetic, optical and acoustic quad-modalities in the world. Synchronous quad-modal brain function imaging can synchronously capture highly sporadic and random brain activities, and is an important research platform in the fields of cognitive neuroscience, brain pathology, brain-computer interface, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a schematic diagram of the magnetoencephalogram control and acquisition module of the present invention;

[0060] Figure 2 This is a schematic diagram of the structure of the multi-channel functional near-infrared spectroscopy module of the present invention;

[0061] Figure 3 A schematic diagram of the near-infrared anode and optical fiber of the present invention;

[0062] Figure 4 It is a schematic diagram of a functional ultrasonic signal processing module of the present invention;

[0063] Figure 5 It is a schematic diagram of a quad-modal synchronization module of the present invention;

[0064] Figure 6Schematic diagram of a quad-modal sensor array of the present invention. DETAILED DESCRIPTION

[0065] In order to make the various technical features and advantages or technical effects in the above technical solutions of the present invention more obvious and easy to understand, they are described in detail below with reference to the accompanying drawings.

[0066] The present invention proposes an electromagnetic photoacoustic synchronous integrated brain functional imaging method and device that integrates electroencephalogram, magnetoencephalogram, functional near-infrared spectroscopy and functional ultrasound imaging technology. The device includes a magnetic field shielding device, a modal data processing module, a four-modal synchronization module and a data acquisition module, wherein the modal data processing module includes an electroencephalogram module, a magnetoencephalogram module, a near-infrared module and an ultrasound module, which are specifically described as follows:

[0067] 1. Magnetic field shielding device

[0068] The magnetic field shielding device is used to shield the earth's magnetic field and external magnetic field interference, and reduce the residual magnetic field, magnetic field gradient and magnetic noise in the imaging area; the magnetic field shielding device includes a multi-layer unequal spacing passive shielding device, a multi-order fingerprint coil active shielding device and a supporting mechanical device. Among them, the multi-layer unequal spacing passive shielding device is a semi-open 4-6 layers of unequal spacing Permalloy shielding tube with an internal diameter of not less than 650mm. One end of the shielding tube is open, and the open end is integrated with a non-magnetic sliding bed. The subject lying on the non-magnetic bed can slide along the track from the open end into the magnetic shielding tube. The multi-order fingerprint coil active shielding device includes a magnetic field compensation coil and a high-precision low-noise current source, and adopts an adaptive dynamic compensation algorithm.

[0069] 2. EEG module

[0070] The EEG module is used to collect EEG signals of the whole brain and provide traceable imaging of EEG; the EEG module includes EEG electrodes and EEG electronic modules. Among them, the EEG electrode is a passive EEG electrode based on gel, which is made by silver / silver chloride powder sintering process (but not limited to this). The EEG electronic module includes processing units such as electrostatic protection, multi-stage filtering, multi-stage amplification, analog-to-digital conversion, and microcontroller. Among them, the multi-stage amplification uses an amplifier with a high common mode rejection ratio to perform high and low pass filtering and notching on the collected signals to filter out power frequency, DC and high frequency noise, and adopts a three-stage amplification circuit to amplify the EEG signal to the millivolt level. Specifically, the first-stage amplification circuit adopts a high input impedance design to facilitate the coupling of EEG signals into the electronic system, the second-stage amplification circuit adopts a high common mode rejection ratio to reduce the interference of common mode signals such as power frequency noise, and the third-stage amplification circuit adopts a dynamically adjustable gain coefficient design to expand the dynamic range of the acquisition system.

[0071] 3. Magnetoencephalography module

[0072] The magnetoencephalography module is used to collect magnetoencephalography signals of the whole brain and provide traceability imaging of brain magnetism; the magnetoencephalography module includes an atomic magnetometer and a magnetoencephalography electronic control and acquisition system.

[0073] Among them, the atomic magnetometer is an atomic magnetometer based on the principle of spin-free exchange relaxation and using alkali metal rubidium atoms as the working material. The atomic magnetometer is a magnetically sensitive sensor for magnetoencephalography. Its core component is a 87 The sensor is a borate glass cube gas cell with Rb and buffer gas (nitrogen). When the sensor is working, the gas cell is heated to ~130°C, causing the atoms in the gas cell to enter a state of relaxation without spin exchange. A small semiconductor laser emits a beam tuned to 87 The circularly polarized pump light of Rb D1 line; another small semiconductor laser emits a beam tuned to the 87 The D2 line of Rb is detuned by ~0.1nm to the linear polarization detection beam. The atomic magnetometer under this principle will be sensitive to the magnetic field perpendicular to the plane formed by the pump beam and the detection beam. By designing the direction of the pump beam and the detection beam incident on the gas chamber, the magnetic field sensitivity direction of the atomic magnetometer is set to be along the longitudinal direction of the sensor. The detection beam passing through the gas chamber is detected by the differential photodetector through the polarization splitter prism. The output signal is transmitted to the controller unit of the atomic magnetometer after preamplification. A small current coil is close to the gas chamber to calibrate the relationship between the magnetic field and the output voltage. The flexible printed circuit installed at the bottom of the sensor is used for data communication and control. Multiple shielded twisted pair cables are used to transmit signals between the flexible printed circuit and the atomic magnetometer electronics. The atomic magnetometer electronics automatically optimizes the sensor control parameters (including heating temperature, laser voltage and current) according to the sensor feedback, completes the calibration process, and outputs parameters, analog signals and digital signals indicating the sensor status. The control computer can monitor multiple channels in real time, control all sensor parameters, and calibrate the atomic magnetometer according to the calibration information measured before the experiment.

[0074] According to the characteristics of the atomic magnetometer output signal and the overall requirements of magnetoencephalography, the following design is made: Figure 1 The magnetoencephalogram electronic control and acquisition system shown in the figure includes physiological electrodes, drivers, air chamber temperature control circuits, laser closed-loop control circuits, acquisition circuits, physiological electrical acquisition amplifiers, and system main controllers. The system uses Ethernet as a communication method to meet the needs of large amounts of data exchange in the magnetoencephalogram of atomic magnetometers. In view of the fact that each atomic magnetometer needs to be independently controlled, each atomic magnetometer is equipped with a dedicated Figure 1The driving control circuit shown in the figure. According to the characteristics of multi-channel coordinated synchronization, the system main controller adopts a field programmable gate array (FPGA) architecture to coordinate the timing synchronization and data protocol between multiple channels. The MEG control and acquisition system adopts a modular design, consisting of multiple cascade chassis and a physiological electrode acquisition module; the cascade chassis contains 32 MEG acquisition control cards and 1 main control communication card; 1 MEG acquisition control card is responsible for controlling 1 atomic magnetometer and collecting its magnetic field; the main control communication card is responsible for receiving and parsing commands from the host computer, and forwarding them to the 32 MEG acquisition control cards, while collecting the status, magnetic field data, and task signals of each atomic magnetometer and uploading them to the host computer. The above modular design enables the MEG to have the ability of rapid assembly, debugging, and maintenance, and has strong scalability.

[0075] 4. Near infrared module,

[0076] The near infrared module uses a multi-channel near infrared spectroscopy system to measure changes in whole brain blood oxygen and provide a cerebral cortex activation map. The structure of the near infrared module is as follows: Figure 2 As shown, it includes a near-infrared photoelectrode, a near-infrared transmitting module, a near-infrared receiving module and a main control circuit. Among them, the near-infrared transmitting module uses frequency division multiplexing to modulate the current of the driving laser diode (or other near-infrared light emitting element) to emit near-infrared light to the brain area to be tested; the modulation range is 1.2kHz to 13.8kHz, the frequency interval between each transmitting module is 200Hz, and the amplitude is constant. The near-infrared receiving module collects the brain output light through an avalanche photodiode (or other near-infrared light receiving element), uses FPGA to perform Fourier transform to obtain the amplitude of each frequency-coded near-infrared light, and realizes frequency division demodulation; the frequency division modulation and demodulation pairs the near-infrared transmitting and receiving modules according to the frequency coding method, identifies the imaging channel, and can avoid the contamination of the imaging optical fiber by low-frequency ambient light (such as DC natural light and 50Hz lighting light, etc.). The main control circuit includes a light source driving module that controls the near-infrared transmitting module, an analog acquisition circuit that collects the voltage signal of the near-infrared receiving module, a power supply module, a control module, and a communication module responsible for data aggregation and uploading.

[0077] The near infrared module uses the laser frequency division modulation and demodulation principle to transmit and receive near dual-band near infrared light, such as 690nm and 830nm, but can also be used for three-band and above near infrared light. Taking 690nm and 830nm wavelength light as an example, Figure 3As shown, the transmitting optical fiber 1 is connected to the transmitting photoelectrode 2 to transmit light with wavelengths of 690nm and 830nm; the receiving optical fiber 3 is connected to the receiving photoelectrode 4 to receive the emitted light; the transmitting photoelectrode 2 and the receiving photoelectrode 4 are both arranged on the helmet 5 and contact the scalp 6; the optical signal passes through the scalp 6 and the skull 7 to detect the blood oxygen data in the cerebral cortex 8. This near-infrared module uses the Modified Beer-Lambert Law (MBLL) to convert the received near-infrared light intensity fluctuation into the relative change of the concentration of oxygenated hemoglobin and deoxygenated hemoglobin.

[0078] 5. Ultrasonic module

[0079] The ultrasound module uses a functional ultrasound imaging system, including an ultrasound emission control and acquisition module, an ultrasound transducer array, and a functional ultrasound signal processing module. The ultrasound transducer array includes a 2MHz transducer array and a 6MHz transducer array with a center frequency. The axial focusing depth is set to 2 to 3cm, aiming to obtain the highest signal-to-noise ratio and resolution near the cerebral cortex. The 2MHz transducer array is suitable for healthy subjects; the 6MHz transducer array is suitable for subjects who have undergone craniectomy. The functional ultrasound signal processing module uses a pre-corrected ultrasound emission sequence, a transcranial echo correction algorithm, a singular value-based spatiotemporal filtering algorithm, and a generalized linear model, such as Figure 4 shown.

[0080] The specific processing process is: combined with the digital skull model obtained by pre-modeling, the self-developed transcranial pre-corrected ultrasound transmission sequence and transcranial echo correction algorithm are used to achieve accurate focusing of the transmission signal and distortion correction of the received signal. Specifically, the finite element method is used to simulate the propagation process of the ultrasound signal emitted by the transducer in the head, and the intensity loss and phase delay caused by the skull to the ultrasound transmission signal are obtained. The above loss and delay are compensated in the ultrasound signal transmission stage to obtain the transcranial pre-corrected ultrasound transmission sequence to achieve accurate focusing of the transmission signal. The above loss and delay are corrected in the ultrasound signal reception stage to obtain the transcranial echo correction algorithm to achieve distortion correction of the received signal. The reconstructed image is obtained by a spatiotemporal filtering algorithm based on singular value decomposition (SVD) to obtain clear images of cerebral cortical blood flow velocity and blood volume. The spatiotemporal filtering algorithm based on singular value decomposition can be used to remove the influence of motion artifacts on image reconstruction, especially the influence on the imaging of small blood vessels with low blood flow velocity, which is crucial for neurovascular coupling. The reconstructed cerebral cortex image will be used to extract functional information through a generalized linear model (GLM) to obtain an activation map of the cerebral cortex.

[0081] 6. Four-mode synchronization module

[0082] The four-modal synchronization module realizes the synchronization of four-modal signals including electroencephalogram, magnetoencephalogram, functional near-infrared spectroscopy and functional ultrasound through clock signals, clock feedback signals and homologous multi-frequency synchronization signals, and compensates and corrects each clock according to clock feedback, such as Figure 5 As shown. The FPGA phase-locked loop is used to generate two synchronization signals with strict phase and fixed frequency, high and low frequencies (e.g., 1kHz, 10Hz). The high-frequency signal is connected to the magnetoencephalogram and electroencephalogram, and the low-frequency signal is connected to the functional near-infrared spectroscopy system and the functional ultrasound system. The two homologous dual synchronization signals are used to align the data of each modality. The above also applies to three or more synchronization signals. In the multi-signal scenario, it is necessary to ensure that the multiple signals have strict phase and fixed frequency, and one of the multiple signals is used as the main signal, and the remaining signals are aligned with the main signal in pairs.

[0083] Specifically, neuroelectrophysiological signals are usually generated tens of milliseconds after the nerve is induced and last for hundreds of milliseconds, so the typical frequency of magnetoencephalography and electroencephalography signals is 0-100Hz, and a few signals can reach the level of hundreds of Hz. Magnetoencephalography and electroencephalography can complete the acquisition of neuroelectrophysiological signals with a sampling rate of more than 1kHz, and the use of synchronization signals of 1kHz and above can meet the data synchronization requirements of the two. Hemodynamic signals are usually generated within a few seconds after neural activity and last for tens of seconds, so the typical frequency of functional near-infrared spectroscopy system and functional ultrasound system signals is below 1Hz. The above two instruments can complete the acquisition of hemodynamic signals with a sampling rate of more than 5Hz, and the use of synchronization signals of 10Hz and above can meet the data synchronization requirements of the two. In view of the different signal synchronization accuracy requirements of the four modal measurements of neuroelectrophysiology and hemodynamics, the homologous dual synchronization method is adopted as the four-modal data synchronization method. This method uses a high-frequency, high-precision, and high-stability clock-driven data synchronization unit to transmit two synchronization signals of high and low frequencies (such as 1kHz and 10Hz) with consistent phases. The high-frequency signal is connected to the magnetoencephalogram and electroencephalogram, and the low-frequency signal is connected to the functional near-infrared spectroscopy system and the functional ultrasound system. These two homologous dual synchronization signals are used in the host computer to align the data of each modality to achieve four-modal synchronous detection. Specific solution: The phase-locked loop of the FPGA is used to generate a multi-channel clock signal with a strict phase and fixed frequency, and each clock is compensated and corrected according to the clock feedback, and a synchronization signal is output to achieve data synchronization of each modality acquisition system. The quartz crystal oscillator is used as the clock source of the system, and the phase-locked loop of the FPGA generates 7 clocks and synchronization signals of different frequencies with fixed phases and delays, among which the clock signal is used as the system clock of the electroencephalogram, magnetoencephalogram, and functional near-infrared spectroscopy system. The above modalities provide local clock compensation function for the system clock through the clock feedback signal to achieve high-speed acquisition and precise synchronization of four-modal signals.

[0084] 7. Data acquisition module

[0085] The data acquisition module uses a quad-modal sensor array for electroencephalography, magnetoencephalography, functional near-infrared spectroscopy system, and functional ultrasound imaging sensors, which are flexibly arranged and matched with the subject's head to perform quad-modal data acquisition, such as Figure 6 As shown, the quad-modal sensor array is located on the helmet, and its components include EEG electrodes 9, atomic magnetometers 10, near-infrared photoelectrodes 11, and ultrasonic transducer arrays 12. According to the pre-scanned magnetic resonance structural image information of the subject, the subject's head is 3D modeled, the curved surface where the sensor array fits closely with the subject's scalp is extracted, the arrangement of the quad-modal sensors on the sensor array is optimized, and finally, the individualized quad-modal sensor array is made using 3D printing technology. That is, the head model provides the relative position relationship between the head and the sensor for manufacturing the quad-modal sensor array device, such as the ultrasonic transducer is installed at the temporal window at the junction of the forehead and the temporal, the near-infrared transmitting and receiving photoelectrodes are close to the scalp and the distance is about 3 cm; and the quad-modal data is linked (as described in step 4 below).

[0086] The steps and algorithms used in the method of the present invention to achieve electromagnetic, photoacoustic and synchronous integrated brain function imaging are described as follows:

[0087] 1) Build a head model.

[0088] For each subject's magnetic resonance or CT head structure image, a head model including the scalp, skull, cerebrospinal fluid, cortical gray matter (and white matter) was constructed using head tissue segmentation algorithms such as the gray gradient method, gray threshold method, and regional segmentation method.

[0089] 2) Preprocessing of electromagnetic, photoacoustic and quad-modal data.

[0090] The collected EEG signals are separated in signal space to remove environmental interference signals other than brain nerve activity. The EEG and EEG signals are further preprocessed, including band-stop filtering to remove the power frequency (50Hz) in the time domain, band-pass filtering to extract the required frequency components, and independent component analysis to remove eye movement, electrocardiogram and probe accessory myoelectric noise, and perform head movement correction.

[0091] The near-infrared brain imaging data were preprocessed by using the Modified Beer-Lambert Law (MBLL) to obtain the relative change value expression of the oxyhemoglobin and deoxyhemoglobin concentrations (as shown in the following formula), and the optical intensity changes were converted into the relative changes of the oxyhemoglobin and deoxyhemoglobin concentrations. The data were further corrected for head motion, respiratory and heartbeat artifacts were removed, and bandpass filtering was performed to extract the required frequency components.

[0092]

[0093] Where Δ[HbO] is the relative change in oxygenated hemoglobin concentration, Δ[HbR] is the relative change in deoxygenated hemoglobin concentration, ΔOD is the relative optical density change, DPF is the differential path length factor, and wavelength λ 1 and λ 2 They are 690nm and 830nm respectively; ε represents the molar extinction coefficient, unit: M -1 cm -1 ; d represents the distance between the light source and the detector, unit: cm.

[0094] Ultrasound imaging data was preprocessed, and the distortion of the received signal was corrected using the transcranial echo correction algorithm to reconstruct the ultrasound image. The spatiotemporal filtering algorithm based on singular value decomposition (SVD) was used to obtain the cerebral cortical blood flow velocity and blood volume images. The spatiotemporal filtering algorithm based on singular value decomposition can be used to remove the influence of motion artifacts on image reconstruction, especially the influence on the imaging of small blood vessels with low blood flow velocity, which is crucial for neurovascular coupling.

[0095] 3) Extracting brain activity signals

[0096] Combined with the experimental paradigm, the generalized linear model (GLM) was used to extract functional information from the preprocessed EEG signals, MEG signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals, respectively, to obtain event-related electric fields, event-related magnetic fields, event-induced blood oxygen responses, and event-induced blood flow responses.

[0097] 4) Brain activity images

[0098] 4-1) Combined EEG-MEG tracing

[0099] The head model is used to construct the source space of brain neural activity, and the transfer matrix from the source space to the EEG and MEG sensor array space is calculated based on the relative position of the subject's head and the EEG and MEG sensor array. The noise covariance matrix of the MEG signal is calculated based on the MEG data collected without the subject, and the minimum norm estimation is used to perform joint EEG-MEG signal tracing in combination with the transfer matrix to obtain the cortical gray matter neural intensity matrix E. The rows of the cortical gray matter neural intensity matrix E represent the time series of neural activity at a cortical gray matter grid, and the columns represent different cortical gray matter grids.

[0100] 4-2) Near-infrared functional imaging

[0101] The event-induced blood oxygen response is projected onto the cortical gray matter of the head model and interpolated to obtain the cortical gray matter blood oxygen activity matrix H, where the rows of H represent the time series of blood oxygen activity at a certain cortical gray matter grid, and the columns represent different cortical gray matter grids.

[0102] 4-3) Functional Ultrasound Imaging

[0103] Using the structural information of the 3D printed helmet, the head model is aligned with the position of the ultrasonic transducer, and the event-induced blood flow response is projected onto the gray matter (and white matter) of the head model to obtain the blood flow activity matrix B, where the rows of B represent the time series of blood flow activity at a certain cortical gray matter grid, and the columns represent different cortical gray matter grids.

[0104] 5) Four-modal image fusion

[0105] Using Laplacian pyramid and wavelet pyramid, the cortical gray matter neural intensity matrix E, the cortical gray matter blood oxygen activity matrix H and the blood flow activity matrix B are fused on the head model to obtain the four-modal imaging results.

[0106] As a further processing result, the following steps 6) may also be included:

[0107] 6) Calculation and characterization of coupling coefficient

[0108] The neurovascular coupling column vector C represents the neurovascular coupling strength of the cortical gray matter. The blood flow-blood oxygen coupling column vector D represents the coupling strength between blood flow and blood oxygen in the cortical gray matter. The values ​​of C and D at each position correspond to the grid points of the cortical gray matter.

[0109] E=C 1 H T +Noise

[0110] E=C 2 B T +Noise

[0111] H=DB T +Noise

[0112] Calculate C using the above formula and the minimum norm estimation method 1 , C 2 and D, T represents matrix transpose, and Noise represents noise.

[0113] According to the above calculation results, the coupling coefficient diagrams are drawn on the head model. The coupling coefficient diagram is used to represent the correlation between E, H, and B. The correlation is different in different locations in the brain.

[0114] A specific embodiment is given below, using the electromagnetic, photoacoustic and four-modality synchronous integrated brain function imaging method and device provided by the present invention to realize the multimodal brain function imaging application of synchronously collecting brain electrophysiological and hemodynamic activity signals. The specific implementation includes two parts: the preparation stage and the experimental stage.

[0115] Preparation

[0116] First, the subject's head was scanned with magnetic resonance imaging to obtain T1-weighted structural images and arterial spin labeling images. Subsequently, different experimental tasks were prepared according to the subject's physical condition and willingness. The alternative experimental tasks included somatosensory, auditory, visual tasks, and advanced tasks such as language, movement, and memory. Based on the subject's condition and experimental design, an individualized multimodal integrated helmet was produced in combination with the subject's magnetic resonance imaging results.

[0117] Experimental Phase

[0118] The electromagnetic, photoacoustic and four-modality synchronous integrated brain function imaging device was used to synchronously collect the whole-brain magnetoencephalogram signals, electroencephalogram signals, functional near-infrared spectroscopy signals, and ultrasound imaging signals of the subjects under the selected tasks, and simultaneously record physiological signals such as electrooculogram and electrocardiogram. After the experiment, various analysis methods such as brain tracing imaging and brain network were used to study the brain function of the subjects.

[0119] The specific experimental process is as follows:

[0120] 1) Before the experiment, the subjects were informed of the requirements: shaving their heads, not wearing makeup, and not wearing underwear with metal.

[0121] 2) Before the experiment, remove any metal objects from the subject, such as belts, watches, mobile phones, keys, wallets, and glasses (if any). If a visual experiment is conducted, the subject should wear special non-magnetic glasses.

[0122] 3) The subjects will be equipped with a multi-modal integrated helmet (including an EEG cap, a near-infrared fiber head, and an ultrasound probe). The EEG cap will use EEG cream to connect the scalp to the EEG electrode leads, and use a cotton swab to stir to reduce the impedance to below 10 kilo-ohms. Apply ultrasound coupling agent to the subjects before installing the ultrasound probe.

[0123] 4) The subjects were equipped with electrooculogram electrodes, electrocardiogram electrodes and other physiological monitoring sensors. To reduce skin resistance, the skin was cleaned with an alcohol cotton ball before the electrodes were placed, and a small amount of conductive paste was applied to the electrodes before they were fixed in the above positions.

[0124] 5) The subject sits or lies down according to the experimental design, and the seat or bed is adjusted so that the top of the subject's head is as close to the measurement helmet as possible without causing discomfort.

[0125] 6) Install and adjust task equipment (such as screens and curtains for visual tasks, headphones for auditory tasks), auxiliary measurement equipment and various modality acquisition devices as needed.

[0126] 7) Explain the experimental process and precautions as well as the time and requirements of the first experiment to the subjects again.

[0127] 8) Start the task and record the first session of data.

[0128] 9) Rest and check the subject's condition.

[0129] 10) Repeat steps 7-9 to complete the experiments in other sections.

[0130] In the above description, the electromagnetic, photoacoustic and four-modal synchronous integrated brain function imaging method and device of the present invention are further described through specific embodiments, so that those skilled in the art can have a more thorough understanding of the features and advantages of the present invention. It should be noted that the above description is only a representative typical application. Obviously, the present invention is not limited to any specific structure, function, device and method described herein, and may also have other embodiments, or a combination of other embodiments. The software / hardware modules described in the present invention or shown in the drawings can also be flexibly adjusted as needed.

[0131] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Any person skilled in the art may modify and alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be as set forth in the claims.

Claims

1. A method for synchronous integrated electromagnetic, photoacoustic and brain function imaging, characterized in that: The following steps are involved: 1) Head model construction steps: Based on the subject's MRI or CT head structure image, a head model including scalp, skull, cerebrospinal fluid, and cortical gray matter is constructed; 2) Four-modal data acquisition steps: collecting the subject's whole-brain EEG signals, brain magnetic signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals; 3) Brain activity signal extraction step: extract brain activity signals from the preprocessed EEG signals, MEG signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals, respectively, to obtain four types of signal data: event-related electric and magnetic fields, event-induced blood oxygen response, and blood flow response; 4) Brain activity image data processing steps; The head model is used to construct the source space of brain neural activity, and the transfer matrix from the source space to the EEG and MEG sensor array space is calculated in combination with the relative position of the subject's head and the EEG and MEG sensor arrays; the noise covariance matrix of the MEG signal is calculated based on the MEG data collected without the subject, and the minimum norm estimation is used in combination with the transfer matrix to perform joint EEG-MEG signal tracing to obtain the cortical gray matter neural intensity matrix E, where the rows of E represent the time series of neural activity at a certain cortical gray matter grid, and the columns represent different cortical gray matter grids; Project the event-induced blood oxygen response onto the cortical gray matter of the head model and interpolate to obtain the cortical gray matter blood oxygen activity matrix H, where the rows of H represent the time series of blood oxygen activity at a certain cortical gray matter grid, and the columns represent different cortical gray matter grids; Project the event-induced blood flow response onto the gray matter of the head model to obtain the blood flow activity matrix B, where the rows of B represent the time series of blood flow activity at a certain cortical gray matter grid, and the columns represent different cortical gray matter grids; 5) Four-modal image fusion step: The cortical gray matter neural intensity matrix E, the cortical gray matter blood oxygen activity matrix H and the blood flow activity matrix B are fused on the head model to obtain the four-modal imaging result.

2. The method according to claim 1, characterized in that In step 1), a head tissue segmentation algorithm is used to construct a head model including the scalp, skull, cerebrospinal fluid, and cortical gray matter.

3. The method according to claim 2, characterized in that The head tissue segmentation algorithm adopts one or more of the gray gradient method, gray threshold method, and region segmentation method.

4. The method according to claim 1, characterized in that In step 2), the collected EEG signals, MEG signals, near-infrared blood oxygen signals and ultrasonic blood flow signals are preprocessed for noise removal and signal correction.

5. The method according to claim 4, characterized in that The steps of preprocessing EEG signals include: In the time domain, the band-stop filter removes the power frequency and the band-pass filter extracts the required frequency components; The independent component analysis method is used to remove the eye movement, electrocardiogram and myoelectric noise of the probe accessories, and the head movement EEG signal is corrected.

6. The method according to claim 4, characterized in that The steps of preprocessing the EEG signals include: Perform spatial separation on the collected brain magnetic signals to remove environmental interference signals other than brain nerve activity; In the time domain, the band-stop filter removes the power frequency and the band-pass filter extracts the required frequency components; The independent component analysis method was used to remove the eye movement, electrocardiogram and myoelectric noise of the probe accessories, and the head movement magnetoencephalogram signal was corrected.

7. The method according to claim 4, characterized in that The steps of preprocessing the near-infrared blood oxygen signal include: The relative change expression of oxygenated hemoglobin and deoxygenated hemoglobin concentrations obtained by using the modified Beer-Lambert Law is used to convert the optical intensity change into the relative change of oxygenated hemoglobin and deoxygenated hemoglobin concentrations. The relative change data is corrected for head motion, respiratory and heartbeat artifacts are removed, and bandpass filtering is performed to extract the required frequency components.

8. The method according to claim 4, characterized in that The steps of preprocessing the ultrasonic blood flow signal include: Using a transcranial echo correction algorithm to correct the distortion of the received ultrasound blood flow signal; Using a spatiotemporal filtering algorithm based on singular value decomposition, the ultrasound image is filtered in the time and space domains to remove the influence of motion artifacts on image reconstruction, obtain cerebral cortical blood flow velocity and blood volume images, and reconstruct the ultrasound image.

9. The method according to claim 1, characterized in that In step 3), a generalized linear model is used to extract brain activity signals from the preprocessed electroencephalogram signals, magnetic brain signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals.

10. The method according to claim 1, characterized in that In step 5), the Laplacian pyramid and wavelet pyramid are used to fuse the cortical gray matter neural intensity matrix E, the cortical gray matter blood oxygen activity matrix H and the blood flow activity matrix B on the head model.

11. The method according to any one of claims 1 to 10, characterized in that: The method also includes the steps of coupling coefficient calculation and characterization: calculating coupling coefficients C1, C2 and D between any two modal data according to the following three formulas and the minimum norm estimation method, and drawing a coupling coefficient diagram on the head model according to the calculated coupling coefficients C1, C2 and D; E=C1H T +Noise E=C2B T +Noise H=DB T +Noise Among them, T represents matrix transpose, and Noise represents noise.

12. The method according to claim 1, characterized in that In step 1), a head tissue segmentation algorithm is used to construct a head model including the scalp, skull, cerebrospinal fluid, and cortical gray matter; the head tissue segmentation algorithm adopts one or more of a gray gradient method, a gray threshold method, and a region segmentation method; In step 2), the collected EEG signals, MEG signals, near-infrared blood oxygen signals and ultrasonic blood flow signals are preprocessed to remove noise and perform signal correction; The steps of preprocessing EEG signals include: In the time domain, the band-stop filter removes the power frequency and the band-pass filter extracts the required frequency components; Through independent component analysis, the noise from eye movement, electrocardiogram and myoelectricity of probe accessories is removed, and the EEG signal of head movement is corrected. The steps of preprocessing the EEG signals include: Perform spatial separation on the collected brain magnetic signals to remove environmental interference signals other than brain nerve activity; In the time domain, the band-stop filter removes the power frequency and the band-pass filter extracts the required frequency components; The independent component analysis method is used to remove the noise of eye movement, electrocardiogram and myoelectric noise of probe accessories, and the head movement EEG signal is corrected; The steps of preprocessing the near-infrared blood oxygen signal include: The relative change expression of oxygenated hemoglobin and deoxygenated hemoglobin concentrations obtained by using the modified Beer-Lambert Law is used to convert the optical intensity change into the relative change of oxygenated hemoglobin and deoxygenated hemoglobin concentrations. Perform head motion correction on the relative change data, remove breathing and heartbeat artifacts, and extract the required frequency components through bandpass filtering; The steps of preprocessing the ultrasonic blood flow signal include: Using a transcranial echo correction algorithm to correct the distortion of the received ultrasound blood flow signal; Using a spatiotemporal filtering algorithm based on singular value decomposition, the ultrasound image is filtered in the time and space domains to remove the influence of motion artifacts on image reconstruction, obtain the cerebral cortex blood flow velocity and blood volume images, and reconstruct the ultrasound image; In step 3), a generalized linear model is used to extract brain activity signals from the preprocessed EEG signals, MEG signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals; In step 5), the Laplacian pyramid and wavelet pyramid are used to fuse the cortical gray matter neural intensity matrix E, the cortical gray matter blood oxygen activity matrix H, and the blood flow activity matrix B on the head model; The method also includes the steps of coupling coefficient calculation and characterization: calculating coupling coefficients C1, C2 and D between any two modal data according to the following three formulas and the minimum norm estimation method, and drawing a coupling coefficient diagram on the head model according to the calculated coupling coefficients C1, C2 and D; E=C1H T +Noise E=C2B T +Noise H=DB T +Noise Among them, T represents matrix transpose, and Noise represents noise.

13. An electromagnetic, photoacoustic and synchronous integrated brain function imaging device, characterized in that: include: A magnetic field shielding device is used to shield the Earth's magnetic field and external magnetic field interference, and reduce the residual magnetic field, magnetic field gradient and magnetic noise in the imaging area; The modal data processing module includes an electroencephalogram module, a magnetoencephalogram module, a near infrared module and an ultrasound module; wherein, The EEG module is used to collect whole-brain EEG signals and provide EEG traceability imaging; The magnetoencephalography module is used to collect magnetoencephalography signals of the whole brain and provide traceability imaging of brain magnetism; The near-infrared module uses a multi-channel functional near-infrared spectroscopy system to measure changes in whole-brain blood oxygen and provide a cerebral cortex activation map; Ultrasound module, using a functional ultrasound imaging system to measure cerebral cortical blood flow velocity and blood volume images; The four-modal synchronization module is used to synchronize the modal signals of electroencephalogram, magnetoencephalogram, functional near-infrared spectroscopy and functional ultrasound through clock signals, clock feedback signals and homologous multi-frequency synchronization signals, and to compensate and correct the clocks of each channel according to the clock feedback; The data acquisition module uses a four-modal sensor array to collect the subject's entire brain electroencephalogram signals, brain magnetic signals, near-infrared blood oxygen signals, and ultrasonic blood flow signals.

14. The device according to claim 13, characterized in that The magnetic field shielding device includes a multi-layer unequal spacing passive shielding device, a multi-order fingerprint coil active shielding device and a supporting mechanical device; the multi-layer unequal spacing passive shielding device is a semi-open 4-6 layers of unequal spacing Permalloy shielding tube, one end of the shielding tube is open, and the open end is integrated with a non-magnetic sliding bed; the multi-order fingerprint coil active shielding device includes a magnetic field compensation coil and a high-precision low-noise current source, and adopts an adaptive dynamic compensation algorithm.

15. The device according to claim 13, characterized in that The EEG module includes EEG electrodes and an EEG electronic module. The processing unit of the EEG electronic module includes electrostatic protection, multi-stage filtering, multi-stage amplification, analog-to-digital conversion and a microcontroller. Multi-stage amplification uses an amplifier with a high common-mode rejection ratio to perform high- and low-pass filtering and notching on the collected signals to filter out power frequency, DC and high-frequency noise, and a three-stage amplification circuit is used to amplify the EEG signal to the millivolt level.

16. The device according to claim 13, characterized in that The magnetoencephalography module includes an atomic magnetometer and a magnetoencephalography electronic control and acquisition system; the magnetoencephalography electronic control and acquisition system includes physiological electrodes, a driver, an air chamber temperature control circuit, a laser closed-loop control circuit, an acquisition circuit, a physiological electrical acquisition amplifier and a system main controller; the system main controller adopts a programmable array logic gate circuit architecture to coordinate the timing synchronization and data protocol between multiple channels.

17. The device according to claim 13, characterized in that The near-infrared module includes a near-infrared photoelectrode, a near-infrared transmitting module, a near-infrared receiving module and a main control circuit; the near-infrared transmitting module uses a frequency-division multiplexing current to transmit near-infrared light to the brain area to be tested; the near-infrared receiving module collects the brain's emitted light, uses FPGA to perform Fourier transform to obtain the amplitude of each frequency-encoded near-infrared light, realizes frequency demodulation, pairs the near-infrared transmitting and receiving modules according to the frequency encoding method, and identifies the imaging channel; the main control circuit includes a light source driving module that controls the near-infrared transmitting module, an analog acquisition circuit that collects the voltage signal of the near-infrared receiving module, a power supply module, a control module, and a communication module responsible for data aggregation and uploading.

18. The device according to claim 13, characterized in that The ultrasound module adopts a functional ultrasound imaging system, including an ultrasound emission control and acquisition module, an ultrasound transducer array and a functional ultrasound signal processing module; the ultrasound transducer array includes a transducer array with a center frequency of 2MHz and a transducer array of 6MHz, which are used to obtain the highest signal-to-noise ratio and resolution near the cerebral cortex; the functional ultrasound signal processing module adopts a pre-corrected ultrasound emission sequence, a transcranial echo correction algorithm, a singular value-based space-time filtering algorithm and a generalized linear model.

19. The device according to any one of claims 13 to 18, characterized in that: The quad-modal synchronization module uses the FPGA's phase-locked loop to generate high-frequency and low-frequency homologous dual-synchronous signals with strict phase and fixed frequency. The high-frequency signal is connected to the magnetoencephalography module and the electroencephalography module, and the low-frequency signal is connected to the near-infrared module and the ultrasound module. These two homologous dual-synchronous signals are used to align the data of each modality.

Images