An auditory nerve representation device based on magnetoencephalography and functional magnetic resonance imaging

Abstract

The application discloses a kind of based on magnetoencephalogram and functional magnetic resonance imaging auditory nerve representation device, device includes: auditory stimulation system is used to generate magnetoencephalogram signal and functional magnetic resonance signal;Head-mounted array sensor system is used to collect generated magnetoencephalogram signal;Magnetoencephalogram signal acquisition system is used to receive the magnetoencephalogram signal;Functional magnetic resonance imaging signal acquisition system is used to receive the functional magnetic resonance signal;Magnetoencephalogram data preprocessing system is used to preprocess magnetoencephalogram signal, obtain magnetoencephalogram processing data;Functional magnetic resonance imaging data preprocessing system is used to preprocess the functional magnetic resonance signal, obtain function processing data;Dual-mode data joint analysis system is used to receive and fuse magnetoencephalogram processing data and function processing data, using support vector machine classifier constructs stimulation two two mutual classification diagonal symmetric representation dissimilarity matrix, according to representation dissimilarity matrix obtains auditory nerve representation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical diagnosis, and particularly relates to an auditory nerve representation device based on magnetoencephalography and functional magnetic resonance imaging. BACKGROUND

[0002] Sound is a very common and important external stimulus in the environment in which humans live, because its acoustic structure not only contains language information, but also carries a lot of information related to the identity, state, motivation, etc. of the sounder. Therefore, accurately perceiving and recognizing this information is of great biological and social significance to humans.

[0003] Auditory cognitive impairment refers to the case that, under the condition of complete hearing, speech (reading, writing, and oral language), etc., the perception and recognition of sound are impaired, which is manifested as the ability to normally hear sound, but unable to recognize and thus unable to understand the speech of others.

[0004] So far, the understanding of auditory cognition by humans is still in a relatively early stage. Due to the influence of medical conditions and psychological states, otology has not reached a consensus on auditory cognitive impairment. In addition, due to low clinical attention, auditory cognitive impairment is often masked by other impairments and is easily mistaken for deafness, aphasia, etc.

[0005] Many studies have shown that the brain contains two information processing flows, ventral information flow and dorsal information flow, in the process of processing speech signals, mainly involving bilateral superior temporal sulcus and superior temporal gyrus regions. In addition, studies have shown that some sub-regions of the above brain regions are selective to different types of sound, for example, the activation intensity of biological sound is higher than that of non-biological sound, and the activation intensity of human sound is higher than that of non-human sound.

[0006] MEG is a non-invasive brain function detection technology for detecting brain physiological signals, which has millisecond-level time resolution and millimeter-level spatial resolution, and the signal is almost not affected by tissue conductivity and skull thickness, etc. It has great advantages in the positioning accuracy of neurons and the sensitivity of measured signals.

[0007] The MEG measurement technology on the scalp is a measurement technology based on an optically pumped magnetometer (OPM). Compared with a superconducting quantum interference device, since it does not require liquid helium at low temperature, it can detect magnetoencephalography signals at room temperature. Therefore, the OPM sensor can be placed close to the scalp, and can more sensitively receive higher intensity signals.

[0008] fMRI can depict the brain activity in real time, track subtle changes in blood flow dynamics, and show the brain activation area when stimulated by external stimuli. However, fMRI obtains the brain slices as a complete image, which limits the speed of data collection, so fMRI has high spatial resolution but low temporal resolution.

[0009] At present, most of the researches on auditory cognitive impairment are concentrated in the field of structural imaging of fMRI, and few researches are in the field of functional imaging of MEG, and there is no report on joint analysis of the two modalities. SUMMARY

[0010] In order to solve the above technical problems, the application provides an auditory nerve representation device based on magnetoencephalogram and functional magnetic resonance imaging, which comprises: an auditory stimulation system, a head-mounted array sensor system, a magnetoencephalogram signal acquisition system, a functional magnetic resonance imaging signal acquisition system, a magnetoencephalogram data preprocessing system, a functional magnetic resonance imaging data preprocessing system, and a dual-modality data joint analysis system.

[0011] The auditory stimulation system is used to generate OPM-MEG signals and fMRI signals, wherein the OPM-MEG signals are magnetoencephalogram signals, and the fMRI signals are functional magnetic resonance signals.

[0012] The head-mounted array sensor system is used to acquire the generated magnetoencephalogram signals.

[0013] The magnetoencephalogram signal acquisition system is used to receive the magnetoencephalogram signals.

[0014] The functional magnetic resonance imaging signal acquisition system is used to receive the functional magnetic resonance signals.

[0015] The magnetoencephalogram data preprocessing system is used to preprocess the magnetoencephalogram signals to obtain magnetoencephalogram processing data.

[0016] The functional magnetic resonance imaging data preprocessing system is used to preprocess the functional magnetic resonance signals to obtain functional processing data.

[0017] The dual-modality data joint analysis system is used to receive and fuse the magnetoencephalogram processing data and the functional processing data, use a support vector machine classifier to construct a diagonal symmetric representation dissimilarity matrix for two-by-two mutual classification of stimuli, and obtain an auditory nerve representation result according to the diagonal symmetric representation dissimilarity matrix for two-by-two mutual classification of stimuli.

[0018] The workflow of the dual-modality data joint analysis system comprises:

[0019] performing a sliding window-based time-frequency analysis on the brain magnetic processing data to obtain a time-frequency spectrum, performing a t-test and a comparison correction on each time point and frequency point according to the time-frequency spectrum to obtain a significant difference level, and obtaining a stimulation activation time period and a frequency period according to the significant difference level;

[0020] based on the stimulation activation time period and the frequency period, performing activation source positioning on the brain magnetic processing data and the functional processing data to obtain and compare differences in activated brain regions under different stimulations;

[0021] based on the brain magnetic processing data, constructing a diagonal symmetric representation dissimilarity matrix for two-by-two stimulation classification using a support vector machine classifier, calculating a Pearson correlation coefficient between a representation dissimilarity matrix of different activated brain regions and a series of time point representation dissimilarity matrices in a time sequence to obtain dynamic representations of different brain regions at different time points, and performing a t-test on the dynamic representations to obtain neural representation results of different brain regions in the brain.

[0022] Optionally, the auditory neural representation device further comprises a magnetic shielding system for avoiding interference of an external magnetic field on generated magnetoencephalogram signals, the magnetic shielding system further comprising a passive shielding module and an active shielding module.

[0023] The passive shielding module is a multi-layer shielding layer composed of high-permeability alloy and high-conductivity alloy, which twists the magnetic field and attracts the magnetic flux into the alloy to reduce the influence of the magnetic field on the internal space of the magnetic shielding chamber.

[0024] The active shielding module is composed of a background magnetic field sensor array and an active compensation coil, and a LabVIEW controller is used to control the compensation coil, the LabVIEW has a built-in dynamic feedback controller based on PID, the controller calculates the corresponding compensation current through the response of the proportional gain and integral gain of the system, and drives the active compensation coil to generate a compensation current, generates a magnetic field equal and opposite to the reference sensor array, and realizes active shielding.

[0025] Optionally, the auditory stimulation system comprises an editing and generating auditory stimulation module and a stimulation presentation device.

[0026] The editing and generating auditory stimulation module is used to edit and execute a preset auditory stimulation paradigm.

[0027] The stimulation presentation device comprises a vacuum rubber tube and a sealed earphone, which is used to deliver sound stimulation to the subject while isolating the subject's auditory system.

[0028] Optionally, the auditory stimulation paradigm includes six different kinds of sound stimuli, including: human language, human emotional sound, animal sound, natural environment sound, life scene sound and musical instrument sound; when playing the sound stimuli, the auditory stimulation module synchronously generates a square wave current pulse signal and sends the square wave current pulse signal to the magnetoencephalography signal acquisition system or the functional magnetic resonance imaging signal acquisition system, and marks the square wave current pulse signal on the magnetoencephalography signal and the functional magnetic resonance signal.

[0029] Optionally, the head-mounted array sensor system includes a flexible magnetoencephalography acquisition cap, an OPM sensor array and a sensor fixing base; the flexible magnetoencephalography acquisition cap is provided with holes according to the 10-20 international standard lead system, the holes are fixed with the sensor fixing base, the flexible magnetoencephalography acquisition cap is connected with the OPM sensor array through the sensor fixing base, and the OPM sensor array is used for recording the magnetic field intensity of different brain sites of the subject under the auditory stimulation.

[0030] Optionally, the functional magnetic resonance imaging signal acquisition system includes a magnetic resonance imaging console, a magnetic resonance imaging machine, a gradient coil and a radio frequency coil.

[0031] The magnetic resonance imaging console is used for controlling the parameters of the MRI scan, collecting and storing the imaging data and being connected with a preprocessing module in the functional magnetic resonance imaging data preprocessing system.

[0032] The magnetic resonance imaging machine is used for generating high-resolution brain structure and function images.

[0033] The gradient coil is used for generating spatial gradients in the magnetic resonance scan.

[0034] The radio frequency coil is used for transmitting and receiving radio frequency signals, responsible for exciting the functional magnetic resonance signals in the brain tissue and capturing the functional magnetic resonance signals for image reconstruction.

[0035] Optionally, the magnetoencephalography processing data is subjected to time-frequency analysis based on a sliding window to obtain a time-frequency spectrum.

[0036] A window function with a preset length is selected, the window is placed on the signal from the starting point of the signal, the signal is segmented and weighted by the window to generate a short data segment;

[0037] The spectrum of the windowed short data segment is calculated by using an autoregressive method, and the window is slid along the time axis to repeatedly calculate the spectrum until the window reaches the end of the signal, and a time-frequency spectrum is obtained.

[0038] Optionally, based on the stimulation activation time period and the frequency period, the activated source is located by combining the magnetoencephalography processing data and the functional processing data, and the differences of the activated brain regions of different stimuli are obtained and compared.

[0039] scanning the subject by a magnetic resonance imaging signal acquisition system to obtain structural MRI data of the subject to obtain anatomical information of the head;

[0040] segmenting the MRI data using a computational biophysical software FreeSurfer to create a head model;

[0041] scanning the subject wearing the sensor headgear using a laser scanner to obtain a scan image, and aligning the sensor array of the OPM-MEG with the head model using a sample consistency initial registration algorithm and an iterative closest point method;

[0042] reconstructing the activity of the brain source using a head surface topology map generated by the MEG signal based on the MNE-python toolkit, and visualizing the source positioning result on the MRI image of the subject to show the position and intensity of the brain activation source under the stimulation, and performing t-test on the source activity to obtain the difference of the activated brain area under different stimulation conditions.

[0043] Compared with the prior art, the present application has the following beneficial effects:

[0044] (1) Comprehensive use of space-time resolution: MEG provides high time resolution of neural activity, while fMRI provides more extensive spatial coverage and depth of brain information. By joint analysis, the time and space dimensions are comprehensively considered.

[0045] fMRI measures blood oxygen level changes, and MEG measures the weak magnetic field generated by the electrical activity of neurons in the brain, which is information from different physiological sources. Combining the neurophysiological data of the two modalities can have high time resolution and spatial resolution at the same time.

[0046] (2) Specific and efficient device system design: The device system designed in the present application includes MEG acquisition module, fMRI acquisition module, stimulation generation and presentation module, and also includes auditory nerve representation data preprocessing and bimodal data joint analysis module.

[0047] On the one hand, the device system supports editing and presenting different auditory stimulation paradigms; on the other hand, the device system also supports customizing the data processing process according to the needs, and can analyze a large amount of data in a short time, improving the processing speed and efficiency.

[0048] (3) More direct neurophysiological evidence: fMRI mainly reflects the changes of blood supply and metabolism, which indirectly reflects the neuronal activity, while MEG measures the magnetic field changes caused by the direct electrical activity of neurons, thus providing a direct indicator of neuronal activity.

[0049] As mentioned above, most of the current researches on auditory cognitive impairment are focused on the structural imaging field of fMRI, which reflects the changes of blood flow and metabolism by detecting the changes of blood oxygen level, so as to infer the active brain regions. Compared with the current fMRI measurement, MEG directly measures the magnetic field changes of neurons, which is a more direct indicator of neural activity.

[0050] (4) Customizable probe layout: A series of holes for different sites according to the 10-20 international standard lead system are opened on the flexible acquisition cap in the present application, which supports free arrangement of sensors, so that the most suitable positions on the head surface can be selected to place the sensors according to the specific research needs. In addition, the acquisition cap has multiple sensor channels, so that multi-channel magnetic field measurement can be carried out at different positions of the head at the same time, thereby improving the spatial resolution. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the present application, the following briefly introduces the drawings needed to be used in the embodiments. Obviously, the drawings described in the following only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without paying the creative labor.

[0052] Figure 1 The structure diagram of each part of the auditory nerve representation device of the embodiment of the present application;

[0053] Figure 2 The structure diagram of each part of the auditory nerve representation device of the embodiment of the present application;

[0054] Figure 3 The schematic diagram of the background magnetic field sensor array of the embodiment of the present application;

[0055] Figure 4 The schematic diagram of the OPM sensor array of the embodiment of the present application;

[0056] In the figure: 1-magnetic shielding room, 2-background magnetic field sensor array, 3-OPM sensor array, 4-flexible brain magnetic acquisition cap, 5-sensor fixing base, 6-vacuum rubber tube and sealed earphone (used for MEG), 7-editing and generating auditory stimulation module (used for MEG), 8-LabVIEW controller, 9-magnetic shielding coil drive, 10-acquisition system, 11-editing and generating auditory stimulation module (used for fMRI), 12-scan room, 13-vacuum rubber tube and sealed earphone (used for fMRI), 14-magnetic resonance imaging machine, 15-radio frequency coil, 16-gradient coil, 17-magnetic resonance imaging console, 18-functional magnetic resonance imaging data preprocessing system, 19-bimodal data joint analysis system, 20-brain magnetic data preprocessing system, 21-active compensation coil. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0058] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0059] Embodiment one

[0060] A kind of auditory nerve representation device based on magnetoencephalogram and functional magnetic resonance imaging, as shown in Figure 1 Specifically comprising: auditory stimulation system, magnetic shielding system, head-mounted array sensor system, magnetoencephalogram signal acquisition system, functional magnetic resonance imaging signal acquisition system, magnetoencephalogram data preprocessing system, functional magnetic resonance imaging data preprocessing system, dual-mode data joint analysis system.

[0061] The auditory stimulation system is used to generate OPM-MEG signals and fMRI signals, wherein the OPM-MEG signals are magnetoencephalography signals, and the fMRI signals are functional magnetic resonance signals; the auditory stimulation system provides auditory stimulation for a subject, and mainly comprises an editing and generating auditory stimulation module and a stimulation presentation device, wherein the editing and generating auditory stimulation module is a computer; the editing and generating auditory stimulation module is used to compile and execute a required stimulation paradigm, and the paradigm comprises six different kinds of sound stimulation, i.e., human language (such as "you", "good"), human emotional sound (such as crying, sighing), animal call (such as horse call, sheep call), natural environment (such as wind, rain), life scene (such as knocking on the door, clinking a cup), and musical instrument (such as erhu, violin), and each category has 15 specific sound stimulation; the duration of the stimulation is approximately the same and is normalized to the same amplitude level, all the stimulation is presented in a random order and the volume is adjusted to a comfortable level, i.e., about 65 dB sound pressure level; the generated stimulation is played by Psychtoolbox (MATLAB toolbox) installed in the editing and generating auditory stimulation module and is transmitted to the subject through a vacuum rubber tube and sealed earphones, and meanwhile, the editing and generating auditory stimulation module is connected with the magnetoencephalography / functional magnetic resonance imaging signal acquisition system; when the sound stimulation is played, the editing and generating auditory stimulation module synchronously generates a square wave current pulse signal (trigger signal) and sends the signal to the magnetoencephalography / functional magnetic resonance imaging signal acquisition system, so as to mark the trigger signal on the OPM-MEG and fMRI signals, thereby facilitating subsequent alignment of the collected data and the sound stimulation time; the stimulation presentation device comprises the vacuum rubber tube and the sealed earphones, which are used to transmit the sound stimulation to the subject and also used to isolate the auditory system of the subject, so as to reduce the noise and interference of the external environment.

[0062] The magnetic shielding system is used to avoid the generated magnetoencephalography signals from being disturbed by external magnetic fields; the magnetic shielding system uses magnetic materials and special structures to block the influence of external magnetic fields, so as to ensure that the measured signals mainly come from the brain activity of the measured object rather than external interference sources; the system mainly comprises a passive shielding module and an active shielding module; the passive shielding module is a multi-layer shielding layer composed of high-permeability alloy and high-conductivity alloy, and the main way is to twist the magnetic field and attract the magnetic flux into the alloy, so as to reduce the influence of the magnetic field on the internal space of the magnetic shielding room; the active shielding module is composed of a background magnetic field sensor array and an active compensation coil, wherein the background magnetic field sensor array is composed of four OPM sensors, each sensor has two sensitive measurement axes, i.e., the axial direction (such as a in Figure 3 the figure) of the sensor and the tangential direction (such as b in the figure) of the bottom surface; the active compensation coil is used to generate a magnetic field that is opposite to the external magnetic field, so as to cancel out the influence of the external magnetic field on the measured signals. Figure 3 ​The four sensors are divided into two groups, each group having two sensors and being placed perpendicular to each other, and are placed on the left and right sides of the subject's brain, respectively, and since the relative position and orientation of the reference sensor array are known, the three components of the magnetic field can be measured using the four sensors and calculating , The compensation coils are controlled using a LabVIEW controller, which has built-in dynamic feedback controllers based on PID. These controllers calculate the corresponding compensation current by observing the response of the system to proportional and integral gain changes and drive the active compensation coils to generate a compensation current, thereby generating a magnetic field equal and opposite to the reference sensor array, achieving the effect of active shielding.

[0063] The head-mounted array sensor system is used to collect and generate magnetoencephalogram signals; the head-mounted array sensor system captures and records the weak magnetic field signals generated by brain activity. The system mainly includes: a flexible magnetoencephalogram acquisition cap, an OPM sensor array, and a sensor fixing base. The flexible magnetoencephalogram acquisition cap has a series of holes according to the 10-20 international standard lead system, and the sensor fixing base is fixed on the hole. The acquisition cap is connected with the OPM sensor array through the fixing base. Since the holes set by the 10-20 international standard lead system almost cover the entire scalp surface, researchers can flexibly arrange the sensors in the brain area of interest according to different research problems. The sensor array is used to record the magnetic field intensity of different brain sites of the subject under auditory stimulation. At the same time, the OPM sensor array is also connected with the acquisition system for sending the collected magnetoencephalogram signals to the magnetoencephalogram signal acquisition system.

[0064] The magnetoencephalogram signal acquisition system is used to receive magnetoencephalogram signals; the magnetoencephalogram signal acquisition system is used to collect a series of magnetic field signals of different time and different brain sites collected by the sensor array. The acquisition system is a digital acquisition system (DAQ) of the National Instruments Corporation (NI), which is used to record the signals of the OPM sensor array. Under the stimulation of the above auditory stimulation system, the subject's brain will produce corresponding neuronal electrical activity, causing current to flow in the tissue around the neuron. According to the Ampere loop law, the current flowing through the conductor will generate a magnetic field, i.e. the MEG signal. This magnetic field is extremely weak, but it is still measurable on the brain surface using OPM sensors. The OPM sensor uses laser to excite atoms. In the excited state, the electron spin of the atom will precess. The frequency of this spin precession is affected by the external magnetic field, i.e. the magnetic resonance frequency. The presence of the external magnetic field affects the spin state of the atom. The greater the magnetic field strength, the more obvious the spin state change. When the strength of the external magnetic field changes, the spin state of the atom will also change. The OPM sensor measures the change in the spin state of the atom to infer the strength and direction of the external magnetic field, thereby quantitatively measuring the magnetic field, i.e. the OPM-MEG signal.

[0065] The functional magnetic resonance imaging (fMRI) signal acquisition system is used to receive fMRI signals. It records hemodynamic changes induced by neuronal activity under the auditory stimulation system and obtains reconstructed three-dimensional head anatomical images of the subject. Under stimulation, neuronal activity increases in certain brain regions, leading to increased energy demand in those regions. To meet this demand, local blood flow rapidly increases. When blood flow increases, oxygen delivery exceeds the brain tissue's oxygen requirements, resulting in elevated local oxygenation levels. High oxygenation levels cause local magnetic field distortion, producing blood oxygen level-dependent (BOLD) signals. The main components of BOLD signals are oxyhemoglobin and deoxyhemoglobin signals, which possess different magnetic properties in a magnetic field. fMRI utilizes techniques such as strong magnetic fields and gradient magnetic fields to measure magnetic resonance signals within the body, generating images of tissue structures and specific signals. Continuous temporal image acquisition generates a series of BOLD signals for brain regions. By using the built-in software of the fMRI console, multiple image sequences can be registered and stacked to generate reconstructed three-dimensional head anatomical images of the subject. The system includes: a magnetic resonance imaging (MRI) console, an MRI machine, radio frequency (RF) coils, and gradient coils. Operators use the MRI console to set various scanning parameters and monitor the scanning process in real time. The MRI machine is the main component of the MRI system, generating images of the internal structures of the human body based on the scanning parameters. The RF coils are used to send radio frequency pulses to the subject's body parts to excite resonance signals and receive signals for imaging. The gradient coils are used to introduce gradient magnetic fields in different directions to locate the signal source within the subject's body.

[0066] All of the systems described above need to be prepared before the subject begins acquiring signals. Similarly, the functional magnetic resonance imaging signal acquisition system also needs to be prepared before the subject begins acquiring signals.

[0067] The brain magnetic data preprocessing system is used for preprocessing the OPM-MEG signal to obtain brain magnetic processing data; after the acquisition is completed, the magnetoencephalogram signal acquisition system sends the brain magnetic data to the brain magnetic data preprocessing system, and the system is used for filtering, average reference, segmentation, baseline correction, bad segment rejection, artifact removal and other preliminary processing of the collected OPM-MEG signal. First, 1Hz high-pass filtering and 40Hz low-pass filtering are performed, which can remove low-frequency drift below 1Hz and noise above 40Hz, and retain the signal in the frequency range of interest. However, due to factors such as resistance between the scalp and the brain and uneven transmission medium, some noise (artifacts) related to the head surface is contained in the recorded signal. In order to eliminate or reduce these artifacts, average reference is performed, the average value of the signals of all other electrodes is taken, and this average value is taken as the signal of the reference electrode. The average value is subtracted from the signal of each electrode to reduce the influence of head surface artifacts. In order to study the stimulus-induced MEG response, the data before and after the start of the stimulus event are segmented, so as to extract the change of MEG activity after the presentation of the stimulus. According to the above trigger signal, the start point of the appearance of the sound stimulus is determined and defined as the "0 time point", and the brain magnetic data is divided into multiple data segments, such as 200ms before the stimulus to 800ms after the stimulus. In order to eliminate the noise caused by spontaneous brain waves, the average baseline of each point of the segmented data is subtracted, and the average baseline is the average value of the MEG signal in the time period before the occurrence of the stimulus. After the above segmentation is completed, the waveform of the data is observed, and it is checked whether there are obvious residual artifacts in all segments and manually rejected bad segments. Finally, the segmented data is subjected to independent component analysis (ICA). ICA decomposes the signal into multiple independent source signals through a method of solving linear equations. According to the topographic map, inter-trial distribution map and frequency distribution map of the decomposed components, the relevant components of the artifacts are identified. Finally, the processed data is sent to the dual-mode data joint analysis system.

[0068] The functional magnetic resonance imaging data preprocessing system is used for preprocessing the fMRI signal to obtain functional processing data; after the acquisition is completed, the functional magnetic resonance imaging signal acquisition system sends the functional nuclear magnetic data to the functional magnetic resonance imaging preprocessing system, and the system is used for time registration, head motion correction, spatial registration, smoothing, cerebrospinal fluid signal removal and other preliminary processing of the collected fMRI signal. Time registration adjusts the acquisition time of the slice by using interpolation technology, so that the signals of each slice are aligned in time, the purpose is to ensure that the fMRI data of the subject at different time points is aligned in time domain, which helps to obtain accurate event-related response. The motion parameter estimation algorithm is used for head motion correction, the position of each brain voxel at each time point is corrected to the reference frame, the influence caused by the head motion of the subject during the scanning process is corrected, and the motion artifact is reduced. Spatial registration is performed by using linear or nonlinear transformation to map the brain image to the standard space, such as MNI (Montreal Neurological Institute) space, and the process aims to map the brain images of different individuals or different time points to a common coordinate system for group statistical analysis or subject comparison. Smoothing operation is performed by applying filtering methods such as Gaussian kernel function to smooth the spatial noise in the data, reduce the spatial noise, and make the signal more significant, which helps subsequent statistical analysis. The extracted cerebrospinal fluid signal is used as a covariate, and linear regression is performed to remove the cerebrospinal fluid signal, so that the influence of the cerebrospinal fluid signal on the fMRI signal is removed, and the influence of the ventricle effect is reduced. After the above preliminary processing is completed, the processed data is sent to the dual-mode data joint analysis system.

[0069] The dual-modal data joint analysis system is used for receiving and fusing the brain magnetic processing data and the functional processing data, using a support vector machine classifier to construct a diagonal symmetric representation dissimilarity matrix (RDM) of stimulus two-by-two mutual classification, and obtaining an auditory nerve representation according to the diagonal symmetric representation dissimilarity matrix of stimulus two-by-two mutual classification; the dual-modal data joint analysis system is used for accepting preprocessed fMRI data and OPM-MEG data and performing subsequent analysis. The preprocessed MEG data is subjected to time-frequency analysis based on a sliding window. Specifically, a window function of a limited length is first selected, the window is placed on the signal from the starting point of the signal, the signal is segmented and weighted by the window to generate a series of short data segments, the spectrum of the windowed short data segments is calculated using an autoregressive method, and the window is slid along the time axis to repeatedly calculate the spectrum until the window reaches the end of the signal.After time-frequency analysis obtains the time-frequency spectrum, t-test and multiple comparison correction are performed on each time point and frequency point to obtain the significant difference level after multiple comparison correction, so as to find the significant stimulation activation time period and frequency period; trace analysis is performed combined with OPM-MEG and MRI data to locate the source and find and compare the differences of brain regions activated by different stimuli. First, the subject is scanned by the magnetic resonance imaging signal acquisition system to obtain the structural MRI data of the subject to obtain the anatomical information of the head, then the MRI data is segmented using the computational biophysics software FreeSurfer to create a head model including the geometric shape and conductivity distribution of the skull, cerebrospinal fluid and brain tissue, then the subject wearing the sensor headgear is scanned using a laser scanner to obtain a scan image, then the sensor array of the OPM-MEG is aligned with the above head model using the existing sample consensus initial alignment algorithm (SAC-IA) and the Iterative Closest Point (ICA), then the activity of the brain source is reconstructed based on the head surface topology map generated by the MEG signal using the MNE-python toolkit, and the source positioning result is visualized on the MRI image of the subject to visually show the position and intensity of the brain activation source under the stimulation, and the difference of the activated brain regions under different stimulation conditions can be obtained by t-testing the source activity; similarity analysis is performed by fusing OPM-MEG and fMRI data, the difference of the neural representation result of the brain is decoded by comparing the representation dissimilarity matrices under different types of stimuli. First, a support vector machine (SVM) classifier is used to classify all possible two sound stimulus pairs (i.e. decoding), a diagonal symmetric representation dissimilarity matrix is constructed for all stimulus two-way classification, and each element of the matrix represents the decoding accuracy of the classifier in distinguishing two sounds. Decoding along the time axis of the OPM-MEG segmented signal at each time point can obtain a series of time point RDMs, and these matrices represent the time processing process of the brain to different sound stimuli. Decoding each voxel of the activated brain region obtained in the trace analysis step on the fMRI signal can obtain the RDM of the different activated brain regions, then the Pearson correlation coefficient of the RDM of the different activated brain regions and the series of time point RDMs in the time sequence is calculated to obtain the decoding of the different brain regions at different time points, which reveals the dynamic representation of the different brain regions. By t-testing the dynamic representation of the different brain regions under different sound stimuli, the difference in the response process of the subject to different types of stimuli under OPM-MEG and fMRI data can be found.

[0070] As shown in Figure 2 , the background magnetic field sensor array 2 is placed in the magnetic shield room 1, which is composed of four OPM sensors, each sensor has two sensitive measurement axes, which are the axial and tangential of the sensor base, respectively, the four sensors are divided into two groups, each group has two sensors and is placed perpendicular to each other, respectively, on the left and right sides of the subject's brain, the sensor array can measure the residual magnetic field in the room in real time and send it to the LabVIEW controller 8, which has a dynamic feedback algorithm based on PID, which can calculate the feedback current in real time and send it to the magnetic shield coil drive 9 via the acquisition system 10, the active compensation coil 21, to achieve active residual magnetic compensation.

[0071] Before acquisition, the OPM sensor array 3 needs to be fixed on the flexible brain magnetic acquisition cap 4 through the sensor fixing base 5, and the flexible acquisition cap is provided with a series of holes at different sites according to the 10-20 international standard lead system, which facilitates the arrangement of different sensor array layouts according to different purposes.

[0072] The scanning room 12 is used to wrap the entire fMRI scanner to prevent external magnetic fields from interfering with the measurement, and the magnetic resonance imaging machine 14 is placed inside for generating high-resolution, clear images of internal human structures. The magnetic resonance imaging machine is built-in with gradient coils 16 and radio frequency coils 15, which are used to generate controllable magnetic field gradients and transmit radio frequency pulses and receive nuclear magnetic resonance signals during scanning, respectively. In addition, the magnetic resonance imaging console 17 is used to set various parameters of the fMRI scan and monitor the scanning process in real time.

[0073] The auditory stimulus editing and generating modules 7 and 11 produce the required sound stimuli and deliver them to the subjects via vacuum rubber tubes and sealed earphones 6 and 13, in addition, the auditory stimulus editing and generating modules 7 and 11 will generate a square wave current pulse signal (trigger signal) simultaneously when playing the sound stimulus and send the signal to the respective acquisition system 10 and magnetic resonance imaging console 17. After acquisition, the respective data and trigger signals are sent to the magnetoencephalographic data preprocessing system 20 and the functional magnetic resonance imaging data preprocessing system 18, respectively, which perform a series of preprocessing on the respective data. After preprocessing, the data is sent to the dual-mode data joint analysis system 19, which performs time-frequency analysis, source analysis, and characterization similarity analysis on the data and performs statistical tests.

[0074] As shown in Figure 3 , the background magnetic field sensor array is composed of four OPM sensors, each sensor has two sensitive measurement axes, which are the axial and tangential of the sensor base, respectively, the four sensors are divided into two groups, each group has two sensors and is placed perpendicular to each other, respectively, on the left and right sides of the subject's brain, the sensor array can measure the residual magnetic field in the room in real time and send it to the LabVIEW controller 8, which has a dynamic feedback algorithm based on PID, which can calculate the feedback current in real time and send it to the magnetic shield coil drive 9 via the acquisition system 10, the active compensation coil 21, to achieve active residual magnetic compensation. Figure 3 sensitive axis) and the tangential of the base (as shown in b of Figure 3 , respectively.​ The four sensors are divided into two groups, each group having two sensors and being placed perpendicular to each other, and are placed on the left and right sides of the brain of the subject, and since the relative position and orientation of the known reference sensor array are known, the three components of the magnetic field can be measured using the four sensors , and the calculation , .

[0075] As shown in Figure 4 , the flexible brain magnetic acquisition cap is provided with a series of holes according to the 10-20 international standard lead system, and the holes are provided with sensor fixing bases, and the acquisition cap is connected with the OPM sensor array through the fixing bases, and since the holes provided by the 10-20 international standard lead system almost cover the entire scalp surface, the sensors can be flexibly arranged in the brain region of interest according to different research problems.

[0076] The above-described embodiments are only descriptions of the preferred modes of the present application, and are not intended to limit the scope of the present application, and various modifications and improvements to the technical solutions of the present application made by those of ordinary skill in the art without departing from the design spirit of the present application shall fall within the protection scope determined by the claims of the present application.

Claims

1. A device for auditory nerve representation based on magnetoencephalography and functional magnetic resonance imaging, characterized by, The device comprises: an auditory stimulation system, a head-mounted array sensor system, a magnetoencephalogram signal acquisition system, a functional magnetic resonance imaging signal acquisition system, a magnetoencephalogram data preprocessing system, a functional magnetic resonance imaging data preprocessing system, and a bimodal data joint analysis system. The auditory stimulation system is used to generate OPM-MEG signals and fMRI signals, wherein the OPM-MEG signals are magnetoencephalogram signals, and the fMRI signals are functional magnetic resonance signals. The head-mounted array sensor system is used to acquire the generated magnetoencephalogram signals. The magnetoencephalogram signal acquisition system is used to receive the magnetoencephalogram signals. The functional magnetic resonance imaging signal acquisition system is used to receive the functional magnetic resonance signals. The magnetoencephalogram data preprocessing system is used to preprocess the magnetoencephalogram signals to obtain magnetoencephalogram processing data. The functional magnetic resonance imaging data preprocessing system is used to preprocess the functional magnetic resonance signals to obtain functional processing data. The bimodal data joint analysis system is used to receive and fuse the magnetoencephalogram processing data and the functional processing data, use a support vector machine classifier to construct a diagonal symmetric representation dissimilarity matrix for classifying two stimuli, and obtain auditory nerve representation results according to the diagonal symmetric representation dissimilarity matrix. The workflow of the bimodal data joint analysis system comprises: Performing time-frequency analysis on the magnetoencephalogram processing data based on a sliding window to obtain a time-frequency spectrum, performing t-test and comparison correction on each time point and frequency point according to the time-frequency spectrum to obtain a significant difference level, and obtaining a stimulus activation time period and a frequency period according to the significant difference level; Based on the stimulus activation time period and the frequency period, combining the magnetoencephalogram processing data and the functional processing data to perform activation source positioning to obtain and compare differences in activated brain regions of different stimuli; According to the magnetoencephalogram processing data, using a support vector machine classifier to construct a diagonal symmetric representation dissimilarity matrix for classifying two stimuli, calculating the Pearson correlation coefficient of the representation dissimilarity matrix of different activated brain regions and a series of time point representation dissimilarity matrices in a time sequence to obtain dynamic representation of different brain regions at different time points, performing t-test on the dynamic representation to obtain nerve representation results of different brain regions of the brain. 2.The magnetoencephalography and functional magnetic resonance imaging based auditory nerve characterization apparatus of claim 1, wherein, The auditory nerve representation device further comprises a magnetic shielding system, which is used to avoid interference of generated magnetoencephalogram signals by external magnetic fields, and the magnetic shielding system further comprises a passive shielding module and an active shielding module. The passive shielding module is a multilayer shielding layer composed of high-permeability alloy and high-conductivity alloy, which twists the magnetic field, attracts the magnetic flux into the alloy, and reduces the influence of the magnetic field on the internal space of the magnetic shielding chamber. The active shielding module is composed of a background magnetic field sensor array and an active compensation coil, and a LabVIEW controller is used to control the compensation coil, the LabVIEW is built-in with a dynamic feedback controller based on PID, the controller calculates the corresponding compensation current through the response of the proportional gain and integral gain of the system, and drives the active compensation coil to generate a compensation current, generates a magnetic field equal and opposite to the reference sensor array, and realizes active shielding. 3.The magnetoencephalography and functional magnetic resonance imaging based auditory nerve characterization apparatus of claim 1, wherein, The auditory stimulation system comprises an editing and generating auditory stimulation module and a stimulation presentation device; The editing and generating auditory stimulation module is used for editing and executing a preset auditory stimulation paradigm; The stimulation presentation device comprises a vacuum rubber tube and a sealed earphone, which is used for transmitting sound stimulation to the subject while isolating the auditory system of the subject.

4. The device for auditory nerve characterization based on magnetoencephalography and functional magnetic resonance imaging according to claim 3, characterized in that, The auditory stimulation paradigm comprises six different kinds of sound stimulation, including human language, human emotional sound, animal call, natural environment sound, life scene sound and musical instrument sound; when the sound stimulation is played, the editing and generating auditory stimulation module synchronously generates a square wave current pulse signal, and sends the square wave current pulse signal to a magnetoencephalogram imaging signal acquisition system or a functional magnetic resonance imaging signal acquisition system, and marks the square wave current pulse signal on the magnetoencephalogram signal and the functional magnetic resonance signal. 5.The magnetoencephalography and functional magnetic resonance imaging based auditory nerve characterization apparatus of claim 1, wherein, The head-mounted array sensor system comprises a flexible magnetoencephalogram acquisition cap, an OPM sensor array and a sensor fixing base; the flexible magnetoencephalogram acquisition cap is provided with holes according to the international standard lead system, the sensor fixing base is fixed on the holes, the flexible magnetoencephalogram acquisition cap is connected with the OPM sensor array through the sensor fixing base, and the OPM sensor array is used for recording the magnetic field intensity of different brain sites of the subject under auditory stimulation. 6.The magnetoencephalography and functional magnetic resonance imaging based auditory nerve characterization apparatus of claim 1, wherein, The functional magnetic resonance imaging signal acquisition system comprises a magnetic resonance imaging console, a magnetic resonance imaging machine, a gradient coil and a radio frequency coil; The magnetic resonance imaging console is used for controlling the parameters of MRI scanning, collecting and storing imaging data, and being connected with a preprocessing module in a functional magnetic resonance imaging data preprocessing system; The magnetic resonance imaging machine is used for generating high-resolution brain structure and function images; The gradient coil is used for generating spatial gradients in magnetic resonance scanning; The radio frequency coil is used for transmitting and receiving radio frequency signals, is responsible for exciting functional magnetic resonance signals in brain tissue, and captures the functional magnetic resonance signals for image reconstruction. 7.The magnetoencephalography and functional magnetic resonance imaging based auditory nerve characterization apparatus of claim 1, wherein, The magnetoencephalogram processing data is subjected to time-frequency analysis based on a sliding window, and a time-frequency spectrum is obtained, which comprises: A window function with a preset length is selected, the window is placed on the signal from the starting point of the signal, the signal is segmented and weighted by the window, and a short data segment is generated; The spectrum of the windowed short data segment is calculated by using an autoregressive method, and the window is slid along the time axis, and the spectrum is repeatedly calculated until the window reaches the end of the signal, and a time-frequency spectrum is obtained. 8.The magnetoencephalography and functional magnetic resonance imaging based auditory nerve characterization apparatus of claim 1, wherein, Based on the stimulation activation time period and frequency period, the activated source positioning is carried out by combining the magnetoencephalogram processing data and the functional processing data, and the differences of the activated brain regions of different stimulations are obtained and compared, which comprises: The subject is scanned by a magnetic resonance imaging signal acquisition system to obtain structural MRI data of the subject to obtain anatomical information of the head; The MRI data is segmented using a computational biophysical software, FreeSurfer, to create a head model; The subject wearing the sensor headgear is scanned using a laser scanner to obtain scan images, and a sensor array of the OPM-MEG is aligned with the head model using a sample consistency initial registration algorithm and an iterative closest point method; The activity of the brain source is reconstructed using a head surface topology map generated based on an MNE-python toolkit through the MEG signal, and the source positioning result is visualized on the MRI image of the subject to show the position and intensity of the activated brain source under the stimulation, and the difference in the activated brain region under different stimulation conditions is obtained by performing a t-test on the source activity.

Images