Brain cognitive function analysis method based on EEG-fNIRS cross-modal multilayer network

Abstract

The invention discloses a brain cognitive function analysis method based on an EEG-fNIRS cross-modal multilayer network. The method comprises the following steps: firstly, collecting multichannel EEG and fNIRS signals under a working memory training normal form, and preprocessing the EEG and fNIRS signals; source space reconstruction is carried out on the signals, corresponding position information is distributed for EEG and fNIRS source time sequences, and regions of interest are divided based on a Desikan-Killiany atlas; the method comprises the following steps: quantizing decomposable directed mutual information between systems by using IID-PSIT, and constructing an EEG-fNIRS multiplexing network; quantizing cross frequency coupling in the brain function network by using GCSIT, and constructing an EEG-fNIRS full-connection multi-layer network; topological characteristics of a multiplexing network and a multi-layer network are analyzed and extracted by using graph theory parameters, brain function changes under different cognitive loads are revealed, and the accuracy of cognitive load classification tasks is improved.

Description

Technical Field

[0001] This invention relates to a method for multimodal analysis and identification of brain cognitive function under the working memory training paradigm, and particularly to a method for analyzing brain cognitive function based on the EEG-fNIRS cross-modal multilayer network. Background Technology

[0002] Neurovascular coupling (NVC) indicates that regional neural activity is often accompanied by corresponding metabolic changes and electrical activity. Working memory, as a crucial foundation of higher cognitive functions, is closely linked to NVC. Impaired NVC function in aging or neurodegenerative diseases can lead to working memory decline. Therefore, accurately measuring the mechanisms of NVC in working memory is of practical necessity and research significance. Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), as effective tools for monitoring neural oscillatory activity and hemodynamic responses, can accurately detect NVC. NVC in working memory is highly dynamic, involving high-intensity neural activity in brain regions such as the prefrontal cortex and parietal lobe. Therefore, elucidating the information transmission mechanisms between neuronal activity in multiple brain regions and blood oxygen metabolism in working memory by exploring the causal relationships of multimodal signals is crucial.

[0003] Functional connectivity is a measure of the interactions between neural activities in brain regions. Graph theory, which represents the brain as a network of nodes and edges, has become an important tool for modeling functional connectivity. Graph theory methods can be used to model the functional networks of neuroimaging techniques such as EEG and fNIRS during working memory.

[0004] However, traditional functionally connected networks can only characterize single-frequency band or single-modal features, lacking the ability to model multiple interactions from temporal, frequency, or modal perspectives. Therefore, a brain cognitive function analysis method based on EEG-fNIRS cross-modal multilayer networks is needed. Multilayer brain networks can integrate systems at different frequencies or modalities, where network nodes represent the same entity, i.e., brain regions at different layers. Currently, multilayer network theory has been applied to brain function impairment research in various mental illnesses, providing a novel tool for identifying disease-specific cross-layer network decoupling patterns and analyzing network abnormalities in mental illnesses. Summary of the Invention

[0005] To address the shortcomings of existing technologies, a brain cognitive function analysis method based on EEG-fNIRS cross-modal multilayer network is provided, which can integrate systems at different frequencies or modalities.

[0006] To achieve the above objectives, the following technical solution is provided: This invention addresses the ambiguity of causal relationships that may arise from the spatiotemporal heterogeneity of cross-modal signals in current single-scale EEG-fNIRS neurovascular coupling analysis, and the limitations of traditional causal analysis frameworks in resolving complex networks involving multi-brain regions and multi-frequency band coupling. By introducing multilayer network theory, this invention proposes a multilayer network analysis method for EEG-fNIRS based on Integrated Information Decomposition-Phase Specific Information Transfer (IID-PSIT) and Gaussian Copula Specific Information Transfer (GCSIT).

[0007] According to the invention, the specific steps include:

[0008] Step 1: Simultaneously acquire multi-channel EEG and fNIRS signals under the working memory training paradigm using EEG and fNIRS acquisition devices;

[0009] Step 2: Preprocess the EEG and fNRIS signals;

[0010] Step 3: Reconstruct the source space of the signal, assign corresponding location information to the EEG and fNIRS source time series, and divide the region of interest (ROI) based on the Desikan-Killiany map;

[0011] Step 4: Construct an EEG-fNIRS multiplexing network by quantizing the decomposable directed mutual information between the IID-PSIT systems.

[0012] Step 5: Quantify the cross-frequency coupling in the brain functional network using GCCSIT, and construct an EEG-fNIRS fully connected multilayer network;

[0013] Step 6: Use graph theory parameter analysis to extract the topological features of the multiplexed network and multilayer network from Step 4 and Step 5;

[0014] Step 7: Use a support vector machine as a classifier to perform binary classification for different cognitive load levels.

[0015] As a preferred method, step 3 maps the original signal into 3D space by tracing the source and locating it;

[0016] The specific steps include the following:

[0017] Step 3-1: For EEG signals, the boundary element method is used to map the dipoles corresponding to the potential brain signal sources onto the cerebral cortex. The scalp surface is segmented into a high-resolution mesh with 15,000 vertices. Finally, a forward model is generated based on the volume conduction model.

[0018] Step 3-2: For the fNIRS signal, the sensitivity matrix is ​​calculated using the Colin27 template. The effect of each photodiode is estimated using Monte Carlo simulation. Sensitivity values ​​are projected onto each vertex of the cortical grid, and the sensitivity values ​​of each voxel in each channel are further obtained, thereby calculating the fNIRS forward model.

[0019] Step 3-3: Based on the obtained forward model, the inverse problems of EEG and fNIRS signals were calculated using the weighted minimum norm (wMNE) estimation and sLORETA respectively.

[0020] Preferably, step 4 includes:

[0021] Step 4-1: Quantify the mutual information between the past and future states of the system using Time-delayed Mutual Information (TDMI) to determine the information flow direction. IID theory can refine TDMI to the 16 atoms of the IID lattice, thus accurately measuring the interaction patterns between signals. The IID-PSIT method is proposed to quantify the decomposable directed mutual information between systems:

[0022]

[0023] Where φ(t) represents the phase information extracted through Hilbert transform, and τ is the phase lag coefficient. Syn represents the cooperative information in the system, Red represents the redundancy information, and Un represents the unique information.

[0024] Step 4-2: Use a modified comprehensive information metric to calculate the functional connectivity between ROIs, reflecting the information gain or loss brought about by the overall synergy of the system:

[0025]

[0026] Where i and j are time series X and Y;

[0027] Step 4-3: Use a data-driven thresholding method to obtain a binary network from the obtained weighted coupling matrix, and construct a multi-layer network model based on multiplexing network theory, allowing only cross-layer connections of the same node between layers.

[0028] Preferably, step 5 includes:

[0029] Step 5-1: Propose the GCSIT method, where each multivariate cumulative distribution function (CDF) F(x1,…,x) k )=P(X1≤x1,…,X k ≤x k All of them can use their marginal CDFF i (x)=P(X i ≤x) and the copula function C:[0,1) k →[0,1] means:

[0030] F(x1,…,x k )=C(F1(x1),…,F k (x k (3)

[0031] Since the negative copula entropy is equivalent to mutual information, the GCSIT between random variables X and Y can be obtained as follows:

[0032]

[0033] Where u and v are the CDF and GC of random variables X and Y under time delay τ, respectively. atom To integrate the different atomic components obtained from the information decomposition, all represents all atomic components except for the atoms on the left. Φ under the GCSIT method WMS,R The indicators constitute the functional connectivity network W for each characteristic signal. M ;

[0034] Step 5-2: Inter-layer coupling between nodes is analyzed using the amplitude-amplitude coupling method. The instantaneous amplitude of each characteristic signal is obtained through Hilbert transform, and the Pearson correlation coefficient between each amplitude feature is calculated, resulting in the following fully connected matrix W:

[0035]

[0036] in and This represents the thresholded intra- and inter-layer coupled network, where M is the number of network layers.

[0037] In summary, the above technical solution has the following beneficial effects:

[0038] Traditional functional connectivity analysis methods cannot integrate multi-scale and multimodal information to understand brain mechanisms, and commonly used correlation metrics lack the ability to subdivide information flow, making it difficult to reveal the complex network characteristics of multi-brain region and multi-frequency band coupling in EEG-fNIRS neurovascular coupling. To address these issues, this invention proposes a multi-layer network analysis method for EEG-fNIRS based on IID-PSIT and GCSIT, constructing a bimodal multi-layer network for EEG-fNIRS. By analyzing and extracting the topological features of cross-frequency band multiplexing networks and cross-frequency coupling multi-layer networks, the changes in brain function under different cognitive loads are quantified, and the accuracy of cognitive load classification tasks is improved. Attached Figure Description

[0039] Figure 1 This is a flowchart of the present invention;

[0040] Figure 2 3D imaging of the distribution and source localization signals of the cerebral cortex channels;

[0041] Figure 3 For EEG-fNIRS fully connected multilayer network;

[0042] Figure 4 Comparison of clustering coefficients for single-layer networks;

[0043] Figure 5 Comparison of network density for single-layer networks;

[0044] Figure 6 Comparison of feature path lengths for single-layer networks;

[0045] Figure 7 For a comparison of the global efficiency of a single-layer network;

[0046] Figure 8 For comparing the multiple participation coefficients of multi-layer networks;

[0047] Figure 9 Comparison of multiple clustering coefficients in multi-layer networks;

[0048] Figure 10 The working characteristic curve for the cognitive load of the first classification task;

[0049] Figure 11 The working characteristic curve for the second type of cognitive load classification task. Detailed Implementation

[0050] The present invention will be further described below with reference to specific embodiments. The following description is for illustrative and explanatory purposes only and does not constitute any limitation on the present invention.

[0051] like Figure 1As shown, the implementation steps of the embodiments of the present invention are as follows:

[0052] Step 1: Use a synchronous acquisition device to acquire EEG and fNIRS signals in the N-back paradigm. The sampling frequency of the EEG signal is 1000Hz and the sampling frequency of the NIRS signal is 10.4Hz.

[0053] Step 2: Preprocess the EEG and fNIRS signals. The specific steps are as follows:

[0054] Step 2-1: Downsample the raw EEG data to 200Hz. Perform bandpass filtering from 1-45Hz using a 6th-order zero-phase Butterworth filter, and remove power line noise using a notch filter at 50Hz. Eliminate common-mode noise using averaged rereference. Use independent component analysis to remove eye movement, head movement, and power line interference. The filtered data is segmented into time series from -5 seconds to 60 seconds, and baseline correction is performed by subtracting the average value from -5 to -2 seconds.

[0055] Step 2-2: Because the acquired light intensity signal is significantly affected by light source fluctuations and instrument noise, it is necessary to convert the light intensity signal into an optical density (OD) signal to eliminate baseline drift. The formula for calculating optical density is:

[0056]

[0057] Among them, I λ This represents the light intensity measured at wavelength λ. This refers to the light intensity measurement at a specific time point. By calculating the change in optical density, the dependence on incident light intensity and scattering parameters can be reduced, making it easier to obtain measurements of changes in blood oxygen concentration. Then, the change in hemoglobin concentration is calculated based on the Modified Beer-Lambert Law (MBLL). Using the known molar extinction coefficient ε and differential path length factor DPF related to wavelength and chromophore, the calculation matrix for the change in hemoglobin concentration is obtained:

[0058]

[0059] Wherein, λ1 and λ2 are wavelength parameters, typically one below the light absorption point and the other above it. In this invention, the wavelengths used for fNIRS acquisition are 760 nm and 850 nm, respectively.

[0060] The converted HbO and HbR concentration data were downsampled to 10 Hz. When measuring brain activity, fNIRS is susceptible to various noise interferences. Among these, the Mayer wave, caused by arterial vasomotor activity, has a frequency of approximately 0.1 Hz, which overlaps with the frequency range of hemodynamic signals related to cognitive activity. The presence of the Mayer wave may mask task-related components in the fNIRS signal. To remove the Mayer wave and other physiological noise, a 6th-order zero-phase Butterworth filter with a cutoff frequency of 0.2 Hz was used for low-pass filtering. The filtered data was segmented into time series from -5 seconds to 60 seconds and baseline correction was performed.

[0061] Step 3: Reconstruct the source space of the signal. The specific steps are as follows:

[0062] Step 3-1: For EEG signals, the boundary element method was used to map the dipoles corresponding to potential brain signal sources onto the cerebral cortex. The scalp surface was segmented into a high-resolution mesh with 15,000 vertices. Finally, a forward model was generated based on the volume conduction model. The fNIRS signal was used to calculate the sensitivity matrix using a Colin27 template. The effect of each photodiode was estimated using Monte Carlo simulation, and sensitivity values ​​were projected onto each vertex of the cortical mesh. The sensitivity values ​​of each voxel for each channel were further obtained to calculate the fNIRS forward model. Based on the obtained forward model, the inverse problems of EEG and fNIRS signals were calculated using wMNE estimation and sLORETA, respectively.

[0063] In wMNE, a weighting matrix W is introduced. S To compensate for the deviation. S It is a diagonal matrix constructed from matrix A, where the non-zero terms are inversely proportional to the norm of the source vector. In this invention, the weighting factor is set to 0.3. The solution to the EEG inverse problem obtained from wMNE is:

[0064]

[0065] Where A is the forward matrix, I n Let A be an n×n identity matrix, and y be the vector of the measured signal. λ is a constant value that serves to transform the singular matrix A. T A. Regularization.

[0066] The sLORETA algorithm is a noise-normalized Minimum Norm Estimation (MNE) method designed to provide accurate source localization through standardized minimum norm estimation. This method takes into account the variance of actual source activity and measurement noise, providing a more accurate estimate of active source signals. The solution to the fNIRS inverse problem obtained from sLORETA is:

[0067]

[0068] in, The minimum norm estimate provides the current density estimate at the l-th voxel. It is a resolution matrix The l-th diagonal block of (estimated variance of current density) is defined as A. T [AA T +λI] -1 λ is generally set to 1×10 by default. -5 ;

[0069] Step 3-2: As Figure 2 As shown, in order to directly compare the EEG and fNIRS source time series, this invention uses the Desikan-Killiany map to map the source signals into 3D space, and selects 34 regions of interest (ROIs) in Table 1 based on the brain region distribution characteristics of cognitive function.

[0070]

[0071] Table 1. ROIs selected based on the Desikan-Killiany map and their corresponding brain regions.

[0072] Step 4: Construct the EEG-fNIRS multiplexing network using the IID-PSIT method. The specific steps are as follows:

[0073] Step 4-1: Use the IID-PSIT method and the modified comprehensive information metric to calculate the functional connectivity between ROIs; yrt

[0074] Step 4-2: Construct a 34*34 symmetric weighted coupling matrix under different frequency bands and blood oxygenation characteristics, and use a data-driven thresholding method to obtain a binarized network;

[0075] Step 4-3: Construct a multi-layer network model based on multiplexing network theory. Only cross-layer connections of the same node are allowed between layers, resulting in a multiplexing network consisting of 34 ROI nodes and 5 layers of feature networks.

[0076] Step 5: Construct the EEG-fNIRS fully connected multilayer network using the GCSIT method based on Gaussian Copula. The specific steps are as follows:

[0077] Step 5-1: Use the GCSIT method and a modified comprehensive information metric to calculate the functional connectivity network W for each feature signal. M ;

[0078] Step 5-2: Employ amplitude-amplitude coupling to quantize the interlayer coupling H between nodes. MThe instantaneous amplitudes of each characteristic signal are obtained through Hilbert transform, and the Pearson correlation coefficients between each amplitude feature are calculated. Finally, the fully connected matrix W is obtained using the same thresholding method as in step 4-2, as shown below. Figure 3 As shown.

[0079] Step 6: Use graph theory parameter analysis to extract the topological features of the EEG-fNIRS multilayer network. The specific steps are as follows:

[0080] Step 6-1: For single-layer networks, clustering coefficients are used to reflect the overall clustering characteristics of the network, measuring the tightness of functional connections throughout the network, such as... Figure 4 As shown. Assume a node i in the network has k i There are 1 neighboring nodes, and the neighboring nodes form an E-group. i If there are 1 edge, then the clustering coefficient of node i is defined as:

[0081]

[0082] Network density reflects the richness of connections within a network. Higher network density indicates more channels involved in information transmission and stronger overall functional connectivity of the brain network. Figure 5 As shown. A network with N nodes and L edges has the following network density:

[0083]

[0084] Average path length is used to reflect the speed at which information is transmitted in brain networks, such as Figure 6 As shown. Defined as the average of the shortest path lengths between all node pairs, the calculation formula is:

[0085]

[0086] Global efficiency is used to measure the overall information transmission efficiency of brain networks. Higher global efficiency indicates greater overall functional coordination and flexibility of the brain. Figure 7 As shown. Local efficiency is the reciprocal of the average path length, that is:

[0087]

[0088] Step 6-2: For multi-layer networks, the Multiple Participation Coefficient (MPC) is used to measure the distribution characteristics of node degree across all layers, i.e.:

[0089]

[0090] in, Let i be the degree of node i in layer α, and ο iLet p be the sum of the degrees of node i across all layers, and M be the number of network layers. MPC reflects the brain's ability to participate in and integrate across different network layers, helping to locate key brain regions. i When p = 0, the information transmission of node i is concentrated in a single network layer; when p = 0, the information transmission of node i is concentrated in a single network layer. i When = 1, the functional connections of node i are evenly distributed across network layers, exhibiting strong cross-network information exchange capabilities. For example... Figure 8 As shown, the MPC of the entire network can be obtained by averaging the MPC of all nodes:

[0091]

[0092] like Figure 9 As shown, the Multiple Clustering Coefficient (MCC) is used to represent the tightness of local connections of brain nodes in a multi-layered network, reflecting the segregation characteristics of brain functions.

[0093]

[0094] in, This represents the connection between node j and node m in layer β. Similarly, by averaging the MCC of all nodes, we can obtain the MCC of the entire network:

[0095]

[0096] A modular metric for community structure in multi-scale networks is used to explain the overall function of the network, defined as:

[0097]

[0098] Where μ is the number of links in a fully connected network, γ is the resolution parameter, and W ijα The adjacency matrix of nodes within a layer, m α H represents the number of network edges in layer α. jαβ Let δ(g) be the connection between layer α and layer β for node j. If node i and node j are in the same community, then δ(g) iα ,g jβ The value is 1.

[0099] Step 7: Evaluate the performance of multi-layer topology features in distinguishing working memory load levels. The specific steps are as follows:

[0100] Step 7-1: Based on the topological features of the fully connected network, 69 features were extracted from 702 tasks of all subjects, including the multi-participation coefficient, multi-clustering coefficient and modularity of nodes. Considering that high-dimensional features require a large amount of computation and may lead to overfitting, the Lasso algorithm was used to screen the features.

[0101] Step 7-2: Select a Support Vector Machine (SVM) as the classifier. Use 5-fold cross-validation to obtain more reliable classification results. To evaluate classification performance, calculate the classification accuracy, sensitivity, and specificity.

[0102]

[0103]

[0104] This invention assumes that low cognitive load conditions are positive examples, TP and TN represent the number of correctly classified positive and negative examples, FN represents the number of positive examples classified as negative examples, and FP represents the number of negative examples classified as positive examples. Figure 10 , 11 As shown, the classifier's ability to classify positive and negative examples is further evaluated by calculating the operating characteristic curve and the area under the curve (AUC).

[0105] Table 2 shows the classification results of different topological features in the n-back task. The results show that, compared to multiple clustering coefficients and modularity, multiple participation coefficients have a stronger ability to identify cognitive load, achieving the highest classification accuracy of 95.94%. When all topological features are combined, the best performance is observed in the classification recognition under 0-back and 2-back conditions, with the classification accuracy increasing to 96.16%. In pairwise classification tasks, MPC values, MCC values, and combinations of all features are observed to be relatively effective classification features. Furthermore, except for the 0-back and 3-back classification tasks using feature combinations, the classification specificity is higher than the sensitivity in all cases, indicating that the topological features of multilayer networks are more sensitive to detecting high cognitive load states than low cognitive load states.

[0106]

[0107] Table 2. Classification results of different topological features in the n-back task.

[0108] Since all feature combinations achieved the best classification results, the classification performance of topological feature combinations under EEG-fNIRS, EEG, and fNIRS fully connected networks was further validated. As shown in Table 3, the feature combination of the EEG-fNIRS fully connected network achieved the highest classification accuracy in all classification tasks. Notably, in the classification between 0-back and 2-back, and 3-back, the classification accuracy of EEG network features is higher than that of fNIRS network features, but in the classification between 2-back and 3-back, fNIRS network features show better classification performance. This demonstrates the classification capability of cross-modal network features in working memory-loaded tasks.

[0109]

[0110] Table 2. Classification results of feature combinations under different modalities in the n-back task.

Claims

1. A method for analyzing brain cognitive function based on EEG-fNIRS cross-modal multilayer networks, characterized in that... Includes the following steps: S1: Simultaneously acquire multi-channel EEG and fNIRS signals under the working memory training paradigm using EEG and fNIRS acquisition devices; S2: Preprocess the EEG and fNRIS signals; S3: Reconstruct the source space of the signal, assign corresponding location information to the EEG and fNIRS source time series, and divide the region of interest based on the Desikan-Killiany map; S4: Construct an EEG-fNIRS multiplexing network by quantizing the decomposable directed mutual information between systems using IID-PSIT. S5: Construct an EEG-fNIRS fully connected multilayer network by quantifying cross-frequency coupling in brain functional networks using GCCSIT; S6: Use graph theory parameters to analyze and extract the topological features of multiplexed and multilayer networks in S4 and S5; S7: Use support vector machines as classifiers to perform binary classification for different cognitive load levels.

2. The brain cognitive function analysis method based on EEG-fNIRS cross-modal multilayer network according to claim 1, characterized in that, S4 includes the following steps: S411: Determine the direction of information flow by quantifying the mutual information between the past and future states of the system through time-delay mutual information quantification. S412: By using IID theory, TDMI is refined into 16 atoms of the IID lattice, accurately measuring the interaction patterns between signals. S413: By proposing a phase-specific information transfer method based on IID, the decomposable directed mutual information between systems is quantified.

3. The brain cognitive function analysis method based on EEG-fNIRS cross-modal multilayer network according to claim 2, characterized in that, S4 also includes the following steps: S420: Calculates the functional connectivity between ROIs using a modified comprehensive information metric, reflecting the information gain or loss resulting from the overall synergistic effect of the system.

4. The brain cognitive function analysis method based on EEG-fNIRS cross-modal multilayer network according to claim 3, characterized in that, S4 also includes the following steps: S431: The obtained weighted coupling matrix is ​​subjected to a data-driven thresholding method to obtain a binary network.

5. The brain cognitive function analysis method based on EEG-fNIRS cross-modal multilayer network according to claim 4, characterized in that, S431 also includes the following steps: S432: Construct a multi-layer network model based on multiplexing network theory, where only cross-layer connections of the same node are allowed between layers.

6. The brain cognitive function analysis method based on EEG-fNIRS cross-modal multilayer network according to claim 1, characterized in that, S5 also includes the following steps: S510: Propose the GCSIT method, and Φ under the GCSIT method WMS,R The indicators constitute the functional connectivity network W for each characteristic signal. M . S520: Interlayer coupling between nodes adopts the amplitude-amplitude coupling analysis method.

7. The brain cognitive function analysis method based on EEG-fNIRS cross-modal multilayer network according to claim 6, characterized in that, The S520 also includes the following steps: S521: Obtain the instantaneous amplitude of each characteristic signal through Hilbert transform; S522: Calculate the Pearson correlation coefficient between each amplitude feature to obtain the fully connected matrix W.

Images