A multimodal covariant network discrimination method fusing electroencephalogram and functional magnetic resonance imaging

Abstract

The application provides a multi-modal covariant network discrimination method fusing electroencephalogram and functional magnetic resonance imaging, aims to reveal specific brain function connection modes corresponding to different motor imagery tasks, and predict motor imagery ability of subjects. The method realizes deep fusion and complementary representation of multi-source neural signals by jointly extracting degree centrality, betweenness centrality, closeness centrality and eigenvector centrality of EEG and fMRI, and constructing a cross-modal covariant network by calculating a Pearson correlation coefficient. The method effectively identifies potential brain function connection modes corresponding to different instructions of motor imagery, selects significant edges based on the correlation between the function connection matrix characteristics and the accuracy of the subjects, uses LASSO (Lasso) regression for leave-one-out prediction, and outputs the classification accuracy and key connection modes of the model under the optimal parameters, so as to realize accurate prediction of different motor imagery tasks. The method provides a new multi-modal neural information fusion strategy for brain-computer interface, neural rehabilitation and clinical function reconstruction, and has important application prospect and academic research value.

Description

Technical Field

[0001] This invention belongs to the field of biomedical engineering, and specifically relates to a method for discriminating covariant networks that integrates multiple modalities. Background Technology

[0002] Motor imagery (MI) is a cognitive process that evokes related neural activity in the brain without requiring actual physical movement. This task is widely used in neurorehabilitation, brain-computer interfaces, and clinical research. Existing research shows that motor imagery, as a higher cognitive activity in humans, typically involves the coordination of multiple brain regions and various cortical pathways, including the motor network, parietal network, attentional network, and prefrontal cortex, and produces temporal-spatial changes similar to actual physical movement. Therefore, accurately characterizing the differences in brain networks evoked by different motor imagery actions is key to understanding the mechanisms of motor representation and improving recognition performance.

[0003] Existing research largely relies on single-modal imaging data, such as the high temporal resolution time-frequency-spatial signals provided by electroencephalography (EEG) or the task-related activation and functional connectivity patterns revealed by functional magnetic resonance imaging (fMRI). However, single-modal approaches are insufficient to comprehensively characterize the dynamics and structural constraints of the nervous system under task conditions: while EEG can keenly reflect millisecond-level neural activity changes, its spatial information is limited by volumetric conduction and electrode aliasing effects; fMRI can depict extensive brain region coordination patterns but lacks the ability to resolve rapidly time-varying processes. Therefore, recent research has gradually shifted towards multimodal fusion and network modeling to fully utilize the complementary information between different modalities and characterize the organization of the nervous system under structure-function constraints. Related methods include structure-function coupling analysis, covariant network modeling, functional connectivity (FC) construction, graph-based topological feature extraction, and cross-modal sparse regression and predictive modeling.

[0004] In this context, covariant network analysis provides an effective approach for describing stable population-consistent networks in multimodal data. By quantifying co-variable changes across different brain regions, this method can capture more granular network reconstruction features than traditional FC or activation analysis, and is particularly suitable for integrating multi-source data such as EEG, fMRI, and behavioral scales, highlighting task-related key connectivity patterns from complementary information.

[0005] In motor imagery tasks, the neural representations corresponding to different movements (such as left hand, right hand, and both feet) exhibit heterogeneity. Factors such as equipment conditions, individual differences, and paradigm design pose challenges to extracting highly discriminative and robust brain activity patterns. Therefore, an analytical framework is needed that can integrate multimodal information, characterize changes in collaborative networks, and sensitively reflect differences in motor imagery categories.

[0006] Based on the aforementioned needs, this study proposes a multimodal covariant network analysis method, aiming to integrate complementary features from modalities such as EEG to capture network coordination changes caused by different motor imagery actions. This method can characterize task-induced functional reconstructions at the network level, providing a more reliable neural representation basis for motor intention recognition, neural modulation strategy design, and brain-computer interface algorithm development. Summary of the Invention

[0007] To address issues such as individual differences, inherent equipment effects, and inherent signal defects, this invention proposes a multimodal covariant network analysis method.

[0008] This invention proposes a method for analyzing multimodal covariant networks, comprising the following steps:

[0009] Step S1: Preprocess the raw EEG and fMRI signals;

[0010] The preprocessing method for EEG is as follows: During the online acquisition phase, the raw data is processed using a 0.3–100Hz bandpass filter. After completion, the signal is rereferenced to reduce the reference point deviation and make it close to the "infinity reference". Then, the offline bandpass filter is applied to 8–30Hz to preserve the rhythms related to motion imagery. The execution period of 500–2500ms is extracted for each trial and baseline correction is performed in [-100,0]ms, while artifacts > ±80μV are removed. Finally, the average of each subject and each motion imagery category is taken after artifact removal to improve the signal-to-noise ratio, resulting in three average motion imagery waveforms for each subject.

[0011] The fMRI preprocessing method was as follows: First, 10 initial scan sequences were discarded for stabilization, followed by temporal correction and realignment. The data were normalized to the MNI space and spatially smoothed using a 4mm FWHM Gaussian kernel. The time series were bandpass filtered by 0.01–0.1Hz and then projected onto 200 regions of interest defined by the Schaefer brain atlas. Finally, the time series were projected onto different actions using a design matrix.

[0012] Step S2: Calculate the functional connectivity matrix of the preprocessed EEG and fMRI signals:

[0013] For the preprocessed EEG time series, phase-locked values ​​were calculated, and the PLV matrix of all trials for each type of motor imagery task was averaged. For the fMRI time series, Pearson correlation coefficients were calculated to obtain the functional connectivity matrix of each type of motor imagery task.

[0014] Step S3: Extract node attributes from the EEG and fMRI functional connectivity matrices:

[0015] For EEG and fMRI functional connectivity matrices, four network properties are extracted: degree centrality, betweenness centrality, proximity centrality, and eigenvector centrality.

[0016] Step S4: Construct a multimodal covariant network:

[0017] The four network attributes obtained from EEG and fMRI are concatenated column by column, and the Pearson correlation coefficient is calculated on the concatenated vector to obtain a multimodal covariant network.

[0018] Step S5: Perform statistical analysis using a multimodal covariant network:

[0019] Perform pairwise differential statistical analysis and visualization of multimodal covariant networks that visualize the movements of the left hand, right hand, and both feet;

[0020] Step S6: Predict motor imagery ability using a multimodal covariant network;

[0021] For each subject's EEG data, multiple machine learning classifiers were used to perform three-class classification predictions: left hand, right hand, and foot. The overall classification accuracy was calculated by comparing the predicted labels of the subject with the true labels for all trials, and this accuracy was used as a quantitative indicator of the subject's motor imagery ability.

[0022] Based on a multimodal covariant network for three motion imagery tasks, LASSO regularization parameters and feature significance thresholds were set, and LASSO constraints were applied. Under different parameter combinations, CPM feature selection and LASSO regression modeling were performed using leave-one-out cross-validation to predict the test subjects. Finally, the parameter combination with the highest prediction correlation coefficient was selected as the optimal model.

[0023] Step S7: Reveal the brain's predictive patterns corresponding to different motor imagery movements;

[0024] Identify the most discriminative connections between left-hand motor imagery, right-hand motor imagery, and bilateral foot motor imagery to derive the brain connectivity patterns that contribute most to the prediction results for different motor imagery actions.

[0025] Furthermore, the data preprocessing step S1 is as follows:

[0026] Step S11: In the EEG acquisition stage, the electrodes were arranged using 64-lead Ag / AgCl electrode caps. The raw EEG was obtained with a sampling rate of 1000 Hz and online bandpass filtering of 0.3–100 Hz. All electrode impedances were controlled below 5kΩ to ensure that the signal has high time resolution, low noise and stable electrode contact, providing a reliable basis for subsequent spatial rereference, filtering and trial analysis.

[0027] Step S12: After acquiring the raw data, the study applies REST rereference to the EEG data obtained in step S11, transforming the data from the actual reference point into an approximate "infinity reference".

[0028] Step S13: After rereference, the data obtained in step S12 is subjected to 8–30 Hz offline bandpass filtering to preserve the key rhythms of motor imagery while suppressing low-frequency drift and high-frequency electromyographic components, so that the signal highlights motor imagery-related activities.

[0029] Step S14: Subsequently, the filtered data is truncated to a motion imagination execution period of 500–2500 ms for each trial, and the mean of each trial is subtracted using [-100,0] ms as the baseline interval to complete the baseline correction. At the same time, an amplitude threshold of ±80 μV is used to remove experiments containing obvious artifacts.

[0030] Step S15: After completing baseline and artifact removal, take the arithmetic mean of the remaining trials s after artifact removal for each subject;

[0031] Step S16: fMRI data were acquired using a 3T GE DiscoveryMR750 scanner with a gradient echo planar imaging sequence.

[0032] Step S17: Discard the first ten scan sequences, and the remaining images are corrected by time layer and head motion; then, the data is spatially normalized to the space of the Montreal Neuroscience Institute, smoothed by Gaussian kernel, bandpass filtered, and the average time series is extracted from the ROI defined by the Schaefer map.

[0033] Step S18: Construct a design matrix based on the MI task, and map the subject's fMRI time series to different actions based on the design matrix.

[0034] Furthermore, the functional connection matrix calculation steps in step S2 are as follows:

[0035] Step S21: For the preprocessed EEG signal, using... The functional connectivity matrix of the EEG signals was obtained;

[0036] ;

[0037] in, This represents the average amplitude of the superimposed complex signal. Indicates the number of trials. This represents the instantaneous phase difference between the same test in different leads. Indicates a point in time. Test Index This means that complex signals can be obtained using the phase principle through Euler's formula. Superimpose the complex signals from all the experiments. Represents the imaginary unit;

[0038] Step S22: For the preprocessed fMRI, the functional connectivity matrix of the fMRI is obtained using the Pearson correlation coefficient;

[0039] ;

[0040] in, These are two time series at the 1st... The signal value at each time point, Let them represent the means of the two time series, respectively. Indicates the quantity of time, This indicates the linear correlation between two brain regions.

[0041] Furthermore, the calculation steps for network attribute extraction in step S3 are as follows:

[0042] Using the functional connectivity matrix of the EEG signal obtained in step S21, calculate four network properties: degree centrality, betweenness centrality, proximity centrality, and eigenvector centrality.

[0043] Using formula Calculate degree centrality;

[0044] in, Represents the weight of the edge. The obtained degree centrality

[0045] Using formula Calculate betweenness centrality;

[0046] in, Represents a node arrive The number of shortest paths, For the nodes passed through in these paths The number of items, Indicates the obtained betweenness centrality;

[0047] For example, using the formula Calculate proximity centrality;

[0048] in, represent The strength of functional connections, Indicates the strength of the "edge". Indicates from node To the node All path sets;

[0049] Using formula Calculation satisfies vector centrality ;

[0050] in, The eigenvector corresponding to the largest eigenvalue. For nodes The corresponding eigenvector centrality, This represents the largest eigenvalue of the adjacency matrix. Represents the adjacency matrix of the network. Represents the adjacency matrix of the th Line number Column elements;

[0051] Furthermore, the calculation steps for constructing the multimodal covariant network in step S4 are as follows:

[0052] Step S41: Concatenate the eight network attributes of the two modalities obtained in step S3 column by column to obtain the fused feature vector;

[0053] Step S42: Utilize the Pearson correlation coefficient to transform the multimodal network obtained in step S41. A multimodal covariant network was derived;

[0054] in, These are two time series at the 1st... The signal value at each time point, Let them represent the means of the two time series, respectively. Indicates the quantity of time, This indicates the linear correlation between two brain regions.

[0055] Furthermore, the statistical analysis step S5 using a multimodal covariant network is as follows:

[0056] Step S51: Define the significance level parameter as 0.05 for subsequent pairing. Inspection and Correction;

[0057] Step S52: Perform pairing on the multimodal covariant networks for any two motion imagination tasks. The test yielded the following results. , Two significance matrices, corresponding to the difference in the direction of strength;

[0058] Step S53: Set the diagonal elements to zero;

[0059] Step S54: Call Functions on two sets The false positive rate is corrected by the value matrix to obtain the threshold; non-significant connections with p < 0.05 are set to zero, and significant connections are retained and assigned a value of 1 to obtain a binary significance matrix, where p represents the significance probability value obtained after the paired statistical test of the connection strength;

[0060] Step S55: Obtain the significant difference matrix by subtracting the binary significance matrices;

[0061] Step S56: Call the function to draw the multi-network ring differential connection diagram.

[0062] Furthermore, the step S6, which utilizes a multimodal covariant network for prediction, is as follows:

[0063] Step S61: Define the set of regularization parameters for LASSO regression and the set of CPM feature selection parameters; set the mixing coefficient of the LASSO model to 1, indicating that LASSO constraints are used completely;

[0064] Step S62: Establish nested loops, with the outer loop iterating through the significance threshold set and the inner loop iterating through the regularization parameter set; perform leave-one-out cross-validation for each parameter combination to evaluate the predictive performance of the model under that parameter configuration;

[0065] Step S63: In each iteration, set aside one subject sample as the test set and use the remaining samples as the training set;

[0066] Step S64: Use the CPM method to calculate the Pearson correlation coefficient and p-value of each edge in the training set with the behavioral variable, and retain only features with p-values ​​less than a set threshold;

[0067] Step S65: Train the LASSO regression model using the selected features, apply the model to the test samples, and obtain the predicted values.

[0068] Furthermore, the steps in step S7 of constructing brain activity patterns are as follows:

[0069] Step S71: Average the weight coefficients of the selected connections in all cross-validation iterations and reconstruct them into a symmetric connection matrix;

[0070] Step S72: Extract the edge weights of the upper triangular region, sort them in descending order of absolute value, and select the first few connections with the largest weights as key prediction features;

[0071] Step S73: Call the ring network visualization function to draw the selected connections according to the node positions and their respective brain networks; use color and line thickness to indicate key interactions between different networks, forming a visual representation of the connection prediction model. The beneficial effects of this invention are:

[0072] By constructing a multimodal covariant network using multimodal data, the problem of the deficiency of single-modal information is made up to a certain extent, and the brain activity patterns corresponding to different motor imagery actions are successfully characterized. Attached Figure Description

[0073] Figure 1 This is a schematic diagram illustrating the steps of a covariant network analysis method that integrates multiple modes proposed in this invention.

[0074] Figure 2 This is a schematic diagram of the data preprocessing involved in the present invention.

[0075] Figure 3 A schematic diagram illustrating the functional connection matrix calculation steps provided by this invention.

[0076] Figure 4 This is a schematic diagram illustrating the calculation steps for network attribute extraction provided by the present invention.

[0077] Figure 5 This is a schematic diagram illustrating the computational steps for constructing a multimodal covariant network provided by the present invention. Figure 6 This is a schematic diagram illustrating the steps of statistical analysis using multimodal covariant networks provided by the present invention.

[0078] Figure 7 This is a schematic diagram illustrating the steps of prediction using a multimodal covariant network provided by the present invention.

[0079] Figure 8 This is a schematic diagram illustrating the steps of extracting brain network patterns using a multimodal covariant network, as provided by the present invention. Detailed Implementation

[0080] The implementation of the present invention will be further described below with reference to the accompanying drawings.

[0081] Firstly, this invention proposes a method for analyzing covariant networks that integrates multiple modalities. Please refer to [link to relevant documentation]. Figure 1 This includes the following steps:

[0082] Step S1: Preprocess the raw EEG and fMRI signals;

[0083] The preprocessing method for EEG is as follows: During the online acquisition phase, the raw data is processed using a 0.3–100Hz bandpass filter. After completion, the signal is rereferenced to reduce the reference point deviation and make it close to the "infinity reference". Then, the offline bandpass filter is applied to 8–30Hz to preserve the rhythms related to motion imagery. The execution period of 500–2500ms is extracted for each trial and baseline correction is performed in [-100,0]ms, while artifacts > ±80μV are removed. Finally, the average of each subject and each motion imagery category is taken after artifact removal to improve the signal-to-noise ratio, resulting in three average motion imagery waveforms for each subject.

[0084] The fMRI preprocessing method was as follows: First, 10 initial scan sequences were discarded for stabilization, followed by temporal correction and realignment. The data were normalized to the MNI space and spatially smoothed using a 4mm FWHM Gaussian kernel. The time series were bandpass filtered by 0.01–0.1Hz and then projected onto 200 regions of interest defined by the Schaefer brain atlas. Finally, the time series were projected onto different actions using a design matrix.

[0085] Step S2: Calculate the functional connectivity matrix of the preprocessed EEG and fMRI signals:

[0086] For the preprocessed EEG time series, phase-locked values ​​were calculated, and the PLV matrix of all trials for each type of motor imagery task was averaged. For the fMRI time series, Pearson correlation coefficients were calculated to obtain the functional connectivity matrix of each type of motor imagery task.

[0087] Step S3: Extract node attributes from the EEG and fMRI functional connectivity matrices:

[0088] For EEG and fMRI functional connectivity matrices, four network properties are extracted: degree centrality, betweenness centrality, proximity centrality, and eigenvector centrality.

[0089] Step S4: Construct a multimodal covariant network:

[0090] The four network attributes obtained from EEG and fMRI are concatenated column by column, and the Pearson correlation coefficient is calculated on the concatenated vector to obtain a multimodal covariant network.

[0091] Step S5: Perform statistical analysis using a multimodal covariant network:

[0092] Perform pairwise differential statistical analysis and visualization of multimodal covariant networks that visualize the movements of the left hand, right hand, and both feet;

[0093] Step S6: Predict motor imagery ability using a multimodal covariant network;

[0094] For each subject's EEG data, multiple machine learning classifiers were used to perform three-class classification predictions: left hand, right hand, and foot. The overall classification accuracy was calculated by comparing the predicted labels of the subject with the true labels for all trials, and this accuracy was used as a quantitative indicator of the subject's motor imagery ability.

[0095] Based on a multimodal covariant network for three motion imagery tasks, LASSO regularization parameters and feature significance thresholds were set, and LASSO constraints were applied. Under different parameter combinations, CPM feature selection and LASSO regression modeling were performed using leave-one-out cross-validation to predict the test subjects. Finally, the parameter combination with the highest prediction correlation coefficient was selected as the optimal model.

[0096] Step S7: Reveal the brain's predictive patterns corresponding to different motor imagery movements;

[0097] Identify the most discriminative connections between left-hand motor imagery, right-hand motor imagery, and bilateral foot motor imagery to derive the brain connectivity patterns that contribute most to the prediction results for different motor imagery actions.

[0098] Further, please refer to Figure 2 The data preprocessing step S1 is as follows:

[0099] Step S11: In the EEG acquisition stage, the electrodes were arranged using 64-lead Ag / AgCl electrode caps. The raw EEG was obtained with a sampling rate of 1000 Hz and online bandpass filtering of 0.3–100 Hz. All electrode impedances were controlled below 5kΩ to ensure that the signal has high time resolution, low noise and stable electrode contact, providing a reliable basis for subsequent spatial rereference, filtering and trial analysis.

[0100] Step S12: After acquiring the raw data, the study applies REST rereference to the EEG data obtained in step S11, transforming the data from the actual reference point into an approximate "infinity reference".

[0101] Step S13: After rereference, the data obtained in step S12 is subjected to 8–30 Hz offline bandpass filtering to preserve the key rhythms of motor imagery while suppressing low-frequency drift and high-frequency electromyographic components, so that the signal highlights motor imagery-related activities.

[0102] Step S14: Subsequently, the filtered data is truncated to a motion imagination execution period of 500–2500 ms for each trial, and the mean of each trial is subtracted using [-100,0] ms as the baseline interval to complete the baseline correction. At the same time, an amplitude threshold of ±80 μV is used to remove experiments containing obvious artifacts.

[0103] Step S15: After completing baseline and artifact removal, take the arithmetic mean of the remaining trials s after artifact removal for each subject;

[0104] Step S16: fMRI data were acquired using a 3T GE DiscoveryMR750 scanner with a gradient echo planar imaging sequence.

[0105] Step S17: Discard the first ten scan sequences, and the remaining images are corrected by time layer and head motion; then, the data is spatially normalized to the space of the Montreal Neuroscience Institute, smoothed by Gaussian kernel, bandpass filtered, and the average time series is extracted from the ROI defined by the Schaefer map.

[0106] Step S18: Construct a design matrix based on the MI task, and map the subject's fMRI time series to different actions based on the design matrix.

[0107] Further, please refer to Figure 3 The steps for calculating the functional connection matrix in step S2 are as follows:

[0108] Step S21: For the preprocessed EEG signal, using... The functional connectivity matrix of the EEG signals was obtained;

[0109] ;

[0110] in, This represents the average amplitude of the superimposed complex signal. Indicates the number of trials. This represents the instantaneous phase difference between the same test in different leads. Indicates a point in time. Test Index This means that complex signals can be obtained using the phase principle through Euler's formula. Superimpose the complex signals from all the experiments. Represents the imaginary unit;

[0111] Step S22: For the preprocessed fMRI, the functional connectivity matrix of the fMRI is obtained using the Pearson correlation coefficient;

[0112] ;

[0113] in, These are two time series at the 1st... The signal value at each time point, Let them represent the means of the two time series, respectively. Indicates the quantity of time, This indicates the linear correlation between two brain regions.

[0114] Further, please refer to Figure 4 The calculation steps for network attribute extraction in step S3 are as follows:

[0115] Using the functional connectivity matrix of the EEG signal obtained in step S21, calculate four network properties: degree centrality, betweenness centrality, proximity centrality, and eigenvector centrality.

[0116] Using formula Calculate degree centrality;

[0117] in, Represents the weight of the edge. The obtained degree centrality

[0118] Using formula Calculate betweenness centrality;

[0119] in, Represents a node arrive The number of shortest paths, For the nodes passed through in these paths The number of items, Indicates the obtained betweenness centrality;

[0120] For example, using the formula Calculate proximity centrality;

[0121] in, represent The strength of functional connections, Indicates the strength of the "edge". Indicates from node To the node All path sets;

[0122] Using formula Calculation satisfies vector centrality ;

[0123] in, The eigenvector corresponding to the largest eigenvalue. For nodes The corresponding eigenvector centrality, This represents the largest eigenvalue of the adjacency matrix. Represents the adjacency matrix of the network. Represents the adjacency matrix of the th Line number Column elements;

[0124] Further, please refer to Figure 5 The calculation steps for constructing the multimodal covariant network in step S4 are as follows:

[0125] Step S41: Concatenate the eight network attributes of the two modalities obtained in step S3 column by column to obtain the fused feature vector;

[0126] Step S42: Utilize the Pearson correlation coefficient to transform the multimodal network obtained in step S41. A multimodal covariant network was derived;

[0127] in, These are two time series at the 1st... The signal value at each time point, Let them represent the means of the two time series, respectively. Indicates the quantity of time, This indicates the linear correlation between two brain regions.

[0128] Further, please refer to Figure 6 The steps in step S5, which utilize a multimodal covariant network for statistical analysis, are as follows:

[0129] Step S51: Define the significance level parameter as 0.05 for subsequent pairing. Inspection and Correction;

[0130] Step S52: Perform pairing on the multimodal covariant networks for any two motion imagination tasks. The test yielded the following results. , Two significance matrices, corresponding to the difference in the direction of strength;

[0131] Step S53: Set the diagonal elements to zero;

[0132] Step S54: Call Functions on two sets The false positive rate is corrected by the value matrix to obtain the threshold; non-significant connections with p < 0.05 are set to zero, and significant connections are retained and assigned a value of 1 to obtain a binary significance matrix, where p represents the significance probability value obtained after the paired statistical test of the connection strength;

[0133] Step S55: Obtain the significant difference matrix by subtracting the binary significance matrices;

[0134] Step S56: Call the function to draw the multi-network ring differential connection diagram.

[0135] Further, please refer to Figure 7 The steps in step S6, which utilize a multimodal covariant network for prediction, are as follows:

[0136] Step S61: Define the set of regularization parameters for LASSO regression and the set of CPM feature selection parameters; set the mixing coefficient of the LASSO model to 1, indicating that LASSO constraints are used completely;

[0137] Step S62: Establish nested loops, with the outer loop iterating through the significance threshold set and the inner loop iterating through the regularization parameter set; perform leave-one-out cross-validation for each parameter combination to evaluate the predictive performance of the model under that parameter configuration;

[0138] Step S63: In each iteration, set aside one subject sample as the test set and use the remaining samples as the training set;

[0139] Step S64: Use the CPM method to calculate the Pearson correlation coefficient and p-value of each edge in the training set with the behavioral variable, and retain only features with p-values ​​less than a set threshold;

[0140] Step S65: Train the LASSO regression model using the selected features, apply the model to the test samples, and obtain the predicted values.

[0141] Further, please refer to Figure 8 The steps in step S7 for constructing brain activity patterns are as follows:

[0142] Step S71: Average the weight coefficients of the selected connections in all cross-validation iterations and reconstruct them into a symmetric connection matrix;

[0143] Step S72: Extract the edge weights of the upper triangular region, sort them in descending order of absolute value, and select the first few connections with the largest weights as key prediction features;

[0144] Step S73: Call the ring network visualization function to draw the selected connections according to the node positions and their respective brain networks; use color and line thickness to indicate key interactions between different networks, forming a visualization of the connection prediction model.

Claims

1. A multimodal covariant network discrimination method integrating electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), comprising the following steps: Step S1: Preprocess the raw EEG and fMRI signals; The preprocessing method for EEG is as follows: During the online acquisition phase, the raw data is processed using a 0.3–100Hz bandpass filter. After completion, the signal is rereferenced to reduce the reference point deviation and make it close to the "infinity reference". Then, the offline bandpass filter is applied to 8–30Hz to preserve the rhythms related to motion imagery. The execution period of 500–2500ms is extracted for each trial and baseline correction is performed in [-100,0]ms, while artifacts > ±80μV are removed. Finally, the average of each subject and each motion imagery category is taken after artifact removal to improve the signal-to-noise ratio, resulting in three average motion imagery waveforms for each subject. The fMRI preprocessing method was as follows: First, 10 initial scan sequences were discarded for stabilization, followed by temporal correction and realignment. The data were normalized to the MNI space and spatially smoothed using a 4mm FWHM Gaussian kernel. The time series were bandpass filtered by 0.01–0.1Hz and then projected onto 200 regions of interest defined by the Schaefer brain atlas. Finally, the time series were projected onto different actions using a design matrix. Step S2: Calculate the functional connectivity matrix of the preprocessed EEG and fMRI signals: For the preprocessed EEG time series, phase-locked values ​​were calculated, and the PLV matrix of all trials for each type of motor imagery task was averaged. For the fMRI time series, Pearson correlation coefficients were calculated to obtain the functional connectivity matrix of each type of motor imagery task. Step S3: Extract node attributes from the EEG and fMRI functional connectivity matrices: For EEG and fMRI functional connectivity matrices, four network properties are extracted: degree centrality, betweenness centrality, proximity centrality, and eigenvector centrality. Step S4: Construct a multimodal covariant network: The four network attributes obtained from EEG and fMRI are concatenated column by column, and the Pearson correlation coefficient is calculated on the concatenated vector to obtain a multimodal covariant network. Step S5: Perform statistical analysis using a multimodal covariant network: Perform pairwise differential statistical analysis and visualization of multimodal covariant networks that visualize the movements of the left hand, right hand, and both feet; Step S6: Predict motor imagery ability using a multimodal covariant network; For each subject's EEG data, multiple machine learning classifiers were used to perform three-class classification predictions: left hand, right hand, and foot. The overall classification accuracy was calculated by comparing the predicted labels of the subject with the true labels for all trials, and this accuracy was used as a quantitative indicator of the subject's motor imagery ability. Based on a multimodal covariant network for three motion imagery tasks, LASSO regularization parameters and feature significance thresholds were set, and LASSO constraints were applied. Under different parameter combinations, CPM feature selection and LASSO regression modeling were performed using leave-one-out cross-validation to predict the test subjects. Finally, the parameter combination with the highest prediction correlation coefficient was selected as the optimal model. Step S7: Reveal the brain's predictive patterns corresponding to different motor imagery movements; Identify the most discriminative connections between left-hand motor imagery, right-hand motor imagery, and bilateral foot motor imagery to derive the brain connectivity patterns that contribute most to the prediction results for different motor imagery actions.

2. The multimodal covariant network discrimination method fusing electroencephalography and functional magnetic resonance imaging as described in claim 1, characterized in that, The data preprocessing steps in step S1 are as follows: Step S11: In the EEG acquisition stage, the electrodes were arranged using 64-lead Ag / AgCl electrode caps. The raw EEG was obtained with a sampling rate of 1000 Hz and online bandpass filtering of 0.3–100 Hz. All electrode impedances were controlled below 5kΩ to ensure that the signal has high time resolution, low noise and stable electrode contact, providing a reliable basis for subsequent spatial rereference, filtering and trial analysis. Step S12: After acquiring the raw data, the study applies REST rereference to the EEG data obtained in step S11, transforming the data from the actual reference point into an approximate "infinity reference"; Step S13: After rereference, the data obtained in step S12 is subjected to 8–30 Hz offline bandpass filtering to preserve the key rhythms of motor imagery while suppressing low-frequency drift and high-frequency electromyographic components, so that the signal highlights motor imagery-related activities. Step S14: Subsequently, the filtered data is truncated to a motion imagination execution period of 500–2500 ms for each trial, and the mean of each trial is subtracted using [-100,0] ms as the baseline interval to complete the baseline correction. At the same time, an amplitude threshold of ±80 μV is used to remove experiments containing obvious artifacts. Step S15: After completing baseline and artifact removal, take the arithmetic mean of the remaining trials s after artifact removal for each subject; Step S16: fMRI data were acquired using a 3T GE DiscoveryMR750 scanner with a gradient echo planar imaging sequence. Step S17: Discard the first ten scan sequences, and the remaining images are corrected by time layer and head motion; then, the data is spatially normalized to the space of the Montreal Neuroscience Institute, smoothed by Gaussian kernel, bandpass filtered, and the average time series is extracted from the ROI defined by the Schaefer map. Step S18: Construct a design matrix based on the MI task, and map the subject's fMRI time series to different actions based on the design matrix.

3. The covariant network analysis method integrating multiple modalities as described in claim 1, characterized in that, The steps for calculating the functional connection matrix in step S2 are as follows: Step S21: For the preprocessed EEG signal, using... The functional connectivity matrix of the EEG signals was obtained; ； in, This represents the average amplitude of the superimposed complex signal. Indicates the number of trials. This represents the instantaneous phase difference between the same test in different leads. Indicates a point in time. Test Index This means that complex signals can be obtained using the phase principle through Euler's formula. Superimpose the complex signals from all the experiments. Represents the imaginary unit; Step S22: For the preprocessed fMRI, the functional connectivity matrix of the fMRI is obtained using the Pearson correlation coefficient; ； in, These are two time series at the 1st... The signal value at each time point, Let them represent the means of the two time series, respectively. Indicates the quantity of time, This indicates the linear correlation between two brain regions.

4. The multimodal covariant network discrimination method fusing electroencephalography and functional magnetic resonance imaging as described in claim 1, characterized in that, The calculation steps for node attribute extraction in step S3 are as follows: Using the functional connectivity matrix of the EEG signal obtained in step S21, calculate four network properties: degree centrality, betweenness centrality, proximity centrality, and eigenvector centrality. Using formula Calculate degree centrality; in, Represents the weight of the edge. The obtained degree centrality Using formula Calculate betweenness centrality; in, Represents a node arrive The number of shortest paths, For the nodes passed through in these paths The number of items, Indicates the obtained betweenness centrality; For example, using the formula Calculate proximity centrality; in, represent The strength of functional connections, Indicates the strength of the "edge". Indicates from node To the node All path sets; Using formula Calculation satisfies vector centrality ; in, The eigenvector corresponding to the largest eigenvalue. For nodes The corresponding eigenvector centrality, This represents the largest eigenvalue of the adjacency matrix. Represents the adjacency matrix of the network. Represents the adjacency matrix of the th Line number The elements of the column.

5. The multimodal covariant network discrimination method fusing electroencephalography and functional magnetic resonance imaging as described in claim 1, characterized in that, The calculation steps for constructing the multimodal covariant network in step S4 are as follows: Step S41: Concatenate the eight network attributes of the two modalities obtained in step S3 column by column to obtain the fused feature vector; Step S42: Utilize the Pearson correlation coefficient to transform the multimodal network obtained in step S41. A multimodal covariant network was derived; in, These are two time series at the 1st... The signal value at each time point, Let them represent the means of the two time series, respectively. Indicates the quantity of time, This indicates the linear correlation between two brain regions.

6. The multimodal covariant network discrimination method fusing electroencephalography and functional magnetic resonance imaging as described in claim 1, characterized in that, The steps in step S5, which involve statistical analysis using a multimodal covariant network, are as follows: Step S51: Define the significance level parameter as 0.05 for subsequent pairing. Inspection and Correction; Step S52: Perform pairing on the multimodal covariant networks for any two motion imagination tasks. The test yielded the following results. , Two significance matrices, corresponding to the difference in the direction of strength; Step S53: Set the diagonal elements to zero; Step S54: Call Functions on two sets The false positive rate is corrected by the value matrix to obtain the threshold; non-significant connections with p < 0.05 are set to zero, and significant connections are retained and assigned a value of 1 to obtain a binary significance matrix, where p represents the significance probability value obtained after the paired statistical test of the connection strength; Step S55: Obtain the significant difference matrix by subtracting the binary significance matrices; Step S56: Call the function to draw the multi-network ring differential connection diagram.

7. The multimodal covariant network discrimination method fusing electroencephalography and functional magnetic resonance imaging as described in claim 1, characterized in that, The steps in step S6, which utilize a multimodal covariant network for prediction, are as follows: Step S61: Define the set of regularization parameters for LASSO regression and the set of CPM feature selection parameters; set the mixing coefficient of the LASSO model to 1, indicating that LASSO constraints are used completely; Step S62: Establish nested loops, with the outer loop iterating through the significance threshold set and the inner loop iterating through the regularization parameter set; perform leave-one-out cross-validation for each parameter combination to evaluate the predictive performance of the model under that parameter configuration; Step S63: In each iteration, set aside one subject sample as the test set and use the remaining samples as the training set; Step S64: Use the CPM method to calculate the Pearson correlation coefficient and p-value of each edge in the training set with the behavioral variable, and retain only features with p-values ​​less than a set threshold; Step S65: Train the LASSO regression model using the selected features, apply the model to the test samples, and obtain the predicted values.

8. The multimodal covariant network discrimination method fusing electroencephalography and functional magnetic resonance imaging as described in claim 1, characterized in that, The steps in step S7 that reveal patterns of brain activity are as follows: Step S71: Average the weight coefficients of the selected connections in all cross-validation iterations and reconstruct them into a symmetric connection matrix; Step S72: Extract the edge weights of the upper triangular region, sort them in descending order of absolute value, and select the first few connections with the largest weights as key prediction features; Step S73: Call the ring network visualization function to draw the selected connection according to the node position and the brain network to which it belongs; By using color and line thickness to indicate key interactions between different networks, a visual representation of the connectivity prediction model is created.

Images