Method and device for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG

Abstract

The present invention discloses a method and device for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG, which belongs to the technical field of spatiotemporal signal classification of functional brain networks, including: obtaining a multiplexed partitioned EEG resting signal segment sequence and a multiplexed partitioned continuous EEG resting signal segment sequence; inputting the multiplexed partitioned EEG resting signal segment sequence into a static functional connectivity spatial topological learning module to obtain highly discriminative local and global spatial topological features; inputting the multiplexed partitioned continuous EEG resting signal segment sequence into a dynamic functional connectivity temporal learning module to obtain highly discriminative temporal features; splicing highly discriminative local and global spatial topological features and highly discriminative temporal features, and inputting the splicing results into a multi-layer perceptron classification layer to obtain a brain cognitive resting task classification result. When exploring the functional brain network, the present invention takes into account the two scales of dynamic functional connectivity and static functional connectivity, and fully considers the spatiotemporal changes of brain activity.

Description

Technical Field

[0001] The present invention belongs to the technical field of functional brain network spatiotemporal signal classification, and specifically relates to an EEG-based dynamic and static functional brain network spatiotemporal topology fusion method and device. Background Art

[0002] Brain cognitive tasks require the coordinated action of various functional areas of the human brain, which is related to how the human brain receives, processes and transmits information. Such brain activity exists in various human behaviors, and is particularly active in brain cognitive tasks. In order to gain a deeper understanding of the neurophysiological basis of brain activity in human cognitive tasks, a variety of neuroimaging technologies have emerged, and EEG (electroencephalogram) is a commonly used technology in this type of research.

[0003] As a non-invasive technology for measuring brain function and brain electrical activity, EEG can record effective patterns of bioelectric activity of brain neurons corresponding to various states from the surface area of ​​the scalp and describe brain activity with extremely high temporal resolution. Relying on its advantages of portability and non-invasiveness, it has been used as a common biomarker in the study of brain cognitive tasks.

[0004] At present, in the functional brain network research based on EEG classification of brain cognitive resting tasks, a single static functional connectivity index is used to describe brain activity, which only studies the relationship between brain areas measured in a stable state, without considering the spatiotemporal changes of brain activity, that is, dynamic functional connectivity. This makes the subsequent analysis limited to a single space in the functional brain network features. The recognition model is mainly based on traditional machine learning and deep learning methods. Traditional machine learning methods such as SVM (support vector machine), RF (random forest) and KNN (K-nearest neighbor) require a lot of manual work to extract corresponding features based on prior knowledge, and then input them into the machine learning model for classification; deep learning models such as CNN (convolutional neural network) have significant effects on image classification, which has led some scholars to automatically extract features by converting EEG data into images. Another example is LSTM (long and short term memory networks) that captures temporal correlation by filtering and retaining temporal signals through gating units. The discussion in the field of functional brain networks has made scholars begin to pay attention to the impact of functional connectivity on subjects in brain cognitive tasks. Although some studies have used adjacency matrices as input into CNN to simulate the relationships in functional brain networks, adjacency matrices are irregular and non-Euclidean data and are not suitable for CNN input. Summary of the invention

[0005] The purpose of the present invention is to address the above-mentioned deficiencies in the prior art and to provide an EEG-based method and device for spatiotemporal topological fusion of dynamic and static functional brain networks, so as to solve the problem that the existing brain cognitive resting task classification is limited to a single spatial field in the functional brain network characteristics and fails to consider the relationship between the local brain region and the global brain as a whole.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG, comprising the following steps:

[0008] S1. Collecting original EEG resting signals based on EEG acquisition equipment, and preprocessing the original EEG resting signals to obtain EEG resting signal segment sequences;

[0009] S2, performing channel multiplexing and brain region division processing on the EEG resting signal segment sequence to obtain a multiplexed and divided EEG resting signal segment sequence and a multiplexed and divided continuous EEG resting signal segment sequence consisting of multiple 1-second segments;

[0010] S3, inputting the multiplexed and divided EEG resting signal segment sequence into the static functional connectivity spatial topology learning module to obtain highly discriminative local and global spatial topological features;

[0011] S4, inputting the multiplexed and divided continuous EEG resting signal segment sequence into the dynamic functional connectivity temporal learning module to obtain highly discriminative temporal features;

[0012] S5. Concatenate the highly discriminative local and global spatial topological features and the highly discriminative temporal features, and input the concatenated results into the multi-layer perceptron classification layer to obtain the classification results of the brain cognitive resting task.

[0013] Furthermore, S1 specifically includes the following sub-steps:

[0014] S11, using EEG acquisition equipment to continuously collect 19 channels of raw EEG resting signals of the 10-20 international standard lead system;

[0015] S12, performing 0.5-50 Hz bandpass filtering and 50 Hz notch filtering on the original EEG resting signal, and downsampling the filtered EEG resting signal to 128 Hz;

[0016] S13, overlap the 50% sliding window of the filtered and downsampled EEG resting signal and divide it into a sequence of 5-second EEG resting signal segments .

[0017] Furthermore, S2 reuses some of the 19 channels of the 10-20 international standard lead system and divides them into multiple brain regions, including:

[0018] Left frontal area: Fp1, F7, F3;

[0019] Right frontal area: Fp2, F4, F8;

[0020] Left temporal lobe area: T3, T5, F7;

[0021] Central area: C3, Cz, C4;

[0022] Right temporal lobe area: T4, T6, F8;

[0023] Parietal area: P3, Pz, P4;

[0024] Occipital region: O1, O2, Pz;

[0025] Among them, Fp1, F7, and F3 in the left frontal area are part of the channels of the frontal lobe of the 10-20 international standard lead system, and the acquisition position is the left front of the human brain; Fp2, F4, and F8 in the right frontal area are part of the channels of the frontal lobe of the 10-20 international standard lead system, and the acquisition position is the right front of the human brain; T3 and T5 in the left temporal area are part of the channels of the temporal lobe of the 10-20 international standard lead system, and the acquisition position is the left middle of the human brain, and F7 is the multiplexed channel of the left frontal area; C3, Cz, and C4 in the central area are part of the channels of the central area of ​​the 10-20 international standard lead system For some channels, the acquisition position is the middle of the human brain, located behind the frontal lobe and in front of the parietal lobe; T4 and T6 in the right temporal lobe area are some channels of the temporal lobe in the 10-20 international standard lead system, and the acquisition position is the right middle of the human brain, and F8 is the multiplexed channel of the right frontal lobe area; P3, Pz, and P4 in the parietal lobe area are some channels of the parietal lobe in the 10-20 international standard lead system, and the acquisition position is the middle and posterior of the human brain; O1 and O2 in the occipital lobe area are some channels of the occipital lobe in the 10-20 international standard lead system, and the acquisition position is the posterior part of the human brain, and Pz is the multiplexed channel of the parietal lobe area.

[0026] Furthermore, S3 specifically includes the following sub-steps:

[0027] S31. Using Fast Fourier Transform to Acquire Multiplexed EEG Resting Signal Segment Sequences Frequency domain characteristics , and then use the squeeze excitation network to adjust the frequency domain features Perform feature weighted extraction to obtain the aggregated feature vector :

[0028]

[0029] in, is the squeezing function; and is the frequency domain feature The first two dimensions of is the channel dimension, is the frequency domain dimension; is the value obtained by traversing the channel dimension C and the frequency domain dimension F; s is the frequency domain feature The number of segments obtained by division; They are temporary variables for traversing the channel dimension C and the frequency domain dimension F respectively;

[0030] S32, aggregate feature vector Each feature in is sample activated to obtain weighted frequency domain features :

[0031]

[0032] in, is the activation function; is the ReLU activation function, is the Sigmoid activation function; It is a general term for the fully connected layer; and There are two fully connected layers;

[0033] S33. Weighted frequency domain features in the channel dimension Divide the brain regions to obtain weighted frequency domain feature sequences of multiple brain regions ;

[0034] S34, weighted frequency domain feature sequences of multiple brain regions Input into the local functional connectivity learning module within the brain region to learn the integration of brain region weights and the integration of highly discriminative local spatial topological features ;

[0035] S35, weighted frequency domain features Integration with brain region weights Input into the global functional connectivity learning module between brain regions to learn highly discriminative global spatial topological features ;

[0036] S36. Integration of highly discriminative local spatial topological features and highly discriminative global spatial topological features Splicing to obtain highly discriminative local and global spatial topological features .

[0037] Furthermore, the learning method of the S34 midbrain region local functional connectivity learning module includes:

[0038] S341, obtain weighted frequency domain feature sequences of multiple brain regions Weighted frequency domain features of single brain regions in , calculate the weighted frequency domain features of a single brain region Functional connectivity index iCOH, and constructed as a static functional connectivity adjacency matrix :

[0039]

[0040] in, represents the imaginary component, is the cross-spectral density, and All are autospectral densities;

[0041] S342, connect the static function adjacency matrix Input into the brain region functional connectivity attention module to learn the Laplace normalized adjacency matrix of the brain region and brain region weights ;

[0042] S343, weighted frequency domain features of single brain region and the Laplace normalized adjacency matrix of brain regions As nodes and edges in GCN, we learn the local spatial topological features of the brain region. :

[0043]

[0044] in, For a simple GCN function, is the batch normalization function;

[0045] S344, repeat S341 to S343 until the brain region weights of all brain regions are calculated and local spatial topological characteristics of brain regions , the brain region weights of all brain regions and local spatial topological features of all brain regions Splice them separately to get the integration of brain area weights and the integration of highly discriminative local spatial topological features .

[0046] Furthermore, the learning method of the S342 midbrain functional connectivity attention module includes:

[0047] S3421, static functional connectivity adjacency matrix of input using multi-scale selective kernel attention blocks Perform weighted feature extraction, integrate and activate the extracted features:

[0048]

[0049]

[0050] in, for and The integration and addition of two features; and Represents the static functional connectivity adjacency matrix Features obtained by one-dimensional convolution using convolution kernel sizes of 1*1 and 3*3; For the entire activation process; is the LeakyReLU function;

[0051] S3422, according to the characteristics and , calculate the weighted static functional connectivity adjacency matrix :

[0052]

[0053]

[0054]

[0055] in, So An exponential function with a as the base, a and b are attention weights of different scales;

[0056] S3423. Calculate weighted static functional connectivity adjacency matrix The Laplace normalized adjacency matrix of the brain region ;

[0057] S3424. Calculate brain region weights based on attention weights of different scales :

[0058]

[0059] in, To find the maximum value function of the matrix, Integrating attention weights for multiple scales.

[0060] Furthermore, the learning method of the S35 midbrain region global functional connectivity learning module includes:

[0061] S351. Calculate weighted frequency domain features Functional connectivity index iCOH, and constructed as a global static functional connectivity adjacency matrix ;

[0062] S352, integration of brain area weights Connect adjacency matrix with global static function Multiply them together to get a weighted global static functional connectivity adjacency matrix similar to channel attention :

[0063]

[0064] in, is the function used to expand the dimension, represents element-wise multiplication;

[0065] S353, calculate weighted global static functional connectivity adjacency matrix The global Laplace normalized adjacency matrix of ;

[0066] S354, weighted frequency domain features and the global Laplace normalized adjacency matrix As nodes and edges in GCN, respectively, we learn highly discriminative global spatial topological features .

[0067] Furthermore, S4 specifically includes the following sub-steps:

[0068] S41, multiplex and divide the continuous EEG resting signal segment sequence 1s multiplexing to divide EEG resting signal segments , calculate 1s multiplexing and dividing EEG resting signal segments The functional connectivity index iCOH is constructed as a dynamic functional connectivity adjacency matrix ;

[0069] S42. Calculate the dynamic functional connectivity adjacency matrix Dynamic Laplace normalized adjacency matrix of :

[0070]

[0071] in, is the identity matrix; is the Laplace normalized adjacency matrix The diagonal matrix of

[0072] S43, multiplexing and dividing the EEG resting signal into segments of 1s and the dynamic Laplace normalized adjacency matrix As nodes and edges in GCN, we learn the spatial topological features at a certain time point. ;

[0073] S44, repeat S41 to S43 until the spatial topological features of all time points are calculated , and the spatial topological features of all time points Splicing is the integration of spatial topological features at each time point ;

[0074] S45. Integrate the spatial topological features of each time point Input into the bidirectional LSTM to capture changes at different time points and learn time-dependent temporal features ;

[0075] S46, using attention pooling for temporal features Automatically process to obtain a set of modulation weights, and then weight the set of modulation weights to obtain highly discriminative time series features :

[0076]

[0077] in, is the SoftMax activation function; It is a fully connected layer in the bidirectional LSTM.

[0078] A device for fusion of dynamic and static functional brain networks spatiotemporal topology based on EEG, the device comprising:

[0079] An acquisition module, used for acquiring original EEG resting signals based on an EEG acquisition device, and preprocessing the original EEG resting signals to obtain an EEG resting signal segment sequence;

[0080] A signal division module is used to perform channel multiplexing and brain region division processing on the EEG resting signal segment sequence to obtain a multiplexed and divided EEG resting signal segment sequence and a multiplexed and divided continuous EEG resting signal segment sequence composed of multiple 1-second segments;

[0081] The signal static processing module is used to input the multiplexed and divided EEG resting signal segment sequence into the static functional connectivity spatial topology learning module to obtain highly discriminative local and global spatial topological features;

[0082] The signal dynamic processing module is used to input the multiplexed and divided continuous EEG resting signal segment sequence into the dynamic functional connectivity time series learning module to obtain highly discriminative time series features;

[0083] The classification module is used to splice the highly discriminative local and global spatial topological features and the highly discriminative temporal features, and input the splicing results into the multi-layer perceptron classification layer to obtain the classification results of the target brain cognitive resting task.

[0084] The method and device for fusion of dynamic and static functional brain network spatiotemporal topology based on EEG provided by the present invention have the following beneficial effects:

[0085] 1. The present invention uses the original EEG resting signal to construct dynamic functional connectivity and static functional connectivity according to time division, and forms a functional brain network; at the same time, based on GCN, the sequential characteristics of specific nodes in the graph domain and the characteristics of topological association characteristics across nodes are used to process highly multi-relational graph data, and then explore the characteristics of the functional brain network in the constructed brain cognitive resting task; and according to different brain regions and the whole brain, discriminative features are extracted locally and globally, thereby improving the model performance and solving the problem of limited feature extraction.

[0086] 2. When exploring the functional brain network, the present invention considers the two scales of dynamic functional connectivity and static functional connectivity, fully considering the spatiotemporal changes of brain activity, thereby improving the performance of the model.

[0087] 3. The present invention analyzes the brain regions locally and the whole brain globally, and links the brain regions with the whole brain, which can better explore the functional connections in the brain network and improve the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 It is a flow chart of the spatiotemporal topological fusion method of dynamic and static functional brain networks based on EEG of the present invention;

[0089] Figure 2 This is a schematic diagram of brain area division based on channel multiplexing of the present invention;

[0090] Figure 3 This is the overall architecture diagram of the present invention based on the dynamic and static functional connectivity brain network;

[0091] Figure 4 This is a structural diagram of a static functional connected space topology learning module of the present invention;

[0092] Figure 5 This is a structural diagram of the attention module of brain area functional connectivity of the present invention;

[0093] Figure 6 This is a structural diagram of the dynamic functional connectivity timing learning module of the present invention. DETAILED DESCRIPTION

[0094] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0095] Example 1

[0096] This embodiment provides a method for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG, referring to Figure 1 and Figure 3 , which specifically includes the following steps:

[0097] Step S1: Collecting the original EEG resting signal based on the EEG acquisition device , and the original EEG resting signal Preprocessing to obtain EEG resting signal fragment sequence , which specifically includes the following sub-steps:

[0098] Step S11, using an EEG acquisition device to continuously acquire 19-channel original EEG resting signals of a 10-20 international standard lead system;

[0099] Step S12, performing 0.5-50 Hz bandpass filtering and 50 Hz notch filtering on the original EEG resting signal, and downsampling the filtered EEG resting signal to 128 Hz;

[0100] Step S13: perform data enhancement on the filtered and downsampled EEG resting signal by overlapping the 50% sliding window and divide it into 5-second EEG resting signal segment sequences. .

[0101] Step S2: perform channel multiplexing and brain region division processing on the EEG resting signal segment sequence to obtain a multiplexed and divided EEG resting signal segment sequence. , and at the same time copy a sliding window with 50% overlap in the time dimension to obtain a multiplexed and divided continuous EEG resting signal segment sequence consisting of multiple 1-second segments ;

[0102] refer to Figure 2 In this embodiment, some of the 19 channels of the 10-20 international standard lead system are reused and divided into multiple brain regions, including:

[0103] Left frontal area: Fp1, F7, F3;

[0104] Right frontal area: Fp2, F4, F8;

[0105] Left temporal lobe area: T3, T5, F7;

[0106] Central area: C3, Cz, C4;

[0107] Right temporal lobe area: T4, T6, F8;

[0108] Parietal area: P3, Pz, P4;

[0109] Occipital region: O1, O2, Pz;

[0110] Among them, Fp1, F7, and F3 in the left frontal area are part of the channels of the frontal lobe of the 10-20 international standard lead system, and the acquisition position is the left front of the human brain; Fp2, F4, and F8 in the right frontal area are part of the channels of the frontal lobe of the 10-20 international standard lead system, and the acquisition position is the right front of the human brain; T3 and T5 in the left temporal area are part of the channels of the temporal lobe of the 10-20 international standard lead system, and the acquisition position is the left middle of the human brain, and F7 is the multiplexed channel of the left frontal area; C3, Cz, and C4 in the central area are part of the channels of the central area of ​​the 10-20 international standard lead system For some channels, the acquisition position is the middle of the human brain, located behind the frontal lobe and in front of the parietal lobe; T4 and T6 in the right temporal lobe area are some channels of the temporal lobe in the 10-20 international standard lead system, and the acquisition position is the right middle of the human brain, and F8 is the multiplexed channel of the right frontal lobe area; P3, Pz, and P4 in the parietal lobe area are some channels of the parietal lobe in the 10-20 international standard lead system, and the acquisition position is the middle and posterior of the human brain; O1 and O2 in the occipital lobe area are some channels of the occipital lobe in the 10-20 international standard lead system, and the acquisition position is the posterior part of the human brain, and Pz is the multiplexed channel of the parietal lobe area.

[0111] In this embodiment, the priority for the frontal lobe region is to select EEG channels to enter the divided brain regions to measure changes in functional connectivity characteristics. Therefore, the Fz channel recorded by the frontal central electrode is not divided into any brain region and is excluded from further analysis.

[0112] Step S3, inputting the multiplexed and divided EEG resting signal segment sequence into the static functional connectivity spatial topology learning module to obtain highly discriminative local and global spatial topological features;

[0113] The static functional connectivity spatial topology learning module of this embodiment includes a local functional connectivity learning module within a brain region and a global functional connectivity learning module between brain regions.

[0114] Specifically, refer to Figure 4 , this step S3 specifically includes the following sub-steps:

[0115] Step S31: Use fast Fourier transform to obtain multiplexed and divided EEG resting signal segment sequences Frequency domain characteristics , and then use the squeeze excitation network to adjust the frequency domain features Perform feature weighted extraction to obtain the aggregated feature vector :

[0116]

[0117] in, is the squeezing function; and is the frequency domain feature The first two dimensions of is the channel dimension, is the frequency domain dimension; is the value obtained by traversing the channel dimension C and the frequency domain dimension F; s is the frequency domain feature The number of segments obtained by division; They are temporary variables for traversing the channel dimension C and the frequency domain dimension F respectively;

[0118] The squeezing operation in this embodiment is to globally pool the feature vector in the channel and frequency domain dimensions.

[0119] Step S32: aggregate feature vectors Each feature in is sample activated to obtain weighted frequency domain features :

[0120]

[0121] in, is the activation function; is the ReLU activation function, is the Sigmoid activation function; It is a general term for the fully connected layer; and There are two fully connected layers;

[0122] Step S33: weight the frequency domain features in the channel dimension Divide the brain regions to obtain weighted frequency domain feature sequences of multiple brain regions ;

[0123] Step S34: weight the frequency domain feature sequences of multiple brain regions Input into the local functional connectivity learning module within the brain region to learn the integration of brain region weights and the integration of highly discriminative local spatial topological features ; Among them, the learning method of the local functional connectivity learning module in the brain region includes:

[0124] Step S341: Obtain weighted frequency domain feature sequences of multiple brain regions Weighted frequency domain features of single brain regions in , calculate the weighted frequency domain features of a single brain region Functional connectivity index iCOH (imaginary part of coherency) and constructed as a static functional connectivity adjacency matrix :

[0125]

[0126] in, represents the imaginary component, is the cross-spectral density, and All are autospectral densities;

[0127] Step S342: connect the static function adjacency matrix Input into the brain region functional connectivity attention module to learn the Laplace normalized adjacency matrix of the brain region and brain region weights , where reference Figure 5 , the learning method of the attention module of brain region functional connectivity includes:

[0128] Step S3421: Use a multi-scale selective kernel attention block to input the static functional connectivity adjacency matrix Perform weighted feature extraction, integrate and activate the extracted features:

[0129]

[0130]

[0131] in, for and The integration and addition of two features; and Represents the static functional connectivity adjacency matrix Features obtained by one-dimensional convolution using convolution kernel sizes of 1*1 and 3*3; For the entire activation process; is the LeakyReLU function;

[0132] Step S3422: Apply the Softmax activation function to the integrated and activated features, and adaptively select static functional connectivity features of different scales according to the integrated feature description after activation to calculate the weighted static functional connectivity adjacency matrix. :

[0133]

[0134]

[0135]

[0136] in, So An exponential function with a as the base, a and b are attention weights of different scales;

[0137] Step S3423: Calculate the weighted static functional connectivity adjacency matrix The Laplace normalized adjacency matrix of the brain region ;

[0138] Step S3424: Calculate brain region weights based on attention weights of different scales :

[0139]

[0140] in, To find the maximum value function of the matrix, Integrating attention weights for multiple scales.

[0141] Step S343: weight the frequency domain features of the single brain region and the Laplace normalized adjacency matrix of brain regions They are used as nodes and edges in GCN (graphical convolutional neural network) to learn the local spatial topological features of the brain area. :

[0142]

[0143] in, For a simple GCN function, is the batch normalization function, is the ReLU activation function;

[0144] Step S344, repeat S341 to S343 until the brain region weights of all brain regions are calculated and local spatial topological characteristics of brain regions , the brain region weights of all brain regions and local spatial topological features of all brain regions Splice them separately to get the integration of brain area weights and the integration of highly discriminative local spatial topological features ;

[0145] Step S35: weight the frequency domain features Integration with brain region weights Input into the global functional connectivity learning module between brain regions to learn highly discriminative global spatial topological features , wherein the learning method of the inter-brain global functional connectivity learning module includes:

[0146] Step S351: Calculate weighted frequency domain features Functional connectivity index iCOH, and constructed as a global static functional connectivity adjacency matrix ;

[0147] Step S352: Integrate the brain area weights Connect adjacency matrix with global static function Multiply them together to get a weighted global static functional connectivity adjacency matrix similar to channel attention :

[0148]

[0149] in, is the function used to expand the dimension, represents element-wise multiplication;

[0150] Step S353: Calculate the weighted global static functional connectivity adjacency matrix The global Laplace normalized adjacency matrix of ;

[0151] Step S354: weight the frequency domain features and the global Laplace normalized adjacency matrix As nodes and edges in GCN, respectively, we learn highly discriminative global spatial topological features .

[0152] Step S36: Integrate the highly discriminative local spatial topological features and highly discriminative global spatial topological features Splicing to obtain highly discriminative local and global spatial topological features .

[0153] Step S4: Input the multiplexed and divided continuous EEG resting signal segment sequence into the dynamic functional connectivity time series learning module to obtain high discriminative time series features. Figure 6 , which specifically includes the following sub-steps:

[0154] Step S41: Get multiplexed and divided continuous EEG resting signal segment sequence 1s multiplexing to divide EEG resting signal segments , calculate 1s multiplexing and dividing EEG resting signal segments The functional connectivity index iCOH is constructed as a dynamic functional connectivity adjacency matrix ;

[0155] Step S42: Calculate the dynamic functional connectivity adjacency matrix Dynamic Laplace normalized adjacency matrix of :

[0156]

[0157] in, is the identity matrix; is the Laplace normalized adjacency matrix The diagonal matrix of

[0158] Step S43: multiplex and divide the EEG resting signal into segments of 1s and the dynamic Laplace normalized adjacency matrix As nodes and edges in GCN, we learn the spatial topological features at a certain time point. ;

[0159] Step S44: Repeat S41 to S43 until the spatial topological features of all time points are calculated. , and the spatial topological features of all time points Splicing is the integration of spatial topological features at each time point ;

[0160] Step S45: Integrate the spatial topological features of each time point Input into the bidirectional LSTM to capture changes at different time points and learn time-dependent temporal features ;

[0161] Step S46: Use attention pooling to analyze the temporal features Automatically process to obtain a set of modulation weights, and then weight the set of modulation weights to obtain highly discriminative time series features :

[0162]

[0163] in, is the SoftMax activation function; It is a fully connected layer in the bidirectional LSTM.

[0164] Step S5, concatenate the highly discriminative local and global spatial topological features and the highly discriminative temporal features, and input the concatenated results into the multi-layer perceptron classification layer after passing through the fully connected layer to obtain the classification results of the target brain cognitive resting task.

[0165] Example 2

[0166] A device for fusion of spatiotemporal topology of dynamic and static functional brain networks based on EEG, characterized in that the device comprises:

[0167] An acquisition module, used for acquiring original EEG resting signals based on an EEG acquisition device, and preprocessing the original EEG resting signals to obtain an EEG resting signal segment sequence;

[0168] A signal division module is used to perform channel multiplexing and brain region division processing on the EEG resting signal segment sequence to obtain a multiplexed and divided EEG resting signal segment sequence and a multiplexed and divided continuous EEG resting signal segment sequence composed of multiple 1-second segments;

[0169] The signal static processing module is used to input the multiplexed and divided EEG resting signal segment sequence into the static functional connectivity spatial topology learning module to obtain highly discriminative local and global spatial topological features;

[0170] The signal dynamic processing module is used to input the multiplexed and divided continuous EEG resting signal segment sequence into the dynamic functional connectivity time series learning module to obtain highly discriminative time series features;

[0171] The classification module is used to splice the highly discriminative local and global spatial topological features and the highly discriminative temporal features, and input the splicing results into the multi-layer perceptron classification layer to obtain the classification results of the target brain cognitive resting task.

[0172] Although the specific implementation of the invention is described in detail in conjunction with the drawings, it should not be understood as limiting the scope of protection of this patent. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still fall within the scope of protection of this patent.

Claims

1. A method for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG, characterized in that: The following steps are involved: S1. Collecting original EEG resting signals based on EEG acquisition equipment, and preprocessing the original EEG resting signals to obtain EEG resting signal segment sequences; S2, performing channel multiplexing and brain region division processing on the EEG resting signal segment sequence to obtain a multiplexed and divided EEG resting signal segment sequence and a multiplexed and divided continuous EEG resting signal segment sequence consisting of multiple 1-second segments; S3, inputting the multiplexed and divided EEG resting signal segment sequence into the static functional connectivity spatial topology learning module to obtain highly discriminative local and global spatial topological features; S4, inputting the multiplexed and divided continuous EEG resting signal segment sequence into the dynamic functional connectivity temporal learning module to obtain highly discriminative temporal features; S5, splicing the highly discriminative local and global spatial topological features and the highly discriminative temporal features, and inputting the splicing results into the multi-layer perceptron classification layer to obtain the classification results of the brain cognitive resting task; The S3 specifically includes the following sub-steps: S31. Using Fast Fourier Transform to Acquire Multiplexed EEG Resting Signal Segment Sequences Frequency domain characteristics , and then use the squeeze excitation network to adjust the frequency domain features Perform feature weighted extraction to obtain aggregated feature vector : ； in, is the squeezing function; and is the frequency domain feature The first two dimensions of is the channel dimension, is the frequency domain dimension; is the value obtained by traversing the channel dimension C and the frequency domain dimension F; s is the frequency domain feature The number of segments obtained by division; They are temporary variables for traversing the channel dimension C and the frequency domain dimension F respectively; S32, aggregate feature vector Each feature in is sample activated to obtain weighted frequency domain features : ； in, is the activation function; is the ReLU activation function, is the Sigmoid activation function; It is a general term for the fully connected layer; and There are two fully connected layers; S33. Weighted frequency domain features in the channel dimension Divide the brain regions to obtain weighted frequency domain feature sequences of multiple brain regions ; S34, weighted frequency domain feature sequences of multiple brain regions Input into the local functional connectivity learning module within the brain region to learn the integration of brain region weights and the integration of highly discriminative local spatial topological features ; S35, weighted frequency domain features Integration with brain region weights Input into the global functional connectivity learning module between brain regions to learn highly discriminative global spatial topological features ; S36. Integration of highly discriminative local spatial topological features and highly discriminative global spatial topological features Splicing to obtain highly discriminative local and global spatial topological features ; The S4 specifically includes the following sub-steps: S41, multiplex and divide the continuous EEG resting signal segment sequence 1s multiplexing to divide EEG resting signal segments , calculate 1s multiplexing and dividing EEG resting signal segments The functional connectivity index iCOH is constructed as a dynamic functional connectivity adjacency matrix ; S42. Calculate the dynamic functional connectivity adjacency matrix Dynamic Laplace normalized adjacency matrix of : ； in, is the identity matrix; is the Laplace normalized adjacency matrix The diagonal matrix of S43, multiplexing and dividing the EEG resting signal into segments of 1s and the dynamic Laplace normalized adjacency matrix As nodes and edges in GCN, we learn the spatial topological features at a certain time point. ; S44, repeat S41 to S43 until the spatial topological features of all time points are calculated , and the spatial topological features of all time points Splicing is the integration of spatial topological features at each time point ; S45. Integrate the spatial topological features of each time point Input into the bidirectional LSTM to capture changes at different time points and learn time-dependent temporal features ; S46, using attention pooling for temporal features Automatically process to obtain a set of modulation weights, and then weight the set of modulation weights to obtain highly discriminative time series features : ； in, is the SoftMax activation function; It is a fully connected layer in the bidirectional LSTM.

2. The method for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG according to claim 1, characterized in that: The S1 specifically includes the following sub-steps: S11, using EEG acquisition equipment to continuously collect 19 channels of raw EEG resting signals of the 10-20 international standard lead system; S12, performing 0.5-50 Hz bandpass filtering and 50 Hz notch filtering on the original EEG resting signal, and downsampling the filtered EEG resting signal to 128 Hz; S13, overlap the 50% sliding window of the filtered and downsampled EEG resting signal and divide it into 5-second EEG resting signal segment sequences .

3. The method for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG according to claim 1, characterized in that: In S2, some of the 19 channels of the 10-20 international standard lead system are reused and divided into multiple brain regions, including: Left frontal area: Fp1, F7, F3; Right frontal area: Fp2, F4, F8; Left temporal lobe area: T3, T5, F7; Central area: C3, Cz, C4; Right temporal lobe area: T4, T6, F8; Parietal area: P3, Pz, P4; Occipital region: O1, O2, Pz; Among them, Fp1, F7, and F3 in the left frontal area are part of the channels of the frontal lobe of the 10-20 international standard lead system, and the acquisition position is the left front of the human brain; Fp2, F4, and F8 in the right frontal area are part of the channels of the frontal lobe of the 10-20 international standard lead system, and the acquisition position is the right front of the human brain; T3 and T5 in the left temporal area are part of the channels of the temporal lobe of the 10-20 international standard lead system, and the acquisition position is the left middle of the human brain, and F7 is the multiplexed channel of the left frontal area; C3, Cz, and C4 in the central area are part of the channels of the central area of ​​the 10-20 international standard lead system For some channels, the acquisition position is the middle of the human brain, located behind the frontal lobe and in front of the parietal lobe; T4 and T6 in the right temporal lobe area are some channels of the temporal lobe in the 10-20 international standard lead system, and the acquisition position is the right middle of the human brain, and F8 is the multiplexed channel of the right frontal lobe area; P3, Pz, and P4 in the parietal lobe area are some channels of the parietal lobe in the 10-20 international standard lead system, and the acquisition position is the middle and posterior of the human brain; O1 and O2 in the occipital lobe area are some channels of the occipital lobe in the 10-20 international standard lead system, and the acquisition position is the posterior part of the human brain, and Pz is the multiplexed channel of the parietal lobe area.

4. The method for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG according to claim 1, characterized in that: The learning method of the S34 midbrain region local functional connectivity learning module includes: S341, obtain weighted frequency domain feature sequences of multiple brain regions Weighted frequency domain features of single brain regions in , calculate the weighted frequency domain features of a single brain region Functional connectivity index iCOH, and constructed as a static functional connectivity adjacency matrix : ； in, represents the imaginary component, is the cross-spectral density, and All are autospectral densities; S342, connect the static function adjacency matrix Input into the brain region functional connectivity attention module to learn the Laplace normalized adjacency matrix of the brain region and brain region weights ; S343, weighted frequency domain features of single brain region and the Laplace normalized adjacency matrix of brain regions As nodes and edges in GCN, we learn the local spatial topological features of the brain region. : ； in, For a simple GCN function, is the batch normalization function; S344, repeat S341 to S343 until the brain region weights of all brain regions are calculated and local spatial topological characteristics of brain regions , the brain region weights of all brain regions and local spatial topological features of all brain regions Splice them separately to get the integration of brain area weights and the integration of highly discriminative local spatial topological features .

5. The method for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG according to claim 4 is characterized in that: The learning method of the S342 midbrain functional connectivity attention module includes: S3421, Static functional connectivity adjacency matrix of input using multi-scale selective kernel attention blocks Perform weighted feature extraction, integrate and activate the extracted features: ； ； in, for and The integration and addition of two features; and Represents the static functional connectivity adjacency matrix Features obtained by one-dimensional convolution using convolution kernel sizes of 1*1 and 3*3; For the entire activation process; is the LeakyReLU function; S3422, according to the characteristics and , calculate the weighted static functional connectivity adjacency matrix : ； ； ； in, So An exponential function with a as the base, a and b are attention weights of different scales; S3423. Calculate weighted static functional connectivity adjacency matrix The Laplace normalized adjacency matrix of the brain region ; S3424. Calculate brain region weights based on attention weights of different scales : ； in, To find the maximum value function of the matrix, Integrating attention weights for multiple scales.

6. The method for spatiotemporal topological fusion of dynamic and static functional brain networks based on EEG according to claim 1, characterized in that: The learning method of the S35 midbrain region global functional connectivity learning module includes: S351. Calculate weighted frequency domain features Functional connectivity index iCOH, and constructed as a global static functional connectivity adjacency matrix ; S352, integration of brain area weights Connect adjacency matrix with global static function Multiply them together to get a weighted global static functional connectivity adjacency matrix similar to channel attention : ； in, is the function used to expand the dimension, represents element-wise multiplication; S353, calculate weighted global static functional connectivity adjacency matrix The global Laplace normalized adjacency matrix of ; S354, weighted frequency domain features and the global Laplace normalized adjacency matrix As nodes and edges in GCN, respectively, we learn highly discriminative global spatial topological features .

7. A device for implementing the EEG-based dynamic and static functional brain network spatiotemporal topological fusion method according to claim 1, characterized in that: The device includes: An acquisition module, used for acquiring original EEG resting signals based on an EEG acquisition device, and preprocessing the original EEG resting signals to obtain an EEG resting signal segment sequence; A signal division module is used to perform channel multiplexing and brain region division processing on the EEG resting signal segment sequence to obtain a multiplexed and divided EEG resting signal segment sequence and a multiplexed and divided continuous EEG resting signal segment sequence composed of multiple 1-second segments; The signal static processing module is used to input the multiplexed and divided EEG resting signal segment sequence into the static functional connectivity spatial topology learning module to obtain highly discriminative local and global spatial topological features; The signal dynamic processing module is used to input the multiplexed and divided continuous EEG resting signal segment sequence into the dynamic functional connectivity time series learning module to obtain highly discriminative time series features; The classification module is used to splice the highly discriminative local and global spatial topological features and the highly discriminative temporal features, and input the splicing results into the multi-layer perceptron classification layer to obtain the classification results of the target brain cognitive resting task.

Images