Epilepsy abnormal brain network identification method based on multi-scale static-dynamic fusion network

Abstract

The invention discloses an epilepsy abnormal brain network identification method based on a multi-scale static-dynamic fusion network, and belongs to the field of brain image analysis. The method comprises the following steps: firstly, constructing a static function connection weighted graph and a dynamic function connection graph; and fusing the static and dynamic representations by adopting a cross attention module. In order to describe a multi-scale spatial relationship, performing lexical meta-processing on brain connection according to anatomical partition and a functional network; and the local-global fusion module is used for integrating the fine granularity and the macroscopic relationship, so that the brain region with diagnostic significance is highlighted. In the training stage, cross entropy, reverse contrast loss and sparse regularization based on contrast graph adjacency matrix entropy are jointly used. The method is verified on multi-center functional magnetic resonance data, compared with other mainstream depth models, the classification accuracy, generalization and interpretability are remarkably improved, an abnormal brain region consistent with an epilepsy network can be positioned, and brain image markers with biological significance can be connected and recognized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of brain image analysis and artificial intelligence assisted diagnosis, and particularly relates to a graph neural network modeling method based on functional magnetic resonance imaging (fMRI) image data, in particular, a multi-scale deep learning framework fusing static-dynamic brain connection features and local-global network representation for abnormal network identification of brain diseases such as epilepsy and biomarker extraction. BACKGROUND

[0002] In recent years, resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used in epilepsy brain network research. By dividing the brain into a plurality of regions of interest (ROI) and calculating the functional connectivity (FC) thereof, a functional connectivity graph reflecting the synchronization relationship of neural activity can be constructed. Traditional methods mostly use full time series averaging or sliding window strategy to depict brain networks, but due to the strong spatiotemporal variation and nonlinearity of neural activity in epilepsy, single-scale or single-time modeling methods are difficult to fully reveal the underlying mechanism.

[0003] Graph neural network (GNN) is a deep learning method suitable for non-Euclidean structure data, which has been introduced into brain network classification tasks and has good feature expression ability. However, existing traditional GNN methods and GNN methods based on brain imaging still face the following key challenges when dealing with highly heterogeneous brain diseases such as epilepsy: (1) Static and dynamic features are modeled separately, and coupling information is missing: Most models only focus on static FC or simply stack dynamic window graphs, lack effective static-dynamic fusion strategies, and limit the model's ability to capture complex epilepsy network patterns. (2) Limited understanding of local and global structures: Traditional GNNs often model from the whole graph level, but fail to distinguish the internal structural differences of functional networks such as vision, hearing, and limbic system, ignoring local representation learning within the network and global dependence across networks. (3) Poor model interpretability and weak generalization ability: Epilepsy image data often presents characteristics such as small sample, multiple centers, and large individual differences. Existing models mostly rely on large-scale single-center data, lack mechanism-based explanation ability, and have weak biological interpretability of the discrimination results.

[0004] Therefore, there is an urgent need for a deep learning method that simultaneously fuses static and dynamic features, local and global information, and has good interpretability and cross-center robustness. SUMMARY

[0005] The present application aims at the deficiencies of the background art, and provides a multi-scale static-dynamic fusion network for identifying abnormal brain networks of epilepsy. Through joint modeling of static and dynamic, multi-scale feature coding, cross attention mechanism and cross-modal fusion technology, a classification framework of epilepsy brain network with discriminability, generalizability and explainability is constructed to classify the brain networks of patients with temporal lobe epilepsy (TLE) and healthy controls (HC). Specifically, the present application first introduces joint modeling of static and dynamic functional connectivity to extract static and dynamic features of brain function, and improves the significance of the features through a weighting mechanism to solve the problem of incomplete static information and redundant dynamic information in traditional methods. Secondly, the present application introduces a cross-modal cross-attention mechanism to fuse static and dynamic feature representations and realize coupling modeling between brain network spatio-temporal features. Further, the present application proposes a dual-scale segmentation strategy based on anatomical structure and functional network to divide the brain map into structure and function subgraphs, generate multi-scale Token representation, and improve the ability of the model to capture local and global dependent structures. Finally, through the construction of a local-global feature fusion module and an attention weighting mechanism, the abnormal brain regions and key connections in the epilepsy brain network are accurately captured to realize effective classification of TLE and HC. The system not only significantly improves the recognition performance of the epilepsy brain network, but also has high explainability, supports the positioning of functional imbalance brain regions related to seizures, and has important clinical application value.

[0006] The technical scheme of the present application is: an epilepsy abnormal brain network identification method based on a multi-scale static-dynamic fusion network, which comprises 7 modules:

[0007] (1) a magnetic resonance functional image acquisition module;

[0008] used for acquiring functional fMRI data of patients, establishing an initial functional connection graph taking brain ROI as nodes and functional connection strength between brain regions as edges; fMRI represents magnetic resonance imaging, and ROI represents a region of interest;

[0009] (2) a functional image data preprocessing module;

[0010] The acquired fMRI data is subjected to standard neuroimaging processing, including image format conversion, removal of time points, head motion correction, spatial registration, image smoothing, covariate regression and signal filtering operations; and static FC and dFC are calculated to provide input for subsequent graph neural modeling; FC represents functional connection strength, and dFC represents dynamic functional connection matrix;

[0011] (3) a dynamic and static weighting module;

[0012] The static FC and dynamic dFC are respectively input into a static weighting submodule and a dynamic graph modeling module: the static part weights the functional connection matrix through a channel attention mechanism to highlight discriminative connections; the dynamic part processes the dFC graph sequence through a diffusion space attention module, introduces multi-hop graph convolution and spatial encoding, extracts potential dynamic topological features, and enhances the modeling capability of the model for indirect connections to obtain static features and dynamic features;

[0013] (4) Cross-attention mechanism module;

[0014] Through a self-attention mechanism, the static features and dynamic features are taken as inputs to learn the joint representation of the two, and static-dynamic fusion features are obtained;

[0015] (5) Multi-scale tokenization module;

[0016] Based on the prior anatomical brain region and functional network division, the brain region is regionally divided to generate two types of structural domain Token and functional domain Token representations, and the information expression within the local subgraph is modeled;

[0017] (6) Local-global feature fusion module;

[0018] The structural domain Token and functional domain Token are respectively extracted through residual convolution and regional attention mechanism, and then local-global feature interaction fusion is performed through a fully connected attention module to form a high-dimensional diagnostic discriminative graph-level representation;

[0019] (7) Epileptic brain disease network classification module;

[0020] The above fused graph-level representation is input into the final classifier module to complete the TLE and normal binary classification; at the same time, the model embeds two kinds of contrast loss and saliency mechanism, has a certain degree of explainability, and can output key abnormal connection atlas and abnormal brain region list for clinical auxiliary analysis; TLE represents the brain network of patients with temporal lobe epilepsy.

[0021] Further, the following calculation steps are adopted in the functional image data preprocessing module:

[0022] Step 1: Obtain magnetic resonance image data of a plurality of epilepsy patients and normal persons, here using the functional item data of resting-state fMRI; perform image format conversion, time point removal, head motion correction, image registration, image smoothing, covariate regression, and signal filtering data preprocessing operations on the functional image data, and the specific steps are as follows:

[0023] Step A1: Convert the original DICOM file exported from the magnetic resonance machine into a NIFTI file suitable for neuroimaging analysis (image format conversion);

[0024] Step A2: remove the first n time points of functional data to make the signal more stationary (time point removal);

[0025] Step A3: calculate head motion parameters at different time points to correct the effects introduced by head motion (head motion correction);

[0026] Step A4: map functional images to a standard brain spatial template (image registration);

[0027] Step A5: spatially smooth the registered images to enhance the signal-to-noise ratio (image smoothing);

[0028] Step A6: remove interference signals from brain signals (covariate regression);

[0029] Step A7: filter out low-frequency drift and high-frequency noise in brain signals (signal filtering).

[0030] Further, the specific calculation method of the static graph weighting module is:

[0031] Step C1: construct a static weighted graph;

[0032] Each static graph , is composed of a node feature matrix and an adjacency matrix , the adjacency matrix is the static FC calculated in step 2, and the node feature matrix is obtained by linear layer mapping of the static FC, denotes a set of real numbers of Nxd dimensions, where the number of brain regions, is the dimension of the node feature;

[0033] Step C2: weight the adjacency matrix of the static graph;

[0034] For any two ROIs: and , calculate the node similarity between their node features , , is the absolute value of the Pearson correlation coefficient; and calculate the spatial distance between the two according to the spatial coordinates of the ROIs , model its spatial similarity through a Gaussian kernel function: , where denotes the spatial distance standard deviation of all ROIs in the graph; then the adjacency matrix of the weighted static graph is:

[0035] , ;

[0036] The weighted static graph is , wherein ;

[0037] Subsequently, an adjacency matrix of the weighted static graph is adopted ;

[0038] Step C3: contrastive pooling

[0039] Let two groups of graphs in the training set be:

[0040] , , , ;

[0041] and respectively represent all unweighted static functional connection graphs of the HC group and the TLE group in the training set; , respectively represent the node feature matrix and the adjacency matrix of the normal brain network, , respectively represent the node feature matrix and the adjacency matrix of the temporal lobe epilepsy brain network; subsequently, a double attention mechanism is used to perform attention weighting on the node features of the two groups of subjects from the ROI scale and the individual scale respectively;

[0042] The attention weighting from the ROI scale is calculated as :

[0043] ;

[0044] Subsequently, the attention weighting from the subject scale is calculated as :

[0045] ;

[0046] wherein the attention mechanism : ;

[0047] ; is a learnable weight matrix, denotes normalization processing on a vector;

[0048] Next, the node features and the neighbor matrices of the TLE group and the HC group are averaged within the groups respectively to obtain

[0049] , , , ;

[0050] Finally, the difference between the two groups of graphs is calculated to obtain a contrast graph :

[0051] = ⊖ , = ⊖ ;

[0052] ;

[0053] where ⊖ is the calculation of the absolute difference between two matrices element by element; the l-layer pooling network is used to aggregate the features of the contrast graph:

[0054] = ( ));

[0055] , is input into the l-layer embedded to obtain the embedding layer feature matrix :

[0056] = ( ) ;

[0057] According to the clustering assignment matrix and the embedding layer node feature matrix , the coarsened feature matrix and the new adjacency matrix can be generated:

[0058] ;

[0059] ;

[0060] This coarsening process can reduce the number of nodes, thereby obtaining a higher level of node representation. Through the above calculation process, high-level node representation can be extracted under the guidance of the contrast graph, thereby generating a high-quality static graph representation for each input graph:

[0061] ;

[0062] Step C4: static graph feature aggregation;

[0063] After obtaining the coarsened static graph , in order to further mine its high-order topological features and node representation, this step inputs it into a multi-layer graph convolutional network GCN for weighted aggregation.

[0064] In GCN, let the input be the node feature matrix. Adjacency Matrix Then the propagation rule for the k-th layer is:

[0065] ;

[0066] in, This represents the adjacency matrix after adding self-connections. Its degree matrix, For the first The trainable weight parameters of the layer, (⋅) is a nonlinear activation function;

[0067] pass The propagation and aggregation operations of layered GCNs can yield the final static graph feature representation:

[0068] ;

[0069] This representation not only preserves the differential features guided by the comparison map, but also further extracts high-order structural information through convolutional aggregation, providing stable and reliable static feature support for subsequent dynamic graph branch modeling and overall classification and recognition tasks.

[0070] Furthermore, the specific calculation method for the dynamic graph weighting module is as follows:

[0071] Step D1: Construction of the dynamic functional connectivity graph. As shown in Step 2, each participant can obtain T dynamic graphs using the sliding window method, i.e. With any time window The following image For example, in the current window The following diagram is composed of node features and adjacency matrix The set constitutes the set; the set of node features. The set of adjacency matrices , where node feature vector Defined as:

[0072] ;

[0073] in, For the spatial one-hot encoding of the nth node, The timestamp encoding feature is a learnable feature derived from a gated recurrent unit (GRU). For connection operations, The learnable parameters are the linear mappings of the features; while the features of the edges... It is obtained by calculating the Pearson correlation of pairwise ROIs under each time window;

[0074] Step D2: Designing the diffusion space attention module.

[0075] Define the time window The graph The representation of the dynamic node is , represents The input feature of the l-th layer attention module at time point is denoted as , and is defined as , , represents the query vector, represents the search vector, represents the numerical vector, and the superscript represents the l-th layer of the attention module. The attention matrix is calculated as follows:

[0076] ;

[0077] ;

[0078] represents the element-wise product, represents the convolution operation, represents the activation of the element with the leaky linear rectifier function;

[0079] Step D3: Perform the diffusion method, and use the multi-hop diffusion method to extend the attention weighted adjacency matrix to the multi-hop attention form The diffusion process is described as:

[0080] ;

[0081] ;

[0082] ;

[0083] In the formula, represents the reduction of the dimension in the row, represents the transition probability, is the weight factor of the adjacency matrix of different hops, and > to attenuate the influence of the adjacent matrix with more hops; The power of indicates the adjacency matrix with hop paths between two points;

[0084] Step D4: independently assign weights to nodes with different path lengths in a sequential manner; for , the information propagation step is calculated according to the graph convolution operation :

[0085] ;

[0086] wherein , represents a multi-layer perceptron, is an information propagation and aggregation operation. Then an aggregated batch normalization multi-layer perceptron is used, is a trainable parameter of the multi-layer perceptron; in order to reduce the complexity of matrix power calculation, an iterative method of power approximation is adopted :

[0087] ;

[0088] ;

[0089] wherein is an intermediate variable of containing the jump attention information of , is a transition probability; when tends to infinity, converges to ;

[0090] Step D5: perform time series feature statistics and aggregation processing;

[0091] For the node hidden representation obtained in the time window , the fluctuation of the node representation at different time points in the time window is obtained by calculating the standard deviation, denoted as:

[0092] ;

[0093] wherein represents the calculation of the standard deviation in the time dimension, depicting the variation intensity of the node features in time. Then, the standard deviation feature set of all time windows is stacked and averaged in the time dimension; the final output will be used as the dynamic feature representation of the dynamic graph branch:

[0094] .

[0095] Further, the specific calculation method of the static and dynamic functional connection matrix is:

[0096] Step B1: Extract the average signal in the ROI of the output image of the functional image data preprocessing module, and time-domain standardize the signal;

[0097] Step B2: For the calculation of static FC, the static is obtained by calculating the Pearson correlation coefficient between the average BOLD time series between ROIs, and the calculation formula is as follows:

[0098]

[0099] Wherein, is the original Pearson correlation coefficient, is the total number of time points, and are the BOLD signal values of the two brain regions at the time point, and BOLD represents blood oxygen level dependence, and are the average values of and ; Apply the transformation to the functional connection matrix, and the calculation formula is as follows:

[0100] ;

[0101] Wherein, is the value after Pearson transformation;

[0102] Step B3: For the construction of dynamic functional connection matrix, the time series of ROI is divided into windows by sliding window method, and for each window, a graph composed of nodes and edges is established; Therefore, each subject establishes graphs composed of nodes and edges, and for the whole time series, there are , wherein is a graph sequence containing graphs, and the nodes in the graph represent different ROIs, and the edges represent the Pearson correlation coefficient between the average BOLD time series between two ROIs.

[0103] Further, the specific calculation method of the cross attention mechanism module is:

[0104] Step E1: Generate attention representation of static and dynamic graphs;

[0105] The static graph is represented as , and the dynamic graph is represented as ​​​Layer normalization; then, the two are input into linear transformation layer respectively to obtain corresponding query vector, key vector, value vector:

[0106] ;

[0107] ;

[0108] ;

[0109] wherein, , , , , , are trainable weight parameter matrices. By calculating the element-wise product of the query and the key and normalizing, the attention weights of the static branch and the dynamic branch are obtained:

[0110] ;

[0111] Step E2: cross-fuse static and dynamic features;

[0112] After obtaining the attention weight matrix, the static feature is updated by the attention weight of the dynamic branch, and the dynamic feature is updated by the attention weight of the static branch:

[0113] ;

[0114] Then, the update results of the two are spliced: ; wherein, denotes a splicing operation, denotes element-wise multiplication; finally, the static-dynamic fusion feature is obtained.

[0115] Further, the specific calculation method of the multi-scale tokenization module is:

[0116] On the basis of the obtained static-dynamic fusion feature , this step groups all ROIs according to the partitioning rules of the domain brain area and the functional domain brain area, to obtain multi-scale token representation;

[0117] In each partition, the feature vector set of the corresponding ROI is extracted by index operation to form the token of the domain partition, denoted as:

[0118] ;

[0119] ;

[0120] , This processing method not only ensures the integrity of the structural domain features, but also highlights the network characteristics of the functional network at the macro level, thus providing a multi-level and more interpretable input representation for subsequent graph sequence modeling and classification.

[0121] Furthermore, the specific calculation method for the local and global feature fusion module is as follows:

[0122] Step G1: Local feature extraction and region-aware weight calculation. The multi-scale lexical set obtained in step six { Building upon this foundation, this step first performs residual convolution operations on the lexical units of each region or network to obtain local feature representations. Specifically, let the lexical units of a certain region be represented as:

[0123] ∈ ;

[0124] This represents the number of ROIs contained in this partition. Input feature dimension; processed by local convolutional units Perform channel-by-channel and residual mapping to obtain the updated partition features:

[0125] ;

[0126] Subsequently, the feature vector of each partition is subjected to global average pooling (GAP) along the ROI dimension to obtain a compact vector representation of that partition:

[0127] ;

[0128] By concatenating the compact representations of all partitions, we obtain the region feature matrix:

[0129] ;

[0130] in The total number of partitions is given; then, the local attention weight matrix is ​​calculated using the region-aware controller:

[0131] ;

[0132] in, The weight matrix is ​​trainable; finally, the local feature matrix is:

[0133] ;

[0134] Step G2: Global feature modeling and local-global stitching fusion;

[0135] In obtaining local features with local weight matrix After that, the local attention weight and all partition tokens are globally weighted and fused by using the multi-head attention mechanism to obtain the global feature:

[0136] ;

[0137] , ;

[0138] ;

[0139] Among them, Finally, the local feature and the global feature are spliced:

[0140] ;

[0141] Step G3: dual-path fusion of structure partition and network partition;

[0142] The fusion feature can be obtained for both structure partition and network partition ; Finally, the two results are spliced in the feature dimension to obtain a unified multi-scale fusion feature for subsequent multi-layer perceptron layer for prediction:

[0143] .

[0144] Further, the specific calculation method of the epilepsy brain disease network classification module is:

[0145] Step H1: adopt ten-fold cross-validation to divide the data set into 10 folds, of which 8 folds are training set, 1 fold is validation set, and 1 fold is training set;

[0146] Step H2: input the multi-scale fusion feature into the readout module to obtain the graph representation at the corresponding time point; the readout function is:

[0147] ;

[0148] Among them, represents the average value, represents the maximum value, is the weight value of the maximum pooling; then the graph representation is sent into the multi-layer perceptron for prediction, and the patient is classified as an epilepsy patient or a normal control;

[0149] Step H3: input the above classification result into the loss function for model optimization, and the loss function is defined as:

[0150] ;

[0151] ;

[0152] ;

[0153] ;

[0154] wherein, is the cross-entropy loss of classification, is the reverse contrastive loss, is the difference map contrastive loss, , is a hyper-parameter; for , wherein the feature vector and the feature vector are the feature vector of the sample and the feature vector of its corresponding positive sample defined from the samples within the same class; the positive sample pair is a pair of samples in the same class; is the index set of all positive samples of the batch of samples ; is the number of positive sample pairs in a small batch; denotes the cosine similarity and ; the average cosine similarity is calculated and multiplied by -1 to maximize the RCL representation of the intra-class samples; at the same time, in order to avoid the problem of over-smoothing of the graph neural network caused by the too dense adjacency matrix of the contrastive graph , an entropy constraint loss is imposed on it to make it tend to be sparse in training; wherein the th row represents the connection probability distribution of the ROI node and other nodes; wherein is used to constrain the clustering allocation to tend to be sparse;

[0155] Step H4: a gradient-based saliency map method is adopted to interpret the model, and the derivation process of the node and edge saliency map is as follows:

[0156] ;

[0157] wherein is the node feature saliency map, is the edge feature saliency map, is the input corresponding class prediction score; and reflect the contribution degree of the input features and to classification.

[0158] The application aims at the single modeling scale problem of traditional brain network recognition, and designs a multi-scale static-dynamic fusion network recognition method. The method can accurately identify the abnormal brain network of the epilepsy patient by multi-scale modeling of the resting state functional magnetic resonance data of the patient and the normal control from static-dynamic and structure-function, and through verification on multi-center functional magnetic resonance data and comparison with other mainstream models, the classification accuracy and explainability of the model of the application are significantly improved, and the abnormal brain regions and connections consistent with the epileptogenic network can be located, which has biological significance and clinical explainability. BRIEF DESCRIPTION OF DRAWINGS

[0159] Figure 1 It is a framework diagram of the "multi-scale static-dynamic fusion network" model of the application.

[0160] Figure 2 It is a comparison diagram of the prediction accuracy of the model of the application and other models on single-center data.

[0161] Figure 3 It is an important ROI distribution diagram returned by the model of the application on single-center data.

[0162] Figure 4 It is an important functional connection distribution between ROIs returned by the model of the application on single-center data. DETAILED DESCRIPTION

[0163] Firstly, the resting state functional brain image data of the temporal lobe epilepsy patients and normal controls from different sites are collected. Then the data is processed as follows:

[0164] A. fMRI data preprocessing

[0165] Step A1: based on Matlab R2018b version and DPABI_V7.0 software, first convert the source file from DICOM format to NIFTI file;

[0166] Step A2: remove the first 10 time points of the data;

[0167] Step A3: estimate the translation and rotation parameters of the head about x, y and z directions at different time points through rigid transformation, and calibrate to weaken the head motion effect;

[0168] Step A4: use EPI template to register the functional image to the Montreal Neurological Institute (MNI) standard space;

[0169] Step A5: use a Gaussian kernel with a full width at half maximum of 6 mm to spatially smooth the image;

[0170] Step A6: Linear trend, head motion parameters, and white matter and cerebrospinal fluid signals were removed using a multiple regression equation.

[0171] Step A7: Frequency domain filtering was performed using a band-pass filter with a range of 0.01-0.08 Hz. If excessive head motion (maximum translation > 3 mm, maximum rotation > 3 degrees, average frame displacement > 0.3 mm) occurred in the sample, it was removed.

[0172] B. Construction of static and dynamic functional connectivity;

[0173] Step B1: The brain signals were extracted based on the Brainnetome template containing 246 ROIs, and the average value of all voxels in each ROI was taken as the signal representation of the local brain region. Each time series was standardized based on z-score.

[0174] Step B2: The Pearson correlation coefficient between each pair of ROIs was calculated as the edge feature to obtain the static functional connectivity graph (FC).

[0175] Step B3: The sliding window method (window length = 50, step = 3) was used to construct the dynamic graph sequence to obtain the dynamic functional connectivity graph (dFC).

[0176] C. Construction and processing method of static and dynamic graph weighting modules;

[0177] Step C1: Calculate the sum of node features and adjacency matrices of static graph weighting. Each static graph , is composed of node feature matrix and adjacency matrix , where represents 246 ROIs, 128, the dimension of the node feature after linear layer weighting. The node similarity uses the absolute value of the Pearson correlation coefficient, and the spatial similarity is modeled by a Gaussian kernel function:

[0178] ;

[0179] where represents the Euclidean distance between ROIs, is the standard deviation of the Euclidean distance of all ROIs. The weighted adjacency matrix is: , ;

[0180] Step C2: Calculate the sum of node features and adjacency matrices of dynamic graph weighting. , The node features of are composed of spatial one-hot encoding After concatenation, linear mapping is performed to obtain:

[0181] ;

[0182] The Pearson correlation between each pair of ROIs in the sliding window is calculated. Then, the diffusion space attention module is constructed, and according to the definition:

[0183] ;

[0184] The diffusion method is performed (P, T) 5, transition probability ) to obtain:

[0185] ;

[0186] wherein , wherein the information propagation operation is then aggregated using a batch-normalized multi-layer perceptron, is a trainable parameter of the multi-layer perceptron, which is randomly set by the algorithm and learned autonomously.

[0187] D. Construction and processing method of cross-attention module

[0188] Step D1: normalize and linearly project the static graph representation and the dynamic graph representation to obtain query, key, and value vectors;

[0189] Step D2: calculate cross-attention weights:

[0190] ( ), ( );

[0191] and update the dynamic weight using the static weight, and update the static feature using the dynamic weight, to obtain the static-dynamic fusion feature after concatenation:

[0192] ;

[0193] E. Construction and processing method of local-global feature fusion module

[0194] Step E1: each partition token extracts local features through a residual convolution unit, and obtains a compact representation matrix wherein represents 7 structural regions or 7 functional networks. The local weight matrix is calculated using the region-aware controller to obtain the local features:​ ;

[0195] Step E2: The multi-head attention mechanism is used to fuse the local and global features, and a local weight bias is added:

[0196] ;

[0197] Finally, the local and global features are spliced: .

[0198] For anatomical partitioning and network partitioning, respectively:

[0199] ;

[0200] Finally, the local and global features are spliced:

[0201] ;

[0202] G. Construction process of the readout module and the multilayer perception machine classification module

[0203] Step G1: For the readout function, , where is set to 0.001;

[0204] Step G2: The sample set is divided using a ten-fold cross-validation method, i.e., the data set is randomly divided into 10 parts, 8 parts for training, 1 part for testing, and 1 part for validation;

[0205] Step G3: The above graph representation vector is input into the classifier to perform a binary classification task of epilepsy patients and normal controls. The classifier uses a multilayer perception machine design, consisting of two fully connected layers. The first layer reduces the input dimension to , then the feature distribution is standardized in small batches, and after nonlinear transformation by the activation function, the output classification result is obtained by inputting a fully connected layer.

[0206] H. The epilepsy patient recognition module is:

[0207] Step H1: Calculate the significance map of the TLE sample on the test sample, as well as the average significance map of 𝑀 and 𝑀𝑎 at all time points, and normalize the gradient between features, then sum the horizontal significance map of the TLE group;

[0208] Step H2: For node features, select the top 20% nodes with the largest absolute value in the node significance map (246 ROIs);

[0209] Step H3: For edge features, select the top 0.1% edges with the largest absolute value in the edge significance map (246x246x0.5); ​

[0210] Step H4: Use the saliency map results for model interpretation.

[0211] 32 epochs and minibatch size of 8 were set for each training. The average classification accuracy of the model for different sites was 85.00%, 74.29%, 81.16%, and the area under the receiver operating characteristic curve was 0.933, 0.744, 0.867, respectively.

Claims

1. A method for identifying abnormal brain networks of epilepsy based on a multi-scale static-dynamic fusion network, comprising seven modules: (1) a magnetic resonance functional image acquisition module; for acquiring functional fMRI data of patients, establishing an initial functional connection graph taking brain ROI as nodes and functional connection strength between brain regions as edges; fMRI represents magnetic resonance imaging, and ROI represents region of interest; (2) a functional image data preprocessing module; standard neuroimaging processing is performed on the acquired fMRI data, including image format conversion, time point removal, head motion correction, spatial registration, image smoothing, covariate regression and signal filtering operations; and static FC and dFC are calculated to provide input for subsequent graph neural modeling; FC represents functional connection strength, and dFC represents dynamic functional connection matrix; (3) a dynamic-static weighting module; static FC and dynamic dFC are input into the static weighting submodule and the dynamic graph modeling submodule respectively: the static part weights the functional connection matrix through the channel attention mechanism to highlight the discriminative connections; the dynamic part processes the dFC graph sequence through the diffusion space attention module, introduces multi-hop graph convolution and spatial encoding, extracts potential dynamic topological features, and enhances the modeling capability of the model for indirect connections to obtain static features and dynamic features; (4) a cross-attention mechanism module; through the self-attention mechanism, the static features and the dynamic features are taken as inputs to learn the joint representation of the two, and the static-dynamic fusion features are obtained; (5) a multi-scale tokenization module; based on the prior anatomical brain regions and functional network division, the brain regions are regionally divided to generate two types of structural domain Token and functional domain Token, and the information expression in the local subgraph is modeled; (6) a local-global feature fusion module; the structural domain Token and the functional domain Token are extracted through residual convolution and regional attention mechanism respectively, and then the local-global feature interaction fusion is performed through the full connection attention module to form a high-dimensional diagnostic discriminative graph-level representation; (7) an epilepsy brain disease network classification module; the above fused graph-level representation is input into the final classifier module to complete the two-classification of TLE and normal; at the same time, the model embeds two kinds of contrast loss and significance mechanism, has a certain degree of explainability, and can output the key abnormal connection graph and abnormal brain region list for clinical auxiliary analysis; TLE represents the brain network of patients with temporal lobe epilepsy.

2. The method of claim 1, wherein the method comprises: The following calculation steps are adopted in the functional image data preprocessing module: Step 1: Obtain magnetic resonance image data of a plurality of epilepsy patients and normal persons, here using the functional item data of resting state fMRI; perform image format conversion, time point removal, head motion correction, image registration, image smoothing, covariate regression and signal filtering data preprocessing operations on the functional image data, and the specific steps are as follows: Step A1: convert the original DICOM file exported from the magnetic resonance machine into NIFTI file suitable for neuroimaging analysis; Step A2: remove the first n time points of the functional data to make the signal more stable; Step A3: Calculate head motion parameters at different time points to correct the effects introduced by head motion; Step A4: Map functional images to a standard brain spatial template; Step A5: Spatially smooth the registered images to enhance signal-to-noise ratio; Step A6: Remove interference signals from brain signals; Step A7: Filter out low-frequency drift and high-frequency noise in brain signals.

3. The method of claim 1, wherein the method comprises: The specific calculation method of the static graph weighting module is: Step C1: Construct a static weighted graph; each static figure , is composed of a node feature matrix and an adjacency matrix , the adjacency matrix is the static FC calculated in step 2, and the node feature matrix is obtained by mapping the static FC through a linear layer, denotes a real number set of Nxd dimensions, where denotes the number of brain regions, is the dimension of the node feature; Step C2: Weight the adjacency matrix of the static graph; For any two ROIs: and , the node similarity between their node features is calculated , , is the absolute value of Pearson correlation coefficient; and the spatial distance between the two is calculated according to the spatial coordinates of the ROIs , and its spatial similarity is modeled by a Gaussian kernel function: where denotes the standard deviation of the spatial distance of all ROIs in the whole graph; and the adjacency matrix of the weighted static graph is: , ； The weighted static image is then , wherein ; subsequently using unifying a weighted adjacency matrix of the static graph; Step C3: Contrastive pooling; Let the two groups of graphs in the training set be: , , , ； and respectively represent the whole unweighted static functional connectivity graph of HC group and TLE group in training set; , respectively represent the node feature matrix and adjacency matrix of normal brain network, , respectively represent the node feature matrix and adjacency matrix of temporal lobe epilepsy brain network; subsequently, the node features of the two groups of subjects are weighted by attention from ROI scale and individual scale respectively by using double attention mechanism; Computing ROI scales for attention weighting : ； Subsequently attention is weighted from the subject scale : ； wherein the attention mechanism : ； ; is a learnable weight matrix, denotes a normalization operation on a vector; Then, the node features and neighbor matrices of the TLE group and the HC group are averaged within the group to obtain: , ， , ； Finally, the difference between the two sets of graphs is calculated to obtain a contrast graph : = ⊖ ， = ⊖ ； ； where is the computation of the element-wise absolute difference between two matrices; and a l-pooling network is used to aggregate the features of the contrast map: where is the computation of the element-wise absolute difference between two matrices; and a l-pooling network is used to aggregate the features of the contrast map: = ( ))； , Input to the l-th layer embedded The embedded layer feature matrix : = ( ) ； According to the clustering assignment matrix and the embedding layer node feature matrix , a roughened feature matrix and a new adjacency matrix can be generated: ； ； This coarse process can reduce the number of nodes and obtain a higher level of node representation. Through the above calculation process, high-level node representation can be extracted under the guidance of the contrast graph, so as to generate high-quality static graph representation for each input graph: ； Step C4: Static graph feature aggregation; After obtaining the coarse static graph To further explore the high-order topological characteristics and node representation, the step inputs it into the multi-layer graph convolution network GCN for weighted aggregation. In GCN, let the input be a node feature matrix and the adjacency matrix , then the propagation rule at the k-th layer is ； wherein, denotes the adjacency matrix plus self-connections, is its degree matrix, is the trainable weight parameter of the layer, (·) is a non-linear activation function; By The propagation and aggregation operations of the layer GCN can obtain the final static graph feature representation: ； This representation not only retains the difference features guided by the contrast graph, but also further extracts high-order structure information through convolution aggregation, providing stable and reliable static feature support for subsequent dynamic graph branch modeling and overall classification recognition tasks.

4. The method of claim 1, wherein the method comprises: The specific calculation method of the dynamic graph weighting module is: Step D1: Construction of the dynamic functional connectivity graph. As shown in Step 2, each participant can obtain T dynamic graphs using the sliding window method, i.e. With any time window The following image For example, in the current window The following diagram is composed of node features and adjacency matrix The set constitutes the set; the set of node features. The set of adjacency matrices , where node feature vectors Defined as: ； wherein, is a spatial one-hot encoding for the nth node, is a learnable timestamp encoding feature derived from a Gated Recurrent Unit (GRU), is a concatenation operation, is a learnable parameter for the linear mapping of features; while the features of an edge is obtained by computing the Pearson correlation of each pair of ROIs under each time window. Step D2: Design a diffusion space attention module; define the time window under the figure characterization of the dynamic nodes is , represents the time point input features of the layer attention module , while , define , , represents the query vector, represents the searched vector, represents the numerical vector, and the superscript represents the l-th layer of the attention module, and the attention matrix is calculated as follows: ； ； denotes an element-wise multiplication, denotes a convolution operation, denotes activation of an element with a leaky linear rectifier function; Step D3: Perform diffusion method, using a multi-hop diffusion method to spread the attention weighted adjacency matrix Extended to multi-hop attention form The diffusion process is expressed as: ； ； ； wherein represents the transition probability, represents the transition probability, represents the transition probability, represents the transition probability, is a weight factor of the adjacency matrix of different hop counts, and > to attenuate the influence of the adjacency matrix of more hop counts; represents the transition probability, represents the transition probability, represents the transition probability, Step D4: Assign weights to nodes of different path lengths independently in a sequential manner; for , the information propagation step is calculated according to the graph convolution operation : ； wherein , denotes a multi-layer perceptron, is an information propagation and aggregation operation. Then an aggregated output is computed using a batch-normalized multi-layer perceptron, are trainable parameters of the multi-layer perceptron; in order to reduce the complexity of the matrix power computation, a power iteration method is used to approximate : ； ； wherein is a function of the intermediate variable , is a transition probability; when tends to infinity, converges to ; Step D5: Perform time series feature statistics and aggregation processing; For the nodes obtained in the time window The node hidden representation The fluctuation of the node representation at different time points in the time window is obtained by calculating the standard deviation, denoted as: ； wherein, represents calculating the standard deviation in the time dimension, depicting the intensity of the change of the node feature over time; then, the standard deviation features of all time windows are collected stacking and de-meaning in the time dimension; finally, the output characterize the dynamic features as a dynamic graph branch: 。 5. The method of claim 1, wherein the method comprises: The specific calculation method of the static and dynamic functional connection matrix is: Step B1: Extract the average signal in the ROI of the image output by the functional image data preprocessing module, and perform time domain standardization on the signal; Step B2: For the calculation of static FC, the Pearson correlation coefficient between the average BOLD time series between ROIs is calculated, resulting in static , the formula is as follows:​ ； wherein, is the original Pearson correlation coefficient, is the total number of time points, and are the BOLD signal values of the two brain regions at the time point, BOLD stands for blood oxygen level dependent, and are and the mean values; Fisher transformation is applied to the functional connectivity matrix, the calculation formula is as follows: ； wherein is the value after Pearson transformation; Step B3: For the construction of dynamic functional connectivity matrix, the time series of ROIs are divided into windows by sliding window method, for each window, a graph consisting of nodes and edges is constructed ; thus, each subject has graphs consisting of nodes and edges , then for the whole time series, we have , where is a graph sequence containing graphs, the nodes in the graph represent different ROIs, and the edges are represented by the Pearson correlation coefficient between the average BOLD time series of each pair of ROIs.

6. The method of claim 1, wherein the method comprises: The specific calculation method of the cross-attention mechanism module is: Step E1: Generate attention representations of static and dynamic graphs; Characterizing static graphs Characterizing dynamic graphs Layer normalization; then, the two are respectively input into linear transformation layers to obtain corresponding query vectors, key vectors, and value vectors: ； ； ； where, , , , , , are trainable weight parameter matrices. The attention weights for the static and dynamic branches are obtained by computing the element-wise product of the query and key and normalizing: ； Step E2: Cross-fuse static and dynamic features; After obtaining the attention weight matrix, the static features are updated by the attention weight of the dynamic branch, and the dynamic features are updated by the attention weight of the static branch: ； Subsequently, the update results of both are spliced: ; wherein, represents a splicing operation, represents element-wise multiplication; finally, a static-dynamic fusion feature is obtained .

7. The method of claim 1, wherein the method comprises: The specific calculation method of the multi-scale tokenization module is: On the basis of the static and dynamic fusion features On the basis of the static and dynamic fusion features On the basis of the static and dynamic fusion features On the basis of the static and dynamic fusion features On the basis of the static and dynamic fusion features On the basis of the static and dynamic fusion features On the basis of the static and dynamic fusion features < In each partition, the feature vector set of the corresponding ROI is extracted through the index operation to form the token of the domain partition, denoted as: ； ； , ; this processing method not only ensures the integrity of the domain features, but also highlights the network characteristics of the functional network at the macro level, thereby providing a multi-level and more interpretable input representation for subsequent graph sequence modeling and classification recognition.

8. The method of claim 1, wherein the method comprises: The specific calculation method of the local and global feature fusion module is: Step G1: Local feature extraction and region-aware weight calculation. Based on the multi-scale token set obtained in Step six , this step first performs residual convolution operation on the tokens of each region or network to obtain local feature representation. Specifically, let a certain partition token be denoted as: ∈ ； This represents the number of ROIs contained in this partition. Input feature dimension; processed by local convolutional units Perform channel-by-channel and residual mapping to obtain the updated partition features: ； Subsequently, the feature vectors of each partition are globally averaged pooled (GAP) in the ROI dimension to obtain the compact vector representation of the partition: ； The compact representations of all partitions are spliced to obtain the regional feature matrix: ； wherein is the total number of partitions; and then computing a local attention weight matrix by the area-aware controller: ； wherein, is a trainable weight matrix; finally, the local feature matrix is: ； Step G2: Global feature modeling and local-global splicing fusion; In obtaining local features with local weight matrix After that, the local attention weight and all partition tokens are globally weighted and fused by using the multi-head attention mechanism to obtain global features: ； ， ； ； wherein, Finally, the local features are spliced with the global features for splicing: ； Step G3: Dual fusion of structure partition and network partition; The fusion features can be obtained for both structural partition and network partition Finally, the two results are spliced in the feature dimension to obtain unified multi-scale fusion features for subsequent multi-layer perceptron layers for prediction. 。 9. The method of claim 1, wherein the method comprises: The specific calculation method of the epilepsy brain disease network classification module is: Step H1: Adopt ten-fold cross-validation to divide the data set into 10 folds, of which 8 folds are training set, 1 fold is validation set, and 1 fold is training set; Step H2: the multi-scale fused features are input into the reading module to obtain the graph representation of the corresponding time point ; The readout function is: ； wherein, denotes the average value, denotes the maximum value, is the weight of the max-pooling; the graph representation is then fed into a multi-layer perceptron to predict whether a patient is an epilepsy patient or a normal control. Step H3: Input the above classification results into the loss function for model optimization, and the loss function is defined as: ； ； ； ； wherein, is the cross-entropy loss for classification, is the reverse contrastive loss, is the difference graph contrastive loss, , is a hyper-parameter; for , wherein the feature vector and the feature vector are the feature vector of the sample and the feature vector of its corresponding positive sample defined from the samples within the same class; the positive sample pair is a pair of samples in the same class; is the index set of all positive samples of the batch of samples ; is the number of positive sample pairs in a small batch; denotes the cosine similarity and ; the average cosine similarity is calculated and multiplied by -1 to maximize the RCL representation of the intra-class samples; at the same time, in order to avoid the problem of over-smoothing of the graph neural network caused by the too dense adjacency matrix of the contrastive graph , an entropy constraint loss is imposed on it to make it tend to be sparse in training; wherein the row represents the connection probability distribution of the ROI node and other nodes; wherein is used to constrain the clustering allocation to tend to be sparse; Step H4: Gradient-based saliency map method is used to explain the model, and the derivation process of node and edge saliency map is as follows: ； wherein is a node feature saliency map, is an edge feature saliency map, is an input corresponding class prediction score; and the size of the input feature and contribution to the classification.

Images