Brain network multi-level information fusion method and system

Abstract

This invention relates to a method for multi-level information fusion of brain networks, comprising: extracting time series of each Region of Interest (ROI) based on a brain atlas and constructing a multi-granularity brain network; extracting multi-channel features from the constructed multi-granularity brain network to obtain a feature matrix; and fusing multi-channel features based on the obtained feature matrix. This invention also relates to a multi-level information fusion system for brain networks. This invention can fuse local-global multi-level information of the brain, deeply mine the features of multi-granularity brain networks, and improve the expressiveness of brain spatial characteristic modeling.

Description

Technical Field

[0001] This invention relates to a method and system for multi-level information fusion in brain networks. Background Technology

[0002] Major Depressive Disorder (MDD), a mental disorder that severely impacts the mental and physical health of patients, broadly affects their emotional state, cognitive abilities, physical health, and social functioning, and in extreme cases may even lead to suicidal behavior. Therefore, early identification and timely treatment of MDD are of significant clinical value.

[0003] Currently, symptom rating scales (such as the Hamilton Depression Rating Scale) and clinical interviews are commonly used methods for early diagnosis. However, these methods have significant limitations: on the one hand, the accuracy of symptom rating scales is highly dependent on the patient's subjective description; on the other hand, the diagnostic effectiveness of clinical interviews is easily affected by the physician's experience level. These limitations reduce the accuracy of MDD diagnosis, which may delay timely intervention and treatment. Furthermore, the current ability to identify different subtypes of MDD remains insufficient, thus limiting the development of precision medicine strategies.

[0004] In recent years, functional magnetic resonance imaging (fMRI) has become an important tool for the diagnosis of mental illnesses. With its high spatial resolution, fMRI can clearly capture changes in blood oxygenation signals in specific brain regions. Furthermore, the functional connectivity network (FCN) constructed based on fMRI provides objective biomarkers for the diagnosis of mental illness disorder (MDD). By quantifying the strength of functional connections between brain regions, it reveals the spatial interrelationships and interactions between brain regions, providing a basis for identifying abnormal patterns in mental illnesses such as MDD. The FCN not only contains tens of thousands of brain region connections but also multi-level information from the local to the global brain (i.e., different levels of functional connectivity patterns exist from local brain regions to the whole-brain network). Therefore, stably extracting discriminative features from the complex data of the FCN has become an extremely challenging task.

[0005] Existing techniques often directly combine functional connectivity networks (FCNs) and graph neural networks (Graph NNNs), extracting topological properties from FCNs through Graph NNNs. While this combined approach has shown great potential in MDD diagnostic research, its diagnostic accuracy fluctuates significantly (61%-91.8%). This is due to limitations in feature extraction techniques, which fail to fully capture or effectively utilize the local-global information within FCNs, thus affecting the model's diagnostic performance. Furthermore, although deep learning techniques can automatically extract FCN features, current techniques lack systematic integration of multi-level local-global information in brain networks, resulting in limited feature representation. Summary of the Invention

[0006] In view of this, it is necessary to provide a method and system for multi-level information fusion of brain networks.

[0007] This invention provides a method for multi-level information fusion of brain networks, which includes the following steps: S1, extracting time series of each ROI based on a brain map and constructing a multi-granular brain network; S2, extracting multi-channel features from the constructed multi-granular brain network to obtain a feature matrix; S3, performing multi-channel feature fusion based on the obtained feature matrix.

[0008] Preferably, step S1 includes:

[0009] Step S11: Extract the corresponding ROI time series based on the brain atlas and construct a fine-grained brain network;

[0010] Step S12: By performing graph coarsening on the fine-grained brain network, a coarse-grained brain network reflecting the connections between modules is obtained.

[0011] Step S13: Construct the intramodule brain network.

[0012] Preferably, step S11 includes:

[0013] First, the corresponding ROI time series are extracted based on the brain atlas; then, the FCN is obtained by calculating the Pearson correlation between different ROI time series. Fine ∈R N×N Where N represents the number of ROIs or brain regions; FCN Fine Represented as graph G Fine ={V Fine E Fine}, where V Fine ={v1,v2,...,v N} represents a set of N nodes, where each node represents a Region of Interest (ROI), e ij ∈E Fine Represents node v i With node v jFunctional connections between them;

[0014] Subsequently, FCN Fine Applying the K-nearest neighbor algorithm, we retain the k edges with the strongest connectivity (excluding ourselves) for each node, resulting in a weighted adjacency matrix A. Fne =[a ij ]∈R N×N , where a ij Represents node v i With node v j The connectivity strength between nodes; the node characteristics of a fine-grained brain network are defined as FCN. Fine The corresponding row vector in the matrix has its node feature matrix represented as X. Fine ∈R N×N .

[0015] Preferably, step S12 includes:

[0016] Graph coarsening is achieved using a module indicator matrix, resulting in a coarse-grained brain network (FCN). Coarse ∈R M×M Defined as:

[0017] FCN Coarse =P T ·FCN Fine ·P

[0018] Where, P∈R N×M Let P be the module indicator matrix, where M is the number of modules after graph coarsening. The module indicator matrix P is a self-learning parameter matrix, where each element represents the degree to which the corresponding node belongs to a specific module.

[0019] Preferably, step S13 includes:

[0020] Based on the module indicator matrix, the nodes within the module are identified, thereby enabling the analysis from the FCN. Fine Extracting the connection strength between nodes within a module yields the brain network set within that module. Let r represent the brain network within the r-th module, where r = {1, 2, ..., M}; similarly, let FCN... IM Set modeling as graph sets And use the weighted adjacency matrix A IM The corresponding topology is represented by X. IM This represents the node feature matrix.

[0021] Preferably, step S2 includes:

[0022] Each feature extraction channel consists of a combined module, including a TRM module and an NFE module. The TRM module fully extracts the topological information in the graph structure through a GNN model and learns the interaction patterns between brain regions. Then, the NFE module uses a CNN model to perform convolution operations on the node features and further extracts the implicit brain region functional connectivity patterns in the node features through local weighted aggregation. The TRM module and the NFE module work together to comprehensively capture information in the brain network from both the topological structure and node feature dimensions.

[0023] Preferably, step S3 includes:

[0024] A constrained attention pooling mechanism is used, given the feature X of the output of a certain channel. C ∈R N×D Where D represents the feature length, X C First, the importance scores of features are dynamically calculated using a learnable attention weight matrix to achieve adaptive feature weighting. Then, average pooling is applied to embed the weighted nodes into a one-dimensional graph embedding matrix X. G ∈R 1 ×D Specifically, it is expressed as follows:

[0025] X G =Pool(W°X) C )

[0026] Where W∈R N×L This represents the learnable attention weight matrix corresponding to this channel, and X. C With the same dimensions, ° indicates element-wise multiplication, and Pool indicates the average pooling mechanism, which performs mean calculation on the node dimension.

[0027] Preferably, step S3 further includes:

[0028] The obtained graph embedding features are standardized to constrain their data range. After the data range of each channel is constrained, the gradient update direction is prevented from being dominated by large numerical features. The final feature matrix is ​​represented as X'∈R. 1×D as follows:

[0029]

[0030] Where μ represents the mean and σ represents the standard deviation.

[0031] Preferably, step S3 further includes:

[0032] Given multi-channel features processed by the CAP mechanism, X′ Fine 、X′ Coarse and X′ IMBy concatenating the above features, the final feature representation X is obtained. final for:

[0033]

[0034] in, Indicates the concatenation operation, X final The system undergoes linear transformation through two fully connected layers, and is then normalized using the Softmax function to output the classification result.

[0035] This invention also provides a multi-level information fusion system for brain networks, which includes a brain network construction module, a feature extraction module, and a feature fusion module, wherein:

[0036] The brain network construction module is used to extract time series of each ROI based on the brain atlas and to construct a multi-granularity brain network.

[0037] The feature extraction module is used to extract multi-channel features from the constructed multi-granularity brain network to obtain a feature matrix;

[0038] The feature fusion module is used to perform multi-channel feature fusion based on the obtained feature matrix.

[0039] This invention proposes the MGBNF model, which can fully learn and integrate multi-level local-global information from the brain. MGBNF constructs multi-granular networks based on different granularity levels, simultaneously capturing functional connectivity patterns within local brain regions, global modules, and modules. Furthermore, through effective multi-channel feature extraction and fusion strategies, MGBNF can deeply mine the features of multi-granular brain networks, enhancing the expressiveness of brain spatial characteristic modeling. The specific beneficial effects of this invention include:

[0040] (1) The constructed multi-granularity brain network can fully learn and integrate local-global multi-level information of the brain. Specifically, fine-grained brain network, coarse-grained brain network and intra-module brain network are constructed in sequence to effectively capture brain functional connection patterns of different granularities, such as fine-grained node connections, coarse-grained module interactions and intra-module topological features.

[0041] (2) The constructed multi-channel feature extraction framework can deeply mine the information embedded in the node features, and at the same time fully capture the topological information of the graph structure.

[0042] (3) The CAP mechanism adopted can adaptively fuse the multi-channel features output by the multi-channel feature extraction framework through a learnable attention weight matrix, thereby enabling the model to focus on more discriminative features. At the same time, by limiting the data range, the channel features can be kept within the same order of magnitude, thereby improving the convergence of the model. Attached Figure Description

[0043] Figure 1 This is a flowchart of the brain network multi-level information fusion method of the present invention;

[0044] Figure 2 This is a schematic diagram of the overall method provided in an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram of inter-module connections provided in an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram of the internal connections within a module provided in an embodiment of the present invention;

[0047] Figure 5 This is a hardware architecture diagram of the brain network multi-level information fusion system of the present invention. Detailed Implementation

[0048] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0049] See Figure 1 The diagram shown is a flowchart of a preferred embodiment of the brain network multi-level information fusion method of the present invention. Please refer to it as well. Figure 2 :

[0050] Step S1 involves extracting time series data for each Region of Interest (ROI) from the brain atlas and constructing multi-granularity brain networks. Specifically, this involves sequentially constructing fine-grained, coarse-grained, and intra-module brain networks to deeply analyze different functional connectivity patterns between nodes, modules, and within modules. This includes:

[0051] Step S11, constructing a fine-grained brain network:

[0052] First, time series of the corresponding Region of Interest (ROI) are extracted based on the brain atlas; then, the FCN is obtained by calculating the Pearson correlation between different ROI time series. Fine ∈R N×N Where N represents the number of ROIs or brain regions. Given two time series x and y of length T, the formula for calculating their Pearson correlation coefficient is shown in equation (1):

[0053]

[0054] in and It is the average of x and y.

[0055] FCN Fine Represented as graph G Fine ={V Fine E Fine}, where V Fine={v1,v2,...,v N} represents a set of N nodes, where each node represents a Region of Interest (ROI), e ij ∈E Fine Represents node v i With node v j Functional connections between them.

[0056] Subsequently, in order to construct the graph topology, FCN was... Fine Applying the K-Nearest Neighbor (KNN) algorithm, we retain the k edges with the strongest connectivity for each node (excluding itself) to obtain the weighted adjacency matrix A. Fne =[a ij ]∈R N×N , where a ij Represents node v i With node v j The strength of connectivity between them. Furthermore, the node features of fine-grained brain networks are defined as FCN. Fine The corresponding row vector in the matrix has its node feature matrix represented as X. Fine ∈R N×N .

[0057] Step S12, constructing a coarse-grained brain network:

[0058] By coarsening the graph of a fine-grained brain network, a coarse-grained brain network reflecting the connections between modules is obtained. Graph coarsening is achieved using a module indicator matrix, resulting in the coarse-grained brain network FCN. Coarse ∈R M×M Defined as:

[0059] FCN Coarse =P T ·FCN Fine ·P (2)

[0060] Where, P∈R N×M Let P be the module indicator matrix, and M be the number of modules in the coarsened graph. The module indicator matrix P is a self-learning parameter matrix, where each element represents the degree to which the corresponding node belongs to a specific module. Given the set of modules in the coarsened graph as {s1, s2, ..., s...} M}, then node v i For module s p Belonging degree P (i,p) The definition is as follows:

[0061]

[0062] Among them, w (i,p) ∈(0,+∞) is a node v i Belongs to module s pThe corresponding degree of belonging. In order to fully learn the complexity of the brain's functional organization itself, the module indicator matrix allows modules to share nodes when guiding module partitioning.

[0063] Formula (2) essentially uses the module indicator matrix to focus on the connections between cross-module nodes, thereby calculating the connection weight matrix between modules.

[0064] like Figure 3 As shown, assume that the module indicator matrix divides nodes v1, v2, and v3 into module s. p The v4, v5, and v6 nodes are assigned to module s. q Then module s p s q Connection weights W c (p,q) is:

[0065]

[0066] Where W(i,j) represents the connection weight between nodes i and j in the corresponding fine-grained brain network.

[0067] Subsequently, FCN Coarse Model as a graph structure G Coarse In the graph structure, G Coarse Nodes represent modules, and edges represent connections between different modules. Coarse The topological structure is represented by the weighted adjacency matrix A. Coarse ∈R M×M This indicates that the connection weight between modules is FCN. Coarse The corresponding values ​​are then used. Accordingly, the feature matrix X corresponding to the coarse-grained brain network is obtained using the modular indicator matrix. Coarse ∈R M×N for:

[0068] X Coarse =P T X Fine (5)

[0069] Step S13, construct the intramodule brain network:

[0070] like Figure 4 As shown, the focus in constructing the intra-module brain network is on the connections between nodes within the module. Based on the module indicator matrix, the nodes within the module are clearly defined, thus enabling the development of the FCN. Fine Extracting the connection strength between nodes within a module yields the brain network set within that module. Let r represent the brain network within the r-th module, where r = {1, 2, ..., M}. Similarly, let FCN... IM Set modeling as graph sets And use the weighted adjacency matrix AIM The corresponding topology is represented by X. IM This represents the node feature matrix.

[0071] Step S2 involves performing multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix. Specifically:

[0072] like Figure 2 As shown in (a), this embodiment constructs a multi-granularity brain network, including three levels: fine-grained brain network, coarse-grained brain network, and intra-module brain network.

[0073] This embodiment is based on a combined module, designing multiple feature extraction channels with shared parameters, where each channel can extract features from a corresponding level of the brain network. For example... Figure 2 As shown in (b), each feature extraction channel consists of a combined module, including a Topological Relationship Mining (TRM) module and a Node Feature Enhancement (NFE) module. The design principle is based on the fact that brain networks are essentially graph structures. The TRM module fully extracts topological information from the graph structure using a GNN model, learning the interaction patterns between brain regions. Then, the NFE module uses a CNN model to perform convolutional operations on the node features, further extracting the implicit functional connectivity patterns between brain regions from the node features through local weighted aggregation. The two modules work together to comprehensively capture information from the brain network from both the topological structure and node feature dimensions.

[0074] In the TRM module, the GraphSAGE model is primarily used to learn the graph structure. As a graph neural network model, GraphSAGE does not consider all neighboring nodes when performing convolutions on nodes; instead, it samples the neighbors of each node and then aggregates the features of these neighbors using an aggregation function, thus improving learning efficiency. To obtain a more global representation, GraphSAGE typically stacks multiple layers, with each layer further updating node features by aggregating neighbor information. As the number of layers increases, the node features include more layers of neighbor information. However, too many layers may lead to oversmoothing; therefore, in this embodiment, the number of stacked layers is set to 3. Given the node embedding features X corresponding to the brain network, for node v, GraphSAGE first samples a subset of its neighbors from its set N(v), and then performs feature aggregation on the neighbors u of node v. During feature aggregation, an averaging operation is used:

[0075]

[0076] in, Let be the feature representation of neighbor node u in the l-th layer, and N(v) be the set of neighbor nodes of node v. This represents the average of the features of all neighbors of node v. Once the features of neighboring nodes are aggregated, GraphSAGE inputs this aggregated information along with the features of the current node into a neural network, updating the node's features through a linear transformation and a non-linear activation function. The features of node v at layer l+1. Represented as:

[0077]

[0078] in, It is the feature of node v at layer l. W is the average of the features of node v's neighbors, || represents the join operation, and W (l) σ is the weight matrix of the l-th layer, and σ is the activation function.

[0079] In addition, Dropout layers and normalization layers were used in the TRM module to prevent overfitting and constrain the distribution of data, respectively. LeakyReLU was used as the activation function to preserve the activity of negative neurons.

[0080] In the NFE module, a CNN model is used as the main model to extract brain region interactions implied in node features, enhancing the richness of node feature representation. The core of the CNN is the convolutional layer, which uses convolutional kernels to perform convolution operations on the input data and extracts features by calculating the weighted sum of local regions through translation filters. Since the feature vector of each node is one-dimensional data, a one-dimensional convolutional kernel of size (1, 5) is used for feature extraction. The kernel size is determined through grid search optimization. To obtain more global features, two stacked convolutional layers are set in the feature enhancement module. Given a node feature matrix X output by the TRM module... T The node feature matrix output after the first convolution layer... for:

[0081]

[0082] Where Conv2d represents the convolution operation. W C b and b represent the kernel parameters and bias term, respectively.

[0083] Furthermore, since multi-granularity brain networks may contain similar connection patterns, parameter sharing is set up across multiple channels to capture such similar patterns.

[0084] Step S3: Perform multi-channel feature fusion based on the obtained feature matrix. Specifically:

[0085] To better integrate multi-channel features, this embodiment proposes a Constrained Attention Pooling (CAP) mechanism. Given a feature X output from a certain channel... C ∈R N×D Where D represents the feature length. X C First, the importance scores of features are dynamically calculated using a learnable attention weight matrix to achieve adaptive feature weighting. Then, average pooling is applied to embed the weighted nodes into a one-dimensional graph embedding matrix X. G ∈R 1×D Specifically, it is expressed as follows:

[0086] X G =Pool(W°X) C (9)

[0087] Where W∈R N×L This represents the learnable attention weight matrix corresponding to this channel, and X. C Same dimension. ° indicates element-wise multiplication. Pool indicates average pooling, which performs mean calculation on the node dimension.

[0088] Since the data ranges of the output features from different channels may vary significantly, it is necessary to standardize the obtained graph embedding features to constrain their data ranges. Constraining the data range of each channel prevents the gradient update direction from being dominated by large numerical features. The final feature matrix is ​​represented as X'∈R. 1×D as follows:

[0089]

[0090] Where μ represents the mean and σ represents the standard deviation. Given multi-channel features processed by the CAP mechanism, X′ Fine 、X′ Coarse and X′ IM These features are concatenated to obtain the final feature representation X. final for:

[0091]

[0092] in, Indicates a splicing operation. X final The system undergoes linear transformation through two fully connected layers, and is then normalized using the Softmax function to output the classification result (MDD or normal control).

[0093] See Figure 5 The diagram shown is a hardware architecture diagram of the multi-level information fusion system 10 for brain networks of the present invention. Please refer to it as well. Figure 2The system includes: a brain network construction module 101, a feature extraction module 102, and a feature fusion module 103. Among them:

[0094] The brain network construction module 101 is used to extract time series of each ROI based on the brain atlas and construct multi-granularity brain networks. That is, it sequentially constructs fine-grained brain networks, coarse-grained brain networks, and intra-module brain networks, deeply analyzing different functional connection patterns between nodes, between modules, and within modules. Specifically, it includes:

[0095] The brain network construction module 101 constructs a fine-grained brain network:

[0096] First, time series of the corresponding Region of Interest (ROI) are extracted based on the brain atlas; then, the FCN is obtained by calculating the Pearson correlation between different ROI time series. Fine ∈R N×N Where N represents the number of ROIs or brain regions. Given two time series x and y of length T, the formula for calculating their Pearson correlation coefficient is shown in equation (1):

[0097]

[0098] in and It is the average of x and y.

[0099] FCN Fine Represented as graph G Fine ={V Fine E Fine}, where V Fine ={v1,v2,...,v N} represents a set of N nodes, where each node represents a Region of Interest (ROI), e ij ∈E Fine Represents node v i With node v j Functional connections between them.

[0100] Subsequently, in order to construct the graph topology, FCN was... Fine Applying the K-Nearest Neighbor (KNN) algorithm, we retain the k edges with the strongest connectivity for each node (excluding itself) to obtain the weighted adjacency matrix A. Fne =[a ij ]∈R N×N , where a ij Represents node v i With node v j The strength of connectivity between them. Furthermore, the node features of fine-grained brain networks are defined as FCN. FineThe corresponding row vector in the matrix has its node feature matrix represented as X. Fine ∈R N×N .

[0101] The brain network construction module 101 constructs a coarse-grained brain network:

[0102] By coarsening the graph of a fine-grained brain network, a coarse-grained brain network reflecting the connections between modules is obtained. Graph coarsening is achieved using a module indicator matrix, resulting in the coarse-grained brain network FCN. Coarse ∈R M×M Defined as:

[0103] FCN Coarse =P T ·FCN Fine ·P (2)

[0104] Where, P∈R N×M Let P be the module indicator matrix, and M be the number of modules in the coarsened graph. The module indicator matrix P is a self-learning parameter matrix, where each element represents the degree to which the corresponding node belongs to a specific module. Given the set of modules in the coarsened graph as {s1, s2, ..., s...} M}, then node v i For module s p Belonging degree P (i,p) The definition is as follows:

[0105]

[0106] Among them, w (i,p) ∈(0,+∞) is a node v i Belongs to module s p The corresponding degree of belonging. In order to fully learn the complexity of the brain's functional organization itself, the module indicator matrix allows modules to share nodes when guiding module partitioning.

[0107] Formula (2) essentially uses the module indicator matrix to focus on the connections between cross-module nodes, thereby calculating the connection weight matrix between modules.

[0108] like Figure 3 As shown, assume that the module indicator matrix divides nodes v1, v2, and v3 into module s. p The v4, v5, and v6 nodes are assigned to module s. q Then module s p s q Connection weights W c (p,q) is:

[0109]

[0110] Where W(i,j) represents the connection weight between nodes i and j in the corresponding fine-grained brain network.

[0111] Subsequently, FCN Coarse Model as a graph structure G Coarse In the graph structure, G Coarse Nodes represent modules, and edges represent connections between different modules. Coarse The topological structure is represented by the weighted adjacency matrix A. Coarse ∈R M×M This indicates that the connection weight between modules is FCN. Coarse The corresponding values ​​are then used. Accordingly, the feature matrix X corresponding to the coarse-grained brain network is obtained using the modular indicator matrix. Coarse ∈R M×N for:

[0112] X Coarse =P T X Fine (5)

[0113] The brain network construction module 101 constructs the brain network within the module:

[0114] like Figure 4 As shown, the focus in constructing the intra-module brain network is on the connections between nodes within the module. Based on the module indicator matrix, the nodes within the module are clearly defined, thus enabling the development of the FCN. Fine Extracting the connection strength between nodes within a module yields the brain network set within that module. Let r represent the brain network within the r-th module, where r = {1, 2, ..., M}. Similarly, let FCN... IM Set modeling as graph sets And use the weighted adjacency matrix A IM The corresponding topology is represented by X. IM This represents the node feature matrix.

[0115] The analysis module 102 is used to extract multi-channel features from the constructed multi-granularity brain network to obtain a feature matrix. Specifically:

[0116] like Figure 2 As shown in (a), in this embodiment, the analysis module 102 constructs a multi-granularity brain network, including three levels: fine-grained brain network, coarse-grained brain network, and intra-module brain network.

[0117] This embodiment is based on a combined module, designing multiple feature extraction channels with shared parameters, where each channel can extract features from a corresponding level of the brain network. For example... Figure 2As shown in (b), each feature extraction channel consists of a combined module, including a Topological Relationship Mining (TRM) module and a Node Feature Enhancement (NFE) module. The design principle is based on the fact that brain networks are essentially graph structures. The TRM module fully extracts topological information from the graph structure using a GNN model, learning the interaction patterns between brain regions. Then, the NFE module uses a CNN model to perform convolutional operations on the node features, further extracting the implicit functional connectivity patterns between brain regions from the node features through local weighted aggregation. The two modules work together to comprehensively capture information from the brain network from both the topological structure and node feature dimensions.

[0118] In the TRM module, the GraphSAGE model is primarily used to learn the graph structure. As a graph neural network model, GraphSAGE does not consider all neighboring nodes when performing convolutions on nodes; instead, it samples the neighbors of each node and then aggregates the features of these neighbors using an aggregation function, thus improving learning efficiency. To obtain a more global representation, GraphSAGE typically stacks multiple layers, with each layer further updating node features by aggregating neighbor information. As the number of layers increases, the node features include more layers of neighbor information. However, too many layers may lead to oversmoothing; therefore, in this embodiment, the number of stacked layers is set to 3. Given the node embedding features X corresponding to the brain network, for node v, GraphSAGE first samples a subset of its neighbors from its set N(v), and then performs feature aggregation on the neighbors u of node v. During feature aggregation, an averaging operation is used:

[0119]

[0120] in, Let be the feature representation of neighbor node u in the l-th layer, and N(v) be the set of neighbor nodes of node v. This represents the average of the features of all neighbors of node v. Once the features of neighboring nodes are aggregated, GraphSAGE inputs this aggregated information along with the features of the current node into a neural network, updating the node's features through a linear transformation and a non-linear activation function. The features of node v at layer l+1. Represented as:

[0121]

[0122] in, It is the feature of node v at layer l. W is the average of the features of node v's neighbors, || represents the join operation, and W (l)σ is the weight matrix of the l-th layer, and σ is the activation function.

[0123] In addition, Dropout layers and normalization layers were used in the TRM module to prevent overfitting and constrain the distribution of data, respectively. LeakyReLU was used as the activation function to preserve the activity of negative neurons.

[0124] In the NFE module, a CNN model is used as the main model to extract brain region interactions implied in node features, enhancing the richness of node feature representation. The core of the CNN is the convolutional layer, which uses convolutional kernels to perform convolution operations on the input data and extracts features by calculating the weighted sum of local regions through translation filters. Since the feature vector of each node is one-dimensional data, a one-dimensional convolutional kernel of size (1, 5) is used for feature extraction. The kernel size is determined through grid search optimization. To obtain more global features, two stacked convolutional layers are set in the feature enhancement module. Given a node feature matrix X output by the TRM module... T The node feature matrix output after the first convolution layer... for:

[0125]

[0126] Where Conv2d represents the convolution operation. W C b and b represent the kernel parameters and bias term, respectively.

[0127] Furthermore, since multi-granularity brain networks may contain similar connection patterns, parameter sharing is set up across multiple channels to capture such similar patterns.

[0128] The model building module 103 is used to perform multi-channel feature fusion based on the obtained feature matrix.

[0129] Specifically:

[0130] To better integrate multi-channel features, this embodiment proposes a Constrained Attention Pooling (CAP) mechanism. Given a feature X output from a certain channel... C ∈R N×D Where D represents the feature length. X C First, the importance scores of features are dynamically calculated using a learnable attention weight matrix to achieve adaptive feature weighting. Then, average pooling is applied to embed the weighted nodes into a one-dimensional graph embedding matrix X. G ∈R 1×D Specifically, it is expressed as follows:

[0131] X G =Pool(W°X)C (9)

[0132] Where W∈R N×L This represents the learnable attention weight matrix corresponding to this channel, and X. C Same dimension. ° indicates element-wise multiplication. Pool indicates average pooling, which performs mean calculation on the node dimension.

[0133] Since the data ranges of the output features from different channels may vary significantly, it is necessary to standardize the obtained graph embedding features to constrain their data ranges. Constraining the data range of each channel prevents the gradient update direction from being dominated by large numerical features. The final feature matrix is ​​represented as X'∈R. 1×D as follows:

[0134]

[0135] Where μ represents the mean and σ represents the standard deviation. Given multi-channel features processed by the CAP mechanism, X′ Fine 、X′ Coarse and X′ IM These features are concatenated to obtain the final feature representation X. final for:

[0136]

[0137] in, Indicates a splicing operation. X final The system undergoes linear transformation through two fully connected layers, and is then normalized using the Softmax function to output the classification result (MDD or normal control).

[0138] Experimental verification:

[0139] To ensure the stability and robustness of the classification results, this embodiment strictly employs a ten-fold cross-validation framework for model training and validation, and finally calculates the average performance index of ten results. This embodiment uses a grid search method to systematically optimize the key hyperparameters of the model, and the final parameter configuration is shown in Tables 1-4.

[0140] Table 1. Demographic information for the MDD vs. Normal Control (NC) classification task (M represents male, F represents female. Mean ± Standard Deviation)

[0141]

[0142] Table 2. Demographic information for rMDD vs. NC classification tasks

[0143]

[0144] Table 3. Demographic information for FEDN vs. NC classification tasks

[0145]

[0146] Table 4. Demographic information for the rMDD vs. FEDN classification task

[0147]

[0148] Specifically, the model training employed the Adam optimizer with an initial learning rate of 0.001 and a fixed step-size decay strategy, where the step size was set to 30 epochs and the decay parameter gamma was set to 0.8. L2 regularization was also introduced to prevent overfitting. To verify the effectiveness of the MGBNF model, this embodiment compared it with several baseline models focusing on spatial feature extraction. All comparative experiments used a 10-fold cross-validation strategy, ensuring consistent data partitioning across all models through a fixed random seed. The hyperparameters of the baseline models strictly adhered to the configurations in their original papers or official open-source implementations, as detailed in Table 5. (Bold values ​​in the table below represent optimal values, and underlined values ​​represent suboptimal values.)

[0149] Table 5 shows the performance comparison results between the model proposed in this embodiment and the baseline model.

[0150]

[0151]

[0152]

[0153] In this embodiment, single-granularity brain networks, pairwise combinations of single-granularity brain networks, and multi-granularity brain networks were used as model inputs, and their classification performance was compared. As shown in Table 6 below, the multi-granularity brain network analysis method achieved the best classification results. ("Fine-granularity" refers to a fine-granularity brain network; "+" indicates a combination of two brain networks)

[0154] Table 6. Results of ablation studies on multi-granularity brain networks

[0155]

[0156] In this embodiment, when only the TRM module is included, the rMDD classification accuracy is 78.05%. When the NFE module is used in combination with the TRM module, not only is the classification accuracy improved by 1.65%, but it also shows significant advantages in other indicators, as detailed in Table 7.

[0157] Table 7 Ablation Study Results of Combined Modules

[0158]

[0159] In this embodiment, the classification performance decreased to varying degrees after replacing GraphSAGE with GCN, GAT, or GIN, as detailed in Table 8.

[0160] Table 8 Ablation Study Results of GraphSAGE in TRM Module

[0161]

[0162] In this embodiment, the classification performance of the model gradually improves as the number of shared channels increases. When a multi-channel parameter sharing strategy is adopted, the model's classification accuracy reaches its optimal level, which is 2.78% higher than the scheme without parameter sharing, as detailed in Table 9.

[0163] Table 9. Ablation study results using multi-channel parameter sharing strategy

[0164]

[0165] In this embodiment, several fusion methods were designed for comparison: CON (direct concatenation after pooling), ADD (direct addition after pooling), w / o Attention (removing the attention weight matrix and constraining the data range after pooling), and w / o Constrained (retaining the attention weight matrix, concatenating after pooling, and not constraining the data range). All methods use pooling operations to embed high-dimensional nodes into low-dimensional graph-level representations to avoid the curse of dimensionality and computational efficiency problems caused by directly concatenating the original high-dimensional features, as detailed in Table 10.

[0166] Table 10 Ablation study results of the CAP mechanism

[0167]

[0168] Although the present invention has been described with reference to the present preferred embodiments, those skilled in the art should understand that the above preferred embodiments are only used to illustrate the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for brain network multi-level information fusion, characterized in that, The method comprises the following steps: S1, extracting each ROI time sequence according to a brain atlas and constructing a multi-granularity brain network; S2, performing multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix; S3, performing multi-channel feature fusion according to the obtained feature matrix; wherein: The step S1 comprises: Step S11, extracting a corresponding ROI time sequence based on the brain atlas, and constructing a fine-granularity brain network; Step S12, obtaining a coarse-granularity brain network reflecting the connection between modules by graph coarsening processing on the fine-granularity brain network; Step S13, constructing an intra-module brain network; The step S11 comprises: First, the corresponding ROI time series are extracted based on the brain atlas; then the Pearson correlation between different ROI time series is calculated to obtain ; wherein N represents the number of ROIs or brain regions; is expressed as a graph , wherein represents a set of nodes, each node representing an ROI, represents the functional connection between node and node ; Subsequently to The K-Nearest Neighbor algorithm is applied to retain the k edges with the strongest connectivity for each node except itself, and a weighted adjacency matrix is obtained wherein represents the connectivity strength between the node and the node ; the node feature of the fine-grained brain network is defined as the corresponding row vector in the equation, and the node feature matrix is represented as ; The step S12 comprises: The graph coarsening process is implemented by means of a module indicator matrix, and a coarse-grained brain network is obtained is defined as: wherein, is a module indicator matrix, is the number of modules after graph coarsening, the module indicator matrix is an autonomously learnable parameter matrix, each element of which represents the degree of belonging of the corresponding node pair to a particular module; The step S13 comprises: Based on the module indicator matrix, the nodes within the module are identified, thereby... Extracting the connection strength between nodes within a module yields the brain network set within that module. ; Indicates the first The brain network within each module, Similarly, Set modeling as graph sets And use a weighted adjacency matrix To represent the corresponding topology, use This represents the node feature matrix.

2. The method of claim 1, wherein, The step S2 comprises: Each feature extraction channel is composed of a combination module, including a TRM module and an NFE module; the TRM module fully extracts the topological structure information in the graph structure through a GNN model, and learns the interaction mode between brain regions; then, the NFE module uses a CNN model to perform convolution operation on the node features, and further extracts the functional connection mode between brain regions hidden in the node features through local weighted aggregation; the TRM module and the NFE module work cooperatively, and respectively capture the information in the brain network from the two dimensions of topological structure and node features.

3. The method of claim 2, wherein, The step S3 comprises: Adopting the constraint attention pooling mechanism, the feature of a certain channel output is wherein, denotes the feature length, First, the importance score of the feature is dynamically calculated through a learnable attention weight matrix to achieve adaptive feature weighting, and then the weighted node embedding matrix is pooled into a one-dimensional graph embedding matrix by using the average pooling operation , which is specifically expressed as follows: wherein, denotes the learnable attention weight matrix corresponding to the channel, and have the same dimension, denotes element-wise multiplication, denotes the average pooling mechanism that performs mean computation in the node dimension.

4. The method of claim 3, wherein, The step S3 further comprises: The obtained graph embedding features are normalized to constrain the data range, and after the data range of each channel is constrained, the gradient update direction is prevented from being dominated by large value features; and finally, a feature matrix representation is obtained As follows: wherein, represents the average value, represents the standard deviation.

5. The method of claim 4, wherein, The step S3 further comprises: Given the multi-channel features processed by the CAP mechanism are , and respectively; the final feature representation is obtained by concatenating the above features. ， wherein, denotes a concatenation operation, linearly transformed by two fully connected layers in turn, and finally normalized by a Softmax function to output a classification result.

6. A brain network multi-level information fusion system using the brain network multi-level information fusion method of claim 1, characterized in that, The system comprises a brain network construction module, a feature extraction module, and a feature fusion module, wherein: The brain network construction module is used for extracting each ROI time sequence according to a brain atlas and constructing a multi-granularity brain network; The feature extraction module is used for performing multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix; The feature fusion module is used for performing multi-channel feature fusion according to the obtained feature matrix.

Images