Brain network multi-level information fusion method and system

Abstract

The invention relates to a brain network multi-level information fusion method. The method comprises the following steps: extracting each ROI time sequence according to a brain map and performing multi-granularity brain network construction; performing multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix; and performing multi-channel feature fusion according to the obtained feature matrix. The invention further relates to a brain network multi-level information fusion system. According to the method, local-global multi-level information of the brain can be fused, the characteristics of the multi-granularity brain network are deeply mined, and the expressive force of brain spatial characteristic modeling is improved.

Description

Technical Field

[0001] The present invention relates to a brain network multi-level information fusion method and system. Background Art

[0002] Major depressive disorder (MDD) is a psychiatric disorder that severely impacts patients' physical and mental health. It extensively affects their emotional state, cognitive abilities, physical health, and social functioning, and in extreme cases can even lead to suicidal behavior. Therefore, early identification and prompt treatment of MDD are of great clinical value.

[0003] Currently, symptom rating scales (such as the Hamilton Depression Rating Scale) and clinical interviews are commonly used methods for early diagnosis. However, these methods have significant limitations: on the one hand, the accuracy of symptom rating scales is highly dependent on the patient's subjective description; on the other hand, the diagnostic effectiveness of clinical interviews is easily affected by the physician's level of experience. These limitations reduce the accuracy of MDD diagnosis and may delay timely intervention and treatment of the disease. In addition, the ability to distinguish different subtypes of MDD is still insufficient, which limits the development of precise diagnosis and treatment strategies.

[0004] In recent years, functional magnetic resonance imaging (fMRI) has become an important tool for diagnosing psychiatric disorders. With its high spatial resolution, fMRI can clearly capture changes in blood oxygenation signals in specific brain regions. Functional connectivity networks (FCNs) constructed based on fMRI provide objective biomarkers for the diagnosis of mental disorders (MDD). By quantifying the strength of functional correlations between brain regions, they reveal spatial correlations and interactions between brain regions, providing a basis for identifying abnormal patterns in mental disorders such as MDD. Functional connectivity networks contain not only the connection relationships between tens of thousands of brain regions but also multi-level local-global information about the brain (i.e., different functional connectivity patterns at different levels, from local brain regions to global networks). Therefore, stably extracting discriminative features from the complex data of functional connectivity networks has become an extremely challenging task.

[0005] Existing technologies often directly combine functional connectivity networks and graph neural networks, and extract topological properties in functional connectivity networks through graph neural networks. Although the method of combining functional connectivity networks with graph neural networks has shown great potential in MDD diagnosis research, its diagnostic accuracy fluctuates greatly (61%-91.8%). This is due to the limitations of feature extraction technology, which fails to fully capture or effectively utilize local-global information in functional connectivity networks, thereby affecting the diagnostic performance of the model. In addition, although deep learning technology can realize the automatic extraction of FCN features, the existing technology lacks systematic integration of local-global multi-level information of brain networks, resulting in limited feature representation. Summary of the Invention

[0006] In view of this, it is necessary to provide a brain network multi-level information fusion method and system.

[0007] The present invention provides a brain network multi-level information fusion method, which includes the following steps: S1, extracting each ROI time series according to a brain map and constructing a multi-granularity brain network; S2, performing multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix; S3, performing multi-channel feature fusion according to the obtained feature matrix.

[0008] Preferably, the step S1 includes:

[0009] Step S11, extracting the corresponding ROI time series based on the brain atlas and constructing a fine-grained brain network;

[0010] Step S12, performing graph coarsening processing on the fine-grained brain network to obtain a coarse-grained brain network reflecting the connections between modules;

[0011] Step S13: constructing a module-internal brain network.

[0012] Preferably, the step S11 includes:

[0013] First, the corresponding ROI time series are extracted based on the brain atlas; then the FCN is obtained by calculating the Pearson correlation between different ROI time series. Fine ∈R N×N ; Where N represents the number of ROIs or brain regions; FCN Fine Represented as graph G Fine ={V Fine ,E Fine}, where V Fine ={v1,v2,...,v N} represents a set of N nodes, each node represents a ROI, e ij ∈E Fine Represents node v i With node v jFunctional connectivity between

[0014] Then FCN Fine Apply the K nearest neighbor algorithm to retain the k edges with the strongest connectivity for each node except itself, and obtain the weighted adjacency matrix A Fne =[a ij ]∈R N×N , where a ij Represents node v i With node v j The connectivity strength between them; the node characteristics of the fine-grained brain network are defined as FCN Fine The corresponding row vector in the node feature matrix is ​​represented by X Fine ∈R N×N .

[0015] Preferably, the step S12 includes:

[0016] The graph coarsening process is achieved with the help of the module indicator matrix, and the resulting coarse-grained brain network FCN Coarse ∈R M×M Defined as:

[0017] FCN Coarse =P T FCN Fine ·P

[0018] Where P∈R N×M is the module indicator matrix, M is the number of modules after graph coarsening, and the module indicator matrix P is a parameter matrix that can be learned autonomously, each element of which represents the degree of belonging of the corresponding node to a specific module.

[0019] Preferably, the step S13 includes:

[0020] Based on the module indicator matrix, the nodes in the module are identified, so as to Fine Extract the connection strength between nodes in the module and obtain the brain network set within the module represents the rth module inner brain network, r={1,2,...,M}; similarly, FCN IM Sets are modeled as graph sets And use the weighted adjacency matrix A IM Indicates the corresponding topological structure, with X IM Represents the node feature matrix.

[0021] Preferably, the step S2 includes:

[0022] Each feature extraction channel is composed of a combination of modules, including a TRM module and an NFE module. The TRM module fully extracts the topological structure information in the graph structure through the GNN model and learns the interaction pattern between brain regions. Then, the NFE module uses the CNN model to perform convolution operations on the node features and further extracts the functional connection pattern between brain regions implied in the node features through local weighted aggregation. The TRM module and the NFE module work together to comprehensively capture the information in the brain network from the two dimensions of topological structure and node features.

[0023] Preferably, the step S3 includes:

[0024] Using the constrained attention pooling mechanism, given the output feature of a certain channel is X C ∈R N×D , where D represents the characteristic length, X C First, the importance score of the feature is dynamically calculated through the learnable attention weight matrix to achieve adaptive feature weighting. Then, the weighted node embedding matrix is ​​pooled into a one-dimensional graph embedding matrix X using the average pooling operation. G ∈R 1 ×D , specifically expressed as follows:

[0025] X G =Pool(W°X C )

[0026] Where W∈R N×L Represents the learnable attention weight matrix corresponding to the channel, and X C The dimensions are the same, ° represents element-by-element multiplication, and Pool represents the average pooling mechanism, performing mean calculation in the node dimension.

[0027] Preferably, the step S3 further includes:

[0028] The obtained graph embedding features are normalized to constrain their data range. After the data range of each channel is constrained, the gradient update direction is prevented from being dominated by large numerical features. The final feature matrix is ​​represented by X'∈R 1×D as follows:

[0029]

[0030] Here, μ represents the mean and σ represents the standard deviation.

[0031] Preferably, the step S3 further includes:

[0032] Given the multi-channel features processed by the CAP mechanism are X′ Fine , X′ Coarse and X′ IM; Splicing the above features, the final feature representation X final for:

[0033]

[0034] in, Indicates the splicing operation, X final It is linearly transformed through two fully connected layers in sequence, and finally normalized by the Softmax function to output the classification result.

[0035] The present invention also provides a brain network multi-level information fusion system, which includes a brain network construction module, a feature extraction module, and a feature fusion module, wherein:

[0036] The brain network construction module is used to extract the time series of each ROI according to the brain map and perform multi-granularity brain network construction;

[0037] The feature extraction module is used to perform multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix;

[0038] The feature fusion module is used to perform multi-channel feature fusion according to the obtained feature matrix.

[0039] The present invention proposes an MGBNF model that can fully learn and integrate the brain's local-global multi-level information. MGBNF constructs multi-granularity networks based on different granularity levels, and can simultaneously capture the functional connectivity patterns of local brain regions, global modules, and within modules. Furthermore, through effective multi-channel feature extraction and fusion strategies, MGBNF can deeply explore the characteristics of multi-granularity brain networks and enhance the expressiveness of modeling the brain's spatial characteristics. The beneficial effects of the present invention include:

[0040] (1) The constructed multi-granularity brain network can fully learn and integrate the local-global multi-level information of the brain. Specifically, fine-grained brain networks, coarse-grained brain networks, and intra-module brain networks are constructed in sequence to effectively capture brain functional connectivity patterns at different granularities, such as fine-grained node connections, coarse-grained module interactions, and intra-module topological features.

[0041] (2) The constructed multi-channel feature extraction framework can deeply mine the information embedded in node features while comprehensively capturing the topological information of the graph structure.

[0042] (3) The adopted CAP mechanism can adaptively fuse the multi-channel features output by the multi-channel feature extraction framework through a learnable attention weight matrix, so that the model can focus on more discriminative features. At the same time, by limiting the data range, the channel features can be kept within the same order of magnitude, thereby improving the convergence of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of the brain network multi-level information fusion method of the present invention;

[0044] Figure 2 A schematic diagram of the overall method provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of the connection between modules provided in an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of the connections within a module provided in an embodiment of the present invention;

[0047] Figure 5 This is the hardware architecture diagram of the brain network multi-level information fusion system of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] See Figure 1 The figure is a flowchart of a preferred embodiment of the brain network multi-level information fusion method of the present invention. Figure 2 :

[0050] Step S1: Extract the time series of each ROI based on the brain map and construct a multi-granularity brain network. That is, sequentially construct a fine-grained brain network, a coarse-grained brain network, and an intra-module brain network, and deeply analyze the different functional connectivity patterns between nodes, between modules, and within modules. Specifically, it includes:

[0051] Step S11, constructing a fine-grained brain network:

[0052] First, the corresponding ROI (Region of Interest) time series is extracted based on the brain atlas; then the FCN is obtained by calculating the Pearson correlation between different ROI time series. Fine ∈R N×N ; where N represents the number of ROIs or brain regions. Given two time series x and y of length T, the calculation formula for their Pearson correlation coefficient is shown in formula (1):

[0053]

[0054] in and is the average of x and y.

[0055] FCN Fine Represented as graph G Fine ={V Fine ,E Fine}, where V Fine={v1,v2,...,v N} represents a set of N nodes, each node represents a ROI, e ij ∈E Fine Represents node v i With node v j Functional connections between .

[0056] Then, in order to construct the graph topology, FCN Fine Apply the K Nearest Neighbor (KNN) algorithm to retain the k edges with the strongest connectivity for each node (excluding itself) and obtain the weighted adjacency matrix A Fne =[a ij ]∈R N×N , where a ij Represents node v i With node v j In addition, the node features of the fine-grained brain network are defined as FCN Fine The corresponding row vector in the node feature matrix is ​​represented by X Fine ∈R N×N .

[0057] Step S12, constructing a coarse-grained brain network:

[0058] By performing graph coarsening on the fine-grained brain network, a coarse-grained brain network reflecting the connections between modules is obtained. Graph coarsening is achieved with the help of the module indicator matrix, and the resulting coarse-grained brain network FCN Coarse ∈R M×M Defined as:

[0059] FCN Coarse =P T FCN Fine ·P (2)

[0060] Where P∈R N×M is the module indicator matrix, M is the number of modules after the graph is coarsened. The module indicator matrix P is a parameter matrix that can be learned autonomously, and each element of it represents the degree to which the corresponding node belongs to a specific module. Given a graph coarsened module set {s1,s2,...,s M}, then node v i For module s p The degree of belonging P (i,p) The definition is as follows:

[0061]

[0062] Among them, w (i,p) ∈(0,+∞) is the node v i Belongs to module s pIn order to fully learn the complexity of brain functional organization itself, the module indicator matrix allows modules to share nodes when guiding module division.

[0063] Formula (2) essentially uses the module indicator matrix to focus on the connections between nodes across modules, thereby calculating the connection weight matrix between modules.

[0064] like Figure 3 As shown, assume that the module indicator matrix divides v1, v2 and v3 nodes into modules s p , divide v4, v5 and v6 nodes into module s q , then module s p 、s q The connection weight W c (p,q) is:

[0065]

[0066] Among them, W(i,j) represents the connection weight between nodes i and j in the corresponding fine-grained brain network.

[0067] Then, FCN Coarse Modeled as a graph structure G Coarse , in the graph structure G Coarse The nodes of G represent modules, and the edges represent the connections between different modules. Coarse The topological structure of the weighted adjacency matrix A Coarse ∈R M×M Indicates that the connection weight between modules is FCN Coarse Corresponding values ​​in . Accordingly, the characteristic matrix X corresponding to the coarse-grained brain network is obtained with the help of the modular indicator matrix Coarse ∈R M×N for:

[0068] X Coarse =P T X Fine (5)

[0069] Step S13, constructing the brain network within the module:

[0070] like Figure 4 As shown in the figure, in the process of constructing the brain network within the module, we focus on the connection of nodes within the module. Based on the module indicator matrix, we can identify the nodes within the module and thus Fine Extract the connection strength between nodes in the module and obtain the brain network set within the module Represents the rth module inner brain network, r={1,2,...,M}. Similarly, FCN IM Sets are modeled as graph sets And use the weighted adjacency matrix AIM Indicates the corresponding topological structure, with X IM Represents the node feature matrix.

[0071] Step S2: Perform multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix. Specifically:

[0072] like Figure 2 As shown in (a), this embodiment constructs a multi-granularity brain network, including three levels: fine-grained brain network, coarse-grained brain network and intra-module brain network.

[0073] This embodiment designs multiple feature extraction channels with parameter sharing based on the combination module, where each channel can extract features from the corresponding level of brain network. Figure 2 As shown in (b), each feature extraction channel consists of a combination of modules: a Topological Relationship Mining (TRM) module and a Node Feature Enhancement (NFE) module. The design principle is that brain networks are essentially graph structures. The TRM module fully extracts the topological structure information from the graph structure through the GNN model and learns the interaction patterns between brain regions. The NFE module then uses the CNN model to perform convolution operations on the node features and further extracts the functional connectivity patterns between brain regions implied by the node features through local weighted aggregation. These two modules work together to comprehensively capture information in the brain network from two dimensions: topological structure and node features.

[0074] In the TRM module, the GraphSAGE model is mainly used to learn the graph structure. As a graph neural network model, GraphSAGE does not consider all neighbor nodes when performing convolution on a node. Instead, it samples the neighbors of the node and then aggregates the features of these neighbors through an aggregation function, which can improve learning efficiency. In order to obtain a more global representation, GraphSAGE usually stacks multiple layers, and each layer further updates the node features by aggregating neighbor information. As the number of layers increases, the node features include more levels of neighbor information. However, when the number of layers is too many, over-smoothing may occur, so the number of stacked layers is set to 3 in this embodiment. Given that the node embedding feature corresponding to the brain network is X, for node v, GraphSAGE first samples a part of the neighbors from its neighbor node set N(v), and then performs feature aggregation on the neighbor u of node v. When aggregating features, the average operation is used:

[0075]

[0076] in, is the feature representation of the neighbor node u in the lth layer, N(v) is the set of neighbor nodes of node v, Represents the average of all neighbor features of node v. Once the features of neighbor nodes are aggregated, GraphSAGE inputs the aggregated information together with the features of the current node into a neural network and updates the node features through a linear transformation and a nonlinear activation function. The features of node v at layer l+1 are Expressed as:

[0077]

[0078] in, is the feature of node v in layer l, is the average value of the neighbor features of node v, || represents the connection operation, W (l) is the weight matrix of the lth layer, and σ is the activation function.

[0079] In addition, the Dropout layer and the Normalization layer are used in the TRM module to prevent overfitting and constrain the data distribution, respectively. LeakyReLU is used as the activation function to retain the activity of negative neurons.

[0080] In the NFE module, the CNN model is used as the main model to extract the brain region interactions contained in the node features and enhance the richness of the node feature expression. The core of CNN is the convolution layer, which uses the convolution kernel to perform a convolution operation on the input data and extracts features by calculating the weighted sum of the local area through the translation filter. Since the feature vector of each node is one-dimensional data, a one-dimensional convolution kernel of size (1, 5) is used for feature extraction. The convolution kernel size is determined by grid search optimization. In order to obtain more global features, two layers of convolution stacking are set in the feature enhancement module. Given the node feature matrix output by the TRM module is X T , then the node feature matrix output by the first layer of convolution is for:

[0081]

[0082] Among them, Conv2d represents the convolution operation. C and b represent the convolution kernel parameters and bias terms, respectively.

[0083] In addition, since multi-granularity brain networks may contain similar connection patterns, parameter sharing is set up between multiple channels to capture such similar patterns.

[0084] Step S3: Perform multi-channel feature fusion based on the obtained feature matrix. Specifically:

[0085] In order to better fuse multi-channel features, this embodiment proposes a Constrained Attention Pooling (CAP) mechanism. Given a channel output feature X C ∈R N×D , where D represents the characteristic length. X C First, the importance score of the feature is dynamically calculated through the learnable attention weight matrix to achieve adaptive feature weighting. Then, the weighted node embedding matrix is ​​pooled into a one-dimensional graph embedding matrix X using the average pooling operation. G ∈R 1×D , specifically expressed as follows:

[0086] X G =Pool(W°X C )(9)

[0087] Where W∈R N×L Represents the learnable attention weight matrix corresponding to the channel, and X C The dimensions are the same. ° indicates element-by-element multiplication. Pool indicates an average pooling mechanism, which performs mean calculation on the node dimension.

[0088] Since the data ranges of the output features of different channels may vary greatly, it is necessary to normalize the obtained graph embedding features to constrain their data ranges. After the data range of each channel is constrained, the gradient update direction can be prevented from being dominated by large numerical features. The final feature matrix is ​​represented by X'∈R 1×D as follows:

[0089]

[0090] Where μ represents the mean value and σ represents the standard deviation. Given the multi-channel features processed by the CAP mechanism, they are X′ Fine , X′ Coarse and X′ IM These features are concatenated and the final feature representation X final for:

[0091]

[0092] in, Indicates a concatenation operation. X final Linear transformation is performed in sequence through two fully connected layers, and finally normalized by the Softmax function to output the classification result (MDD or normal control).

[0093] See Figure 5 The figure shows the hardware architecture of the brain network multi-level information fusion system 10 of the present invention. Figure 2The system includes: a brain network construction module 101, a feature extraction module 102, and a feature fusion module 103.

[0094] The brain network construction module 101 is used to extract the time series of each ROI based on the brain map and construct a multi-granular brain network. In other words, it sequentially constructs a fine-grained brain network, a coarse-grained brain network, and an intra-module brain network, and deeply analyzes the different functional connection patterns between nodes, between modules, and within modules. Specifically, it includes:

[0095] The brain network construction module 101 constructs a fine-grained brain network:

[0096] First, the corresponding ROI (Region of Interest) time series is extracted based on the brain atlas; then the FCN is obtained by calculating the Pearson correlation between different ROI time series. Fine ∈R N×N ; where N represents the number of ROIs or brain regions. Given two time series x and y of length T, the calculation formula for their Pearson correlation coefficient is shown in formula (1):

[0097]

[0098] in and is the average of x and y.

[0099] FCN Fine Represented as graph G Fine ={V Fine ,E Fine}, where V Fine ={v1,v2,...,v N} represents a set of N nodes, each node represents a ROI, e ij ∈E Fine Represents node v i With node v j Functional connections between .

[0100] Then, in order to construct the graph topology, FCN Fine Apply the K Nearest Neighbor (KNN) algorithm to retain the k edges with the strongest connectivity for each node (excluding itself) and obtain the weighted adjacency matrix A Fne =[a ij ]∈R N×N , where a ij Represents node v i With node v j In addition, the node features of the fine-grained brain network are defined as FCN FineThe corresponding row vector in the node feature matrix is ​​represented by X Fine ∈R N×N .

[0101] The brain network construction module 101 constructs a coarse-grained brain network:

[0102] By performing graph coarsening on the fine-grained brain network, a coarse-grained brain network reflecting the connections between modules is obtained. Graph coarsening is achieved with the help of the module indicator matrix, and the resulting coarse-grained brain network FCN Coarse ∈R M×M Defined as:

[0103] FCN Coarse =P T FCN Fine ·P (2)

[0104] Where P∈R N×M is the module indicator matrix, M is the number of modules after the graph is coarsened. The module indicator matrix P is a parameter matrix that can be learned autonomously, and each element of it represents the degree to which the corresponding node belongs to a specific module. Given a graph coarsened module set {s1,s2,...,s M}, then node v i For module s p The degree of belonging P (i,p) The definition is as follows:

[0105]

[0106] Among them, w (i,p) ∈(0,+∞) is the node v i Belongs to module s p In order to fully learn the complexity of brain functional organization itself, the module indicator matrix allows modules to share nodes when guiding module division.

[0107] Formula (2) essentially uses the module indicator matrix to focus on the connections between nodes across modules, thereby calculating the connection weight matrix between modules.

[0108] like Figure 3 As shown, assume that the module indicator matrix divides v1, v2 and v3 nodes into modules s p , divide v4, v5 and v6 nodes into module s q , then module s p 、s q The connection weight W c (p,q) is:

[0109]

[0110] Among them, W(i,j) represents the connection weight between nodes i and j in the corresponding fine-grained brain network.

[0111] Then, FCN Coarse Modeled as a graph structure G Coarse , in the graph structure G Coarse The nodes of G represent modules, and the edges represent the connections between different modules. Coarse The topological structure of the weighted adjacency matrix A Coarse ∈R M×M Indicates that the connection weight between modules is FCN Coarse Corresponding values ​​in . Accordingly, the characteristic matrix X corresponding to the coarse-grained brain network is obtained with the help of the modular indicator matrix Coarse ∈R M×N for:

[0112] X Coarse =P T X Fine (5)

[0113] The brain network building module 101 builds a brain network within the module:

[0114] like Figure 4 As shown in the figure, in the process of constructing the brain network within the module, we focus on the connection of nodes within the module. Based on the module indicator matrix, we can identify the nodes within the module and thus Fine Extract the connection strength between nodes in the module and obtain the brain network set within the module Represents the rth module inner brain network, r={1,2,...,M}. Similarly, FCN IM Sets are modeled as graph sets And use the weighted adjacency matrix A IM Indicates the corresponding topological structure, with X IM Represents the node feature matrix.

[0115] The analysis module 102 is used to perform multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix. Specifically:

[0116] like Figure 2 As shown in (a), in this embodiment, the analysis module 102 constructs a multi-granularity brain network, including three levels: fine-grained brain network, coarse-grained brain network and intra-module brain network.

[0117] This embodiment designs multiple feature extraction channels with parameter sharing based on the combination module, where each channel can extract features from the corresponding level of brain network. Figure 2As shown in (b), each feature extraction channel consists of a combination of modules: a Topological Relationship Mining (TRM) module and a Node Feature Enhancement (NFE) module. The design principle is that brain networks are essentially graph structures. The TRM module fully extracts the topological structure information from the graph structure through the GNN model and learns the interaction patterns between brain regions. The NFE module then uses the CNN model to perform convolution operations on the node features and further extracts the functional connectivity patterns between brain regions implied by the node features through local weighted aggregation. These two modules work together to comprehensively capture information in the brain network from two dimensions: topological structure and node features.

[0118] In the TRM module, the GraphSAGE model is mainly used to learn the graph structure. As a graph neural network model, GraphSAGE does not consider all neighbor nodes when performing convolution on a node. Instead, it samples the neighbors of the node and then aggregates the features of these neighbors through an aggregation function, which can improve learning efficiency. In order to obtain a more global representation, GraphSAGE usually stacks multiple layers, and each layer further updates the node features by aggregating neighbor information. As the number of layers increases, the node features include more levels of neighbor information. However, when the number of layers is too many, over-smoothing may occur, so the number of stacked layers is set to 3 in this embodiment. Given that the node embedding feature corresponding to the brain network is X, for node v, GraphSAGE first samples a part of the neighbors from its neighbor node set N(v), and then performs feature aggregation on the neighbor u of node v. When aggregating features, the average operation is used:

[0119]

[0120] in, is the feature representation of the neighbor node u in the lth layer, N(v) is the set of neighbor nodes of node v, Represents the average of all neighbor features of node v. Once the features of neighbor nodes are aggregated, GraphSAGE inputs the aggregated information together with the features of the current node into a neural network and updates the node features through a linear transformation and a nonlinear activation function. The features of node v at layer l+1 are Expressed as:

[0121]

[0122] in, is the feature of node v in layer l, is the average value of the neighbor features of node v, || represents the connection operation, W (l)is the weight matrix of the lth layer, and σ is the activation function.

[0123] In addition, the Dropout layer and the Normalization layer are used in the TRM module to prevent overfitting and constrain the data distribution, respectively. LeakyReLU is used as the activation function to retain the activity of negative neurons.

[0124] In the NFE module, the CNN model is used as the main model to extract the brain region interactions contained in the node features and enhance the richness of the node feature expression. The core of CNN is the convolution layer, which uses the convolution kernel to perform a convolution operation on the input data and extracts features by calculating the weighted sum of the local area through the translation filter. Since the feature vector of each node is one-dimensional data, a one-dimensional convolution kernel of size (1, 5) is used for feature extraction. The convolution kernel size is determined by grid search optimization. In order to obtain more global features, two layers of convolution stacking are set in the feature enhancement module. Given the node feature matrix output by the TRM module is X T , then the node feature matrix output by the first layer of convolution is for:

[0125]

[0126] Among them, Conv2d represents the convolution operation. C and b represent the convolution kernel parameters and bias terms, respectively.

[0127] In addition, since multi-granularity brain networks may contain similar connection patterns, parameter sharing is set up between multiple channels to capture such similar patterns.

[0128] The model building module 103 is used to perform multi-channel feature fusion according to the obtained feature matrix.

[0129] Specifically:

[0130] In order to better fuse multi-channel features, this embodiment proposes a Constrained Attention Pooling (CAP) mechanism. Given a channel output feature X C ∈R N×D , where D represents the characteristic length. X C First, the importance score of the feature is dynamically calculated through the learnable attention weight matrix to achieve adaptive feature weighting. Then, the weighted node embedding matrix is ​​pooled into a one-dimensional graph embedding matrix X using the average pooling operation. G ∈R 1×D , specifically expressed as follows:

[0131] X G =Pool(W°XC )(9)

[0132] Where W∈R N×L Represents the learnable attention weight matrix corresponding to the channel, and X C The dimensions are the same. ° indicates element-by-element multiplication. Pool indicates an average pooling mechanism, which performs mean calculation on the node dimension.

[0133] Since the data ranges of the output features of different channels may vary greatly, it is necessary to normalize the obtained graph embedding features to constrain their data ranges. After the data range of each channel is constrained, the gradient update direction can be prevented from being dominated by large numerical features. The final feature matrix is ​​represented by X'∈R 1×D as follows:

[0134]

[0135] Where μ represents the mean value and σ represents the standard deviation. Given the multi-channel features processed by the CAP mechanism, they are X′ Fine , X′ Coarse and X′ IM These features are concatenated and the final feature representation X final for:

[0136]

[0137] in, Indicates a concatenation operation. X final Linear transformation is performed in sequence through two fully connected layers, and finally normalized by the Softmax function to output the classification result (MDD or normal control).

[0138] Experimental verification:

[0139] To ensure the stability and robustness of the classification results, this example strictly uses a ten-fold cross-validation framework for model training and validation, ultimately calculating the average performance metrics of the ten results. This example uses a grid search method to systematically optimize the key hyperparameters of the model. The final parameter configuration is shown in Tables 1-4.

[0140] Table 1 Demographic information of the classification task for MDD vs. normal controls (NC) (M represents male, F represents female. Mean ± SD)

[0141]

[0142] Table 2 Demographic information of rMDD vs. NC classification task

[0143]

[0144] Table 3 Demographic information of FEDN vs. NC classification tasks

[0145]

[0146] Table 4 Demographic information of rMDD vs. FEDN classification tasks

[0147]

[0148] Specifically, the model training uses the Adam optimizer, the initial learning rate is set to 0.001, and a fixed step size decay strategy is adopted, where the step size is set to 30 epochs and the decay parameter gamma is set to 0.8. At the same time, the model introduces L2 regularization technology to prevent overfitting. To verify the effectiveness of the MGBNF model, this embodiment compares it with a variety of baseline models that focus on spatial feature extraction. All comparative experiments use a ten-fold cross-validation strategy, and a fixed random seed is used to ensure the consistency of data partitioning for all models. The hyperparameters of the baseline model are strictly configured with reference to their original papers or official open source implementations, as shown in Table 5. (The bold values ​​in the table below are the optimal values, and the underlined values ​​are the suboptimal values.)

[0149] Table 5 Performance comparison results of the model proposed in this example and the baseline model

[0150]

[0151]

[0152]

[0153] In this example, a single-granularity brain network, a pairwise combination of single-granularity brain networks, and a multi-granularity brain network were used as model inputs to compare their classification performance. As shown in the table below, the multi-granularity brain network analysis method achieved the best classification results, as detailed in Table 6. ("Fine-granularity" indicates a fine-granularity brain network; "+" indicates a combination of two brain networks.)

[0154] Table 6 Ablation study results of multi-granularity brain networks

[0155]

[0156] In this example, when only the TRM module is included, the rMDD classification accuracy is 78.05%. When the NFE module is used in combination with the TRM module, the classification accuracy is improved by 1.65%, and other indicators also show significant advantages, as shown in Table 7.

[0157] Table 7 Ablation study results of combined modules

[0158]

[0159] In this example, after using GCN, GAT, or GIN to replace GraphSAGE, the classification performance showed different degrees of decline, as shown in Table 8 for details.

[0160] Table 8 Ablation study results of GraphSAGE in TRM module

[0161]

[0162] In this example, as the number of shared channels increases, the model's classification performance gradually improves. When a multi-channel parameter sharing strategy is adopted, the model's classification accuracy reaches the best, increasing by 2.78% compared to the solution without parameter sharing. See Table 9 for details.

[0163] Table 9 Ablation study results of multi-channel parameter sharing strategy

[0164]

[0165] In this example, several fusion methods were designed for comparison: CON (direct concatenation after pooling), ADD (direct addition after pooling), w / o Attention (removing the attention weight matrix and constraining the data range after pooling), and w / o Constrained (retaining the attention weight matrix, concatenating after pooling, and not constraining the data range). All methods use the pooling operation to embed high-dimensional nodes and reduce their dimensionality to a low-dimensional graph-level representation, avoiding the dimensionality curse and computational efficiency issues caused by direct concatenation of the original high-dimensional features. See Table 10 for details.

[0166] Table 10 Ablation study results of CAP mechanism

[0167]

[0168] Although the present invention has been described with reference to the current preferred embodiments, those skilled in the art should understand that the above-mentioned preferred embodiments are only used to illustrate the present invention and are not used to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A brain network multi-level information fusion method, characterized in that: The method comprises the following steps: S1, extracting the time series of each ROI based on the brain atlas and constructing a multi-granularity brain network; S2, perform multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix; S3, perform multi-channel feature fusion based on the obtained feature matrix.

2. The method according to claim 1, wherein The step S1 includes: Step S11, extracting the corresponding ROI time series based on the brain atlas and constructing a fine-grained brain network; Step S12, performing graph coarsening processing on the fine-grained brain network to obtain a coarse-grained brain network reflecting the connections between modules; Step S13, constructing the brain network within the module.

3. The method according to claim 2, wherein The step S11 includes: First, the corresponding ROI time series are extracted based on the brain atlas; then the FCN is obtained by calculating the Pearson correlation between different ROI time series. Fine ∈R N×N ; Where N represents the number of ROIs or brain regions; FCN Fine Represented as graph G Fine ={V Fine ,E Fine }, where V Fine ={v1,v2,...,v N } represents a set of N nodes, each node represents a ROI, e ij ∈E Fine Represents node v i With node v j Functional connectivity between Then FCN Fine Apply the K nearest neighbor algorithm to retain the k edges with the strongest connectivity for each node except itself, and obtain the weighted adjacency matrix A Fne =[a ij ]∈R N×N , where a ij Represents node v i With node v j The connectivity strength between them; the node characteristics of the fine-grained brain network are defined as FCN Fine The corresponding row vector in the node feature matrix is ​​represented by X Fine ∈R N×N .

4. The method according to claim 3, wherein The step S12 includes: The graph coarsening process is achieved with the help of the module indicator matrix, and the resulting coarse-grained brain network FCN Coarse ∈R M×M Defined as: FCN Coarse =P T ·FCN Fine ·P Where P∈R N×M is the module indicator matrix, M is the number of modules after graph coarsening, and the module indicator matrix P is a parameter matrix that can be learned autonomously, each element of which represents the degree of belonging of the corresponding node to a specific module.

5. The method according to claim 4, wherein The step S13 includes: Based on the module indicator matrix, the nodes in the module are identified, so as to Fine Extract the connection strength between nodes in the module and obtain the brain network set within the module represents the rth module inner brain network, r={1,2,...,M}; similarly, FCN IM Sets are modeled as graph sets And use the weighted adjacency matrix A IM Indicates the corresponding topological structure, with X IM Represents the node feature matrix.

6. The method according to claim 5, wherein The step S2 includes: Each feature extraction channel is composed of a combination of modules, including a TRM module and an NFE module. The TRM module fully extracts the topological structure information in the graph structure through the GNN model and learns the interaction pattern between brain regions. Then, the NFE module uses the CNN model to perform convolution operations on the node features and further extracts the functional connection pattern between brain regions implied in the node features through local weighted aggregation. The TRM module and the NFE module work together to comprehensively capture the information in the brain network from the two dimensions of topological structure and node features.

7. The method according to claim 6, wherein The step S3 includes: Using the constrained attention pooling mechanism, given the output feature of a certain channel is X C ∈R N×D , where D represents the characteristic length, X C First, the importance score of the feature is dynamically calculated through the learnable attention weight matrix to achieve adaptive feature weighting. Then, the weighted node embedding matrix is ​​pooled into a one-dimensional graph embedding matrix X using the average pooling operation. G ∈R 1×D , specifically expressed as follows: Where W∈R N×L Represents the learnable attention weight matrix corresponding to the channel, and X C The same dimensions, Represents element-by-element multiplication, Pool represents the average pooling mechanism, and performs mean calculation in the node dimension.

8. The method according to claim 7, wherein The step S3 further includes: The obtained graph embedding features are normalized to constrain their data range. After the data range of each channel is constrained, the gradient update direction is prevented from being dominated by large numerical features. The final feature matrix is ​​represented by X'∈R 1×D as follows: Here, μ represents the mean and σ represents the standard deviation.

9. The method according to claim 8, wherein The step S3 further includes: Given the multi-channel features processed by the CAP mechanism are X' Fine 、X' Coarse and X' IM ; Splicing the above features, the final feature representation X final for: in, Indicates the splicing operation, X final It is linearly transformed through two fully connected layers in sequence, and finally normalized by the Softmax function to output the classification result.

10. A brain network multi-level information fusion system, characterized in that: The system includes a brain network construction module, a feature extraction module, and a feature fusion module, among which: The brain network construction module is used to extract the time series of each ROI according to the brain map and perform multi-granularity brain network construction; The feature extraction module is used to perform multi-channel feature extraction on the constructed multi-granularity brain network to obtain a feature matrix; The feature fusion module is used to perform multi-channel feature fusion according to the obtained feature matrix.

Images