A method and system for classifying and predicting brain diseases based on multimodal topological sensing graph networks.

Abstract

This invention discloses a brain disease classification and prediction method and system based on a multimodal topological sensing graph network, relating to the field of medical image processing technology. By fusing temporal signals from functional magnetic resonance imaging (fMRI) and spatial structure signals from diffusion tensor imaging (DTI), a spatial structure prior matrix is ​​introduced as a topology modulator to impose physical connectivity constraints on the constructed multi-view functional interaction network. Furthermore, a spatiotemporally coupled global fusion graph is generated through a cross-modal attention mechanism. Graph neural networks are used to extract local nodes, mesoscale subnetworks, and global topological statistical features to construct a global state feature vector. Finally, a two-stage optimization learning framework is employed, combining multi-objective self-supervised pre-training and supervised fine-tuning to improve the model's classification performance and robustness under small sample conditions. This invention effectively solves the problems of functional connectivity redundancy, lack of structural constraints, and loss of topological information in traditional methods, significantly improving the accuracy and interpretability of brain disease classification.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a brain disease classification and prediction method and system based on a multimodal topological sensing graph network. Background Technology

[0002] Brain diseases such as Alzheimer's disease (AD), schizophrenia (SCZ), and major depressive disorder (MDD) pose a serious threat to human health and quality of life. Early and accurate diagnosis of these diseases is crucial for timely intervention and treatment. Functional magnetic resonance imaging (fMRI) is a non-invasive neuroimaging technique that captures blood oxygen level-dependent (BOLD) signals, reflecting spontaneous neural activity in the brain. Brain networks constructed based on fMRI data can describe the functional interactions between different brain regions and have become an important tool for studying brain diseases. In recent years, graph neural networks (GNNs), with their powerful relational data modeling capabilities, have been widely applied to brain network analysis.

[0003] Most existing graph neural network-based brain disease diagnostic methods rely on single-view functional connectivity (FC) indicators such as Pearson correlation coefficient. However, these single-view FC indicators have inherent limitations: Pearson correlation can only capture linear, static functional interactions and cannot reflect direct causal relationships between brain regions, nor can it reflect the dynamic changes in functional connectivity over time. They are prone to extracting statistically correlated but physically or anatomically unconnected redundant edges, and lack the constraints of underlying physical structure priors. In addition, the brain is a complex system with unique topological characteristics. Topological characteristics such as clustering coefficient, average path length, and modularity are closely related to brain function and disease pathogenesis. Ignoring these topological characteristics can lead to the loss of key diagnostic information. Summary of the Invention

[0004] The purpose of this invention is to provide a brain disease classification and prediction method and system based on multimodal topological sensing graph networks, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a brain disease classification and prediction method based on a multimodal topological sensing graph network, comprising the following steps:

[0006] Step 1: Multimodal data acquisition, acquiring the first modal timing signal and the second modal spatial structure signal of the target object.

[0007] Step 2: Construction of multi-view functional interaction network. Based on the first modal temporal signal, multi-dimensional interaction features are extracted to construct a multi-view functional interaction network that represents the dynamic and static interaction relationship of the target object. The multi-view includes an overall linear correlation view, a conditionally independent correlation view that eliminates indirect transmission effects, and a dynamic correlation view that extracts local temporal evolution features.

[0008] Step 3: Spatial structure prior extraction and cross-modal interactive modulation. Based on the second modal spatial structure signal, physical connectivity is extracted to construct a spatial structure prior matrix representing the connection relationship of the target object entities. The spatial structure prior matrix is ​​used as a topology modulator to constrain the edge weights of the multi-view functional interaction network with spatial connectivity. Entity connection probability is used to penalize and suppress redundant interactive edges lacking physical support. The constrained network is then weighted and integrated through a cross-modal attention mechanism to generate a spatiotemporally coupled global fusion graph.

[0009] Step 4: Hierarchical topological feature interaction and global representation construction. A graph neural network model is used to transmit information on the global fusion graph, extracting local node representations. A learnable assignment matrix is ​​used to map and aggregate these local node representations into mesoscale sub-network representations. Simultaneously, graph theory invariants of the global fusion graph, such as clustering coefficients, feature path lengths, and modularity, are calculated to generate a global topological statistical representation. Subsequently, the local node representations, mesoscale sub-network representations, and global topological statistical representations are concatenated and interacted along the feature dimension to construct the global state feature vector of the target object.

[0010] Step 5: Two-stage optimization and classification output. The global state feature vector is input into the two-stage optimization learning framework. In the pre-training stage, the network feature extraction parameters are updated under unsupervised conditions using a multi-objective joint loss function that includes topological consistency loss, sparsity and community constraint loss, cross-view reconstruction loss, and cross-modal alignment loss. In the fine-tuning stage, the pre-feature extraction network parameters are frozen, and the classification error gradient is separately backpropagated to the downstream classifier and node allocation matrix to output the final target object state classification prediction result.

[0011] As a priority, extracting multi-dimensional interaction features based on the first modal time-series signal includes:

[0012] Extracting the overall linear correlation view: Calculating the pairwise linear dependencies of time-series signals over the entire time span;

[0013] Extracting conditionally independent related views: Introducing regularization constraints to eliminate the indirect transmission effect in the linear dependency relationship and obtain the direct interaction relationship;

[0014] Extracting dynamic evolution relationship views: Local interaction relationships are extracted by time-series segmentation and aggregated along the time dimension to supplement the missing time-series evolution information in the overall linear relationship view and conditionally independent relationship view.

[0015] As a preferred method, the spatial structure prior matrix is ​​used as a topology modulator to constrain the edge weights of a multi-view functional interaction network, comprising the following steps:

[0016] The spatial structure prior matrix is ​​normalized into connectivity probability weights;

[0017] The connectivity probability weights are multiplied by the adjacency matrix of the multi-view functional interaction network using a Hadamard product. The entity probability of the spatial structure is used to penalize and suppress redundant edges in the functional interaction network that lack physical connection support, thereby outputting a candidate network graph modulated by the spatial structure.

[0018] As a preferred method, the method for generating a spatiotemporally coupled global fusion graph through the cross-modal attention mechanism includes the following steps:

[0019] The multi-view functional interaction network and the candidate network map modulated by spatial structure are respectively mapped to the shared latent space to generate corresponding view feature vectors;

[0020] The relative importance of each view feature vector in the classification task is evaluated through an attention mechanism, and the corresponding weights are calculated.

[0021] By using the corresponding weights to adaptively linearly combine the adjacency matrices of each view, the final generated global fusion map can dynamically balance the proportion of temporal features and prior information of spatial structure.

[0022] As a preferred method, the local node representation method includes the following steps:

[0023] Based on a preset edge filtering threshold, the global fusion graph is sparsified to establish information propagation paths between nodes;

[0024] A local feature aggregator based on an attention mechanism is used to aggregate the neighborhood information of the target node along the information propagation path according to the dynamic attention coefficients between adjacent nodes, thereby generating a node representation that includes local topology awareness.

[0025] As a preferred method, the method for extracting the representation of the mesoscale sub-network includes the following steps:

[0026] Initialize a differentiable node assignment matrix to learn the mapping probability of each node in the network graph belonging to a specific mesoscale functional module;

[0027] The local node representation and the node allocation matrix are interacted by matrix multiplication. Based on the mapping probability, the discrete node features are weighted and fused to generate a mesoscale subnetwork representation of the local dense interactive community of the representation system.

[0028] As a preferred method, the method for generating the global topological statistical representation includes the following steps:

[0029] For the global fusion graph, its clustering coefficient, feature path length, and modularity are explicitly calculated as graph topological statistical invariants;

[0030] The graph topological statistical invariants are encoded in high dimension using a nonlinear mapping network and used as global prior information. This information is then concatenated and interacted with the mesoscale sub-network representation in terms of feature dimensions, so that the final global state feature vector has micro, meso and macro structural attributes.

[0031] As a priority, in the pre-training phase of the two-stage optimized learning framework, the inverse constraint of the loss function on the feature extraction network includes:

[0032] Topology consistency loss interaction: By perturbing the network input, the similarity between the global state feature vectors before and after the perturbation in the latent space is maximized, thereby driving the network to learn noise-resistant features;

[0033] Interaction between sparsity and community constraint loss: Combining the L1 norm of the network tensor and the graph modularity index, the generation process of the global fusion graph is constrained in reverse, forcing it to conform to the sparsity of real physical or biological connections.

[0034] Cross-view reconstruction loss interaction: The driver decoder uses static view features to inversely reconstruct dynamic view features, thereby enhancing the feature extraction network's ability to capture the inherent temporal dependencies between different views.

[0035] As a priority, the two-stage optimization learning framework also includes:

[0036] Under the constraint of freezing the parameters of the pre-feature extraction network, the global state feature vector is input into the downstream classifier;

[0037] The cross-entropy loss of the classification error is calculated, and the error gradient is fed back to the classifier parameters and the node allocation matrix separately. While maintaining the stability of the global feature extraction capability, the mapping relationship of the driving module is adaptively adjusted to the specific classification task.

[0038] Furthermore, this invention also provides a brain disease classification and prediction system based on a multimodal topological sensing graph network, comprising:

[0039] The multimodal signal data processing module is used to acquire the first modal temporal signal and the second modal spatial structure signal of the target object, extract multi-dimensional interaction features based on the first modal temporal signal, construct a multi-view functional interaction network representing the dynamic and static interaction relationship of the target object, and extract physical connectivity based on the second modal spatial structure signal to construct a spatial structure prior matrix representing the entity connection relationship of the target object.

[0040] The spatial constraint module is used to use the spatial structure prior matrix as a topology modulator to constrain the spatial connectivity of the edge weights of the multi-view functional interaction network, and to perform weighted integration of the constrained network through a cross-modal attention mechanism to generate a spatiotemporally coupled global fusion graph.

[0041] The hierarchical topological feature interaction and global representation construction module is used to transmit information on the global fusion graph using a graph neural network model, extract local node representations, map and aggregate the local node representations into mesoscale sub-network representations using a learnable assignment matrix, and simultaneously calculate the graph theory invariants of the global fusion graph to generate a global topological statistical representation.

[0042] The two-stage optimization and classification output module is used to construct a global state feature vector of the target object through the local node representation, the mesoscale sub-network representation and the global topological statistical representation. The global state feature vector is input into the two-stage optimization learning framework, and the network feature extraction parameters are updated using a loss function with multi-objective constraints. The classification prediction result of the target object is then output.

[0043] In summary, the beneficial effects of this invention are:

[0044] 1. Structure-Function Joint Driving Significantly Reduces False Positive Connections in Networks: This invention overcomes the limitations of single-modality temporal modeling and creatively uses spatial structure signals as a "topology modulator." By introducing hard constraints and soft penalties based on the Hadamard product of physical connectivity probabilities before feature fusion, it effectively filters out pseudo-interaction edges in a purely statistical sense, resulting in a globally fused graph with extremely high physical or anatomical rationality.

[0045] 2. Deep interaction between multiple views and multi-level features enhances representation completeness: This invention not only constructs a multi-view network covering linear, direct causal, and dynamic evolution, but also achieves hierarchical feature interaction from micro (local nodes), meso (functional communities / sub-networks) to macro (global statistics) through learnable assignment matrices and explicit graph theory index calculations. This full-scale feature stitching ensures that the final state feature vector possesses both local details and a global perspective.

[0046] 3. Introducing a multi-objective cross-modal self-supervised mechanism to effectively overcome the bottleneck of data scarcity: In response to the pain point of difficulty in obtaining labeled data, this invention designs a multi-objective self-supervised pre-training framework, and in particular, innovatively introduces "cross-modal alignment contrast loss" and "cross-view reconstruction loss", which force the model to learn the inherent shared consistency between temporal dynamic features and spatial entity structures in the latent space, greatly improving the model's generalization ability and noise robustness under small sample conditions.

[0047] 4. Excellent interpretability and dynamic adaptability: The cross-modal attention mechanism in the model can dynamically evaluate the contribution weight of temporal features and spatial priors in different classification tasks; at the same time, in the fine-tuning stage, the node allocation matrix is ​​updated separately by classifying error, so that the module division in the graph can adapt and evolve for specific downstream tasks, such as specific disease types, providing a reliable source interpretation basis for the classification results. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of the overall architecture of a brain disease classification and prediction method based on a multimodal topological sensing graph network according to the present invention.

[0050] Figure 2 This is a schematic diagram illustrating the construction of a multi-view functional interactive network for a brain disease classification and prediction method based on a multimodal topological sensing graph network, according to the present invention.

[0051] Figure 3 This is a schematic diagram illustrating the spatial structural constraints and cross-modal fusion mechanism of a brain disease classification and prediction method based on a multimodal topological sensing graph network according to the present invention.

[0052] Figure 4 This is a schematic diagram illustrating the hierarchical topological feature extraction and splicing process of a brain disease classification and prediction method based on a multimodal topological sensing graph network according to the present invention.

[0053] Figure 5 This is a schematic diagram of the two-stage optimization learning framework for a brain disease classification and prediction method based on a multimodal topological sensing graph network, as described in this invention.

[0054] Figure 6 This is a conceptual diagram illustrating the effect of structural prior on the topological modulation of functional networks in a brain disease classification and prediction method based on multimodal topological sensing graph networks, as described in this invention. Detailed Implementation

[0055] The present invention will now be described in further detail with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. These drawings are simplified schematic diagrams, which are only used to illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0056] To facilitate understanding of the present invention, a more complete description of the invention will be given below with reference to the accompanying drawings, which illustrate several embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the invention will be more thorough and complete.

[0057] All features disclosed in this specification, or steps in all methods or processes disclosed herein, may be combined in any way, except for mutually exclusive features and / or steps.

[0058] Any feature disclosed in this specification (including any appended claims, abstract, and drawings) may be replaced by other equivalent or similar features for a similar purpose, unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is merely one example of a series of equivalent or similar features.

[0059] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; they can refer to the internal communication of at least two elements or the interaction relationship of at least two elements, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0060] Please see Figures 1-6 The present invention provides an embodiment of a brain disease classification and prediction method based on a multimodal topological sensing graph network. Taking the early diagnosis of brain diseases such as Alzheimer's disease and schizophrenia as an example, the specific execution steps of the method of the present invention are described in detail. Let N be the number of regions of interest (ROIs), that is, the number of nodes, and T be the number of time points.

[0061] Step 1: Multimodal Data Acquisition and Preprocessing

[0062] The first modal temporal signal of the target object is acquired. In this embodiment, the first modal temporal signal is specifically the functional magnetic resonance imaging (fMRI) data of the subject, denoted as . Simultaneously, the second modal spatial structure signal of the same subject is acquired. In this embodiment, the second modal spatial structure signal is specifically the diffusion tensor imaging of the subject, i.e., DTI data.

[0063] Fiber tract tracking was performed on DTI data to determine the number or density of white matter fiber tracts between different regions of the brain (ROI), and a tractor of size was constructed. Spatial structure prior matrix Its elements represent the probability of physical connections between brain regions or the density of fiber bundles. As a "map" of real physical connections, it guides the learning of subsequent functional networks.

[0064] Step 2: Building a Multi-View Functional Interaction Network

[0065] refer to Figure 2 To capture complementary information about brain function, this invention constructs a multi-view functional interaction network based on first-modal temporal signals, specifically including the following three views:

[0066] In this embodiment, the overall linear correlation view is specifically represented as a static linear view based on the Pearson correlation coefficient. Centering the fMRI signal eliminates mean bias.

[0067]

[0068] in, , This represents the tensor outer product operation. It is a length of A row vector of all 1s, covariance matrix The calculation formula is:

[0069]

[0070] Calculate the covariance matrix Then, normalization is performed to obtain the Pearson correlation matrix. :

[0071]

[0072] in, It is a very small positive number. This represents the diagonal operator.

[0073] In this embodiment, the conditionally independent associated view is specifically manifested as a static direct view that introduces a precision matrix. To eliminate indirect transmission effects, such as the transmission path A→C→B, regularization is used to ensure matrix invertibility.

[0074]

[0075]

[0076] in, Given an N×N identity matrix, the partial correlation matrix A2 is defined as follows, and the partial correlation matrix is ​​then calculated. :

[0077]

[0078] In this embodiment, the dynamically evolving related view is specifically manifested as a dynamic functional view based on a sliding window. : Use window size A sliding window with a step size of S captures the temporal evolution of brain connectivity features. The formula for calculating the total number of windows is:

[0079]

[0080] Calculate the local Pearson correlation matrix for each window. The local correlation matrices of all windows are aggregated to obtain a dynamic representation, and all inputs are stacked to finally obtain a dynamic connectivity matrix. :

[0081]

[0082] Step 3: Prior extraction of spatial structure and cross-modal interactive modulation

[0083] refer to Figure 3 and Figure 6 The normalized spatial structure prior matrix As a topology modulator, spatial connectivity constraints are applied to the multi-view functional interaction network via the Hadamard product:

[0084]

[0085] in It is a functional connectivity matrix, k∈1,2,3. It is a small constant that preserves a small number of edges that have no direct physical connection but have strong indirect functional cooperation;

[0086] Subsequently, a cross-modal attention mechanism is used for feature integration: a two-dimensional convolutional network (Conv2d) and a multilayer perceptron (MLP) are used to encode each view into a feature vector. ,

[0087] Attention weights are calculated using a learnable projection and a softmax function. This weight is used to adaptively linearly combine the candidate network graphs to generate a spatiotemporally coupled global fusion graph, i.e., a fusion adjacency matrix. .

[0088] When calculating attention weights, the model not only evaluates the importance of the three functional views, but also learns the complementary weights between "structure" and "function." This allows the model to adaptively prioritize the more diagnostically valuable modalities when disease causes brain dysfunction but no obvious structural lesions (or vice versa).

[0089] Step 4: Hierarchical topological feature interaction and global representation construction

[0090] refer to Figure 4 Using a graph neural network model to fuse the global graph Extracting hierarchical features:

[0091] Local node representation: extracted by the graph attention network GAT, with a set threshold. Constructing sparse graphs A graph attention network with a multi-head attention mechanism is used to aggregate local information based on the attention coefficients between adjacent nodes, and the node features are output after passing through a LayerNorm layer. .

[0092] Aggregate local information:

[0093] ;

[0094] ;

[0095] .

[0096] Mesoscale subnetwork representation, i.e., module-level features: introducing a learnable node assignment matrix. ,in, Representing local nodes Interacting with the allocation matrix for matrix multiplication:

[0097] ;

[0098] The features are aggregated into module features and then reduced in dimensionality by linear layer projection. Finally, the average of all module features is taken to obtain the sub-network representation. :

[0099] :

[0100] ;

[0101] .

[0102] Global topological statistical representation: explicit computation of clustering coefficients of fused graphs Feature path length and modularity Encode it as a topological vector. And generate topological representations through MLP mapping. Finally, the above features are concatenated and interacted to construct a global state feature vector: .

[0103] Step 5: Two-stage optimization and classification output

[0104] refer to Figure 5 The global state feature vector is input into a two-stage optimization learning framework, which includes the following stages:

[0105] Unsupervised pre-training stage: The feature extraction network is constrained using a multi-objective joint loss function;

[0106] Topology consistency loss:

[0107] By adding adjacency tensors to the multiview Add Gaussian noise Generate positive sample pairs:

[0108] ;

[0109] Employing an InfoNCE-based loss function to maximize the original global features Global features after perturbation Similarity between them:

[0110] ;

[0111] ;

[0112] in, This refers to the temperature parameter. and All have undergone L2 normalization. Labels indicating positive samples.

[0113] Sparsity and community constraint loss:

[0114] To make the merged adjacency matrix In accordance with biological rationale, we introduce sparsity and modular structure constraints:

[0115] ;

[0116] ;

[0117] The formula for calculating the sparse module loss is:

[0118] ;

[0119] in, The module adjacency matrix is ​​the average value along the batch dimension. The module degree is constrained by the Louvainal algorithm to ensure the biological rationality of the graph.

[0120] Cross-view reconstruction loss:

[0121] The model is required to be derived from a static view ( Reconstructing the dynamic view in ) This strengthens the dependencies between views. The reconstruction process is implemented through a convolutional neural network decoder (CNNDecoder):

[0122] ;

[0123] Reconstruction loss is calculated using mean squared error (MSE):

[0124] ;

[0125] The total loss during pre-training is the sum of the contrastive loss, the sparse module loss, and the reconstruction loss.

[0126] Cross-modal alignment loss: Maximizes the cosine similarity between the functional coding features extracted from fMRI and the structural coding features extracted from DTI for the same object in the latent space;

[0127] Supervised fine-tuning stage: Freeze the parameters of the graph neural network backbone, and feed the classification error gradient back to the downstream classifier separately, classifying the head parameters. and node allocation matrix, mask parameters The classification loss uses cross-entropy loss (CE):

[0128] ;

[0129] And introduce L1 regularization term Maintaining the sparsity of the graph, the final output is the diagnostic probability of brain diseases.

[0130] To further demonstrate the effectiveness and operability of a brain disease classification and prediction method based on a multimodal topological sensing graph network, a detailed example with specific simulation data is provided below. This example uses an early diagnosis experiment of Alzheimer's disease (AD) as a background to demonstrate in detail the flow and calculation process of data in each algorithm module.

[0131] We extracted multimodal imaging data from an early-stage Alzheimer's disease patient from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset.

[0132] First modality (fMRI data): Assuming the brain is divided into 90 regions of interest (ROIs) using automated anatomical calibration (AAL) atlas (N=90 nodes), and the resulting time series contains 200 time points (T=200 time points), the input fMRI time series data is represented as a matrix. .

[0133] Second modality (DTI data): After processing with a fiber tract tracking algorithm, the white matter fiber tract connectivity probabilities between 90 brain regions were extracted to construct... Spatial structure prior matrix For example, assuming a strong physical connection exists between brain region 1 (such as the left hippocampus) and brain region 2 (such as the left amygdala), its structural weights, after normalization, are: If there is almost no physical connection between brain region 1 and brain region 10, then .

[0134] Step 1: Building a Multi-View Functional Interaction Network

[0135] The model is first based on fMRI data Construct three complementary view matrices to capture the brain's complex functional interactions:

[0136] Static linear connection view The signal is centered to eliminate mean bias, and the covariance matrix is ​​calculated and normalized to obtain the Pearson correlation matrix. Assuming the calculated correlation coefficient between brain region 1 and brain region 2 is... To ensure numerical stability, a very small positive constant is introduced into the formula. For example, set as .

[0137] Static direct-connected view To eliminate indirect transmission effects, such as spurious correlations generated through brain region 3, a precision matrix is ​​introduced. The partial correlation matrix is ​​then calculated by regularizing and inverting the covariance matrix. After eliminating indirect effects, the direct connection weights between brain region 1 and brain region 2 may decrease, assuming the calculated result is... .

[0138] Dynamic Function Connection View : Capture the temporal evolution of brain connectivity features, and set the sliding window size. That is, it contains 50 time points, with a step size of 50. According to the formula The calculation generates a total of K=16 sliding windows, and the local Pearson correlation matrix is ​​calculated within each window. Then, the average of these 16 matrices is calculated and aggregated to obtain... Assuming the dynamic weights of brain region 1 and brain region 2 after aggregation are... .

[0139] The above three The matrices are concatenated along the channel dimension to form a multi-view adjacency tensor. .

[0140] Step 2: Spatial Structure Prior Extraction and Cross-Modal Interactive Modulation

[0141] To suppress false positive connections in functional networks, DTI-generated connections are introduced. Used as a topology modulator for spatial connectivity constraints.

[0142] Combining the function matrix with structural weights using the Hadamard product:

[0143] ;

[0144] Substitute the data from brain region 1 and brain region 2: .

[0145] Assuming functional correlation between brain regions 1 and 10. However, it lacks physical structural support. Then the constrained weights become This step effectively penalizes redundant edges with no structured foundation.

[0146] The module then proceeds to the multi-view attention fusion module, which uses a 2D convolutional network (Conv2d) and a multilayer perceptron (MLP) to encode each view into a feature vector. Attention weights are calculated using the Softmax function. .

[0147] Assume the model adaptively assigns weights in the current training iteration as follows: Linear view Direct view Dynamic View The model then performs a weighted sum of the three constrained graphs, outputting the final fused adjacency matrix. .

[0148] Step 3: Hierarchical Topological Feature Interaction and Global Representation Construction

[0149] In the fusion of adjacency matrix Based on this, the topology-aware graph neural network module begins to extract hierarchical features:

[0150] Node-level feature extraction: setting an absolute value threshold Constructing sparse graphs Weak connections with weights below 0.2 are filtered out. The Graph Attention Network (GAT) aggregates local node information and outputs a dimensionality-reduced node feature matrix. Assume the feature dimension is 64.

[0151] Module-level feature extraction: This aims to capture mesoscale functional modules. Assuming the brain network in its current state is divided into 5 functional subnetworks (M=5), Softmax is used to generate a feature of size [missing value]. The node allocation matrix is ​​used to aggregate the features of 90 nodes into the features of 5 modules through matrix operations. Then, the features are reduced to 2D dimension through a linear layer and averaged to output the module-level feature vector. .

[0152] Topological statistical feature calculation: This involves explicitly calculating core graph theory indices for the graph. Assuming the graph is sparse for this AD patient, the following calculations are performed:

[0153] Clustering coefficient The level was lower than that of the healthy control group, indicating a decrease in the efficiency of local network information transmission.

[0154] Feature path length The levels were higher than those in the healthy control group, indicating impaired global integration ability;

[0155] Modularity These three scalars are concatenated to form a topological vector. Encoded by MLP .

[0156] Ultimately, module-level features will be... Topological statistical characteristics The features are concatenated to form a global representation vector that encompasses multi-scale information. .

[0157] Step 4: Two-stage optimization and final state classification

[0158] During the pre-training phase, the model is trained by feeding the multi-view adjacency tensor. Add variance Gaussian noise is used to generate positive sample pairs, and the topological consistency contrast loss is calculated. Meanwhile, the average value of the calculation module matrix along the batch dimension is also calculated. The module degree is calculated using the Lewin algorithm, and then the sparse module loss is obtained. Furthermore, a convolutional neural network decoder is used to attempt to reconstruct a dynamic view from a static view. This results in mean square error loss during cross-view reconstruction. Under the joint constraints of this triple self-supervised pre-training loss and cross-modal alignment loss, the model can learn highly robust network feature representations without disease labels.

[0159] Once the fine-tuning phase begins, the weights of the backbone network are frozen, and adjustments are made only to the classification head parameters. and mask parameters To optimize, let's assume that the extracted global representation vector is... The input is fed into a classifier consisting of a multilayer perceptron (MLP), and the classification probability is calculated through a Softmax layer.

[0160] An L1 regularization term to maintain the sparsity of the graph was added. Guided by the cross-entropy loss function, the model ultimately outputs a probability of 0.87 for the sample belonging to Alzheimer's disease (AD) and a probability of 0.13 for the sample belonging to healthy controls (NC), thus successfully and accurately completing the early diagnosis and prediction of brain diseases.

[0161] The above description is merely a specific embodiment of the invention, but the scope of protection of the invention is not limited thereto. Any variations or substitutions conceived without inventive effort should be included within the scope of protection of the invention. Therefore, the scope of protection of the invention should be determined by the scope defined in the claims.

Claims

1. A brain disease classification and prediction method based on a multimodal topological sensing graph network, characterized in that: Includes the following steps: Acquire the first modal timing signal and the second modal spatial structure signal of the target object; Based on the first modal time-series signal, multi-dimensional interaction features are extracted to construct a multi-view functional interaction network that characterizes the dynamic and static interaction relationship of the target object; Physical connectivity is extracted based on the second modal spatial structure signal, and a spatial structure prior matrix representing the connection relationship of the target object entity is constructed. The spatial structure prior matrix is ​​used as a topology modulator to constrain the spatial connectivity of the edge weights of the multi-view functional interaction network, and the constrained network is weighted and integrated through a cross-modal attention mechanism to generate a spatiotemporally coupled global fusion graph. Information is transmitted on the global fusion graph using a graph neural network model to extract local node representations; the local node representations are mapped and aggregated into mesoscale subnetwork representations using a learnable assignment matrix; and the graph theory invariants of the global fusion graph are calculated to generate a global topological statistical representation. Based on the local node representation, mesoscale subnetwork representation and global topological statistical representation, a global state feature vector of the target object is constructed. The global state feature vector is input into a two-stage optimization learning framework, and the network feature extraction parameters are updated using a loss function with multi-objective constraints to output the classification prediction result of the target object.

2. The brain disease classification and prediction method based on a multimodal topological sensing graph network according to claim 1, characterized in that: Extracting multi-dimensional interaction features based on the first modal time-series signal includes: Extracting the overall linear correlation view: Calculating the pairwise linear dependencies of time-series signals over the entire time span; Extracting conditionally independent related views: Introducing regularization constraints to eliminate the indirect transmission effect in the linear dependency relationship and obtain the direct interaction relationship; Extracting dynamic evolution relationship views: Local interaction relationships are extracted by time-series segmentation and aggregated along the time dimension to supplement the missing time-series evolution information in the overall linear relationship view and conditionally independent relationship view.

3. The brain disease classification and prediction method based on a multimodal topological sensing graph network according to claim 2, characterized in that: The method for constraining the edge weights of a multi-view functional interaction network using the spatial structure prior matrix as a topology modulator includes the following steps: The spatial structure prior matrix is ​​normalized into connectivity probability weights; The connectivity probability weights are multiplied by the adjacency matrix of the multi-view functional interaction network using the Hadamard product operation. The entity probability of the spatial structure is used to penalize and suppress redundant edges in the functional interaction network that lack physical connection support, thereby outputting a candidate network graph modulated by the spatial structure.

4. The brain disease classification and prediction method based on a multimodal topological sensing graph network according to claim 3, characterized in that: The method for generating a spatiotemporally coupled global fusion map through the cross-modal attention mechanism includes the following steps: The multi-view functional interaction network and the candidate network map modulated by spatial structure are respectively mapped to the shared latent space to generate corresponding view feature vectors; The relative importance of each view feature vector in the classification task is evaluated through an attention mechanism, and the corresponding weights are calculated. By using the corresponding weights to adaptively linearly combine the adjacency matrices of each view, the final generated global fusion map can dynamically balance the proportion of temporal features and prior information of spatial structure.

5. The brain disease classification and prediction method based on a multimodal topological sensing graph network according to claim 1, characterized in that: The method for representing local nodes includes the following steps: Based on a preset edge filtering threshold, the global fusion graph is sparsified to establish information propagation paths between nodes; A local feature aggregator based on an attention mechanism is used to aggregate the neighborhood information of the target node along the information propagation path according to the dynamic attention coefficients between adjacent nodes, thereby generating a node representation that includes local topology awareness.

6. The brain disease classification and prediction method based on a multimodal topological sensing graph network according to claim 5, characterized in that: The method for extracting the mesoscale subnetwork representation includes the following steps: Initialize a differentiable node assignment matrix to learn the mapping probability of each node in the network graph belonging to a specific mesoscale functional module; The local node representation and the node allocation matrix are interacted by matrix multiplication. Based on the mapping probability, the discrete node features are weighted and fused to generate a mesoscale subnetwork representation of the local dense interactive community of the representation system.

7. The brain disease classification and prediction method based on a multimodal topological sensing graph network according to claim 6, characterized in that: The method for generating the global topological statistical representation includes the following steps: For the global fusion graph, its clustering coefficient, feature path length, and modularity are explicitly calculated as graph topological statistical invariants; The graph topological statistical invariants are encoded in high dimension using a nonlinear mapping network and used as global prior information. This information is then concatenated and interacted with the mesoscale sub-network representation in terms of feature dimensions, so that the final global state feature vector has micro, meso and macro structural attributes.

8. The brain disease classification and prediction method based on a multimodal topological sensing graph network according to claim 1, characterized in that: In the pre-training phase of the two-stage optimized learning framework, the inverse constraints of the loss function on the feature extraction network include: Topology consistency loss interaction: By perturbing the network input, the similarity between the global state feature vectors before and after the perturbation in the latent space is maximized, thereby driving the network to learn noise-resistant features; Interaction between sparsity and community constraint loss: Combining the L1 norm of the network tensor and the graph modularity index, the generation process of the global fusion graph is constrained in reverse, forcing it to conform to the sparsity of real physical or biological connections. Cross-view reconstruction loss interaction: The driver decoder uses static view features to inversely reconstruct dynamic view features, thereby enhancing the feature extraction network's ability to capture the inherent temporal dependencies between different views.

9. A brain disease classification and prediction method based on a multimodal topological sensing graph network according to claim 8, characterized in that: The two-stage optimization learning framework also includes: Under the constraint of freezing the parameters of the pre-feature extraction network, the global state feature vector is input into the downstream classifier; The cross-entropy loss of the classification error is calculated, and the error gradient is fed back to the classifier parameters and the node allocation matrix separately. While maintaining the stability of the global feature extraction capability, the mapping relationship of the driving module is adaptively adjusted to the specific classification task.

10. A brain disease classification and prediction system based on a multimodal topological sensing graph network, characterized in that: include: The multimodal signal data processing module is used to acquire the first modal temporal signal and the second modal spatial structure signal of the target object, extract multi-dimensional interaction features based on the first modal temporal signal, construct a multi-view functional interaction network representing the dynamic and static interaction relationship of the target object, and extract physical connectivity based on the second modal spatial structure signal to construct a spatial structure prior matrix representing the entity connection relationship of the target object. The spatial constraint module is used to use the spatial structure prior matrix as a topology modulator to constrain the spatial connectivity of the edge weights of the multi-view functional interaction network, and to perform weighted integration of the constrained network through a cross-modal attention mechanism to generate a spatiotemporally coupled global fusion graph. The hierarchical topological feature interaction and global representation construction module is used to transmit information on the global fusion graph using a graph neural network model, extract local node representations, map and aggregate the local node representations into mesoscale sub-network representations using a learnable assignment matrix, and simultaneously calculate the graph theory invariants of the global fusion graph to generate a global topological statistical representation. The two-stage optimization and classification output module is used to construct a global state feature vector of the target object through the local node representation, the mesoscale sub-network representation and the global topological statistical representation. The global state feature vector is input into the two-stage optimization learning framework, and the network feature extraction parameters are updated using a loss function with multi-objective constraints. The classification prediction result of the target object is then output.

Images