Multi-mode collaborative characterization brain disease prediction method and device based on coupling network

Abstract

The invention discloses a multi-mode collaborative characterization brain disease prediction method and device based on a coupling network. The method comprises the steps of obtaining a function connection diagram and a structure connection diagram of a to-be-tested object; inputting the Gaussian noise and the coarse fusion image into a trained diffusion model to obtain a high-order coupling brain network of the to-be-detected object output by the diffusion model; generating a motif set of the to-be-tested object based on the function connection diagram and the structure connection diagram of the to-be-tested object; constructing a heterogeneous graph by a to-be-detected object; based on the heterogeneous graph, a feature extraction model capable of extracting node information and side information in the heterogeneous graph is adopted, and feature vectors of the to-be-detected object are generated; and inputting the feature vector into a trained classification model to obtain a prediction result of the brain disease of the to-be-detected object output by the classification model. The motif in the heterogeneous graph captures the local pathological association of the two modes, so that the feature vector focuses on the disease core mode, and the accuracy of disease prediction is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and apparatus for predicting brain diseases based on multimodal collaborative representation of coupled networks. Background Technology

[0002] Early prediction of brain diseases (such as Alzheimer's disease, mental disorders, and brain tumors) is an effective means of disease prevention and control. Based on the prediction results, clinical intervention and related treatments before the onset of the disease can, to some extent, prevent the occurrence of the disease or reduce its severity.

[0003] In existing technologies, fusing multimodal data to construct and analyze brain networks has become a common method for predicting brain diseases. This method typically involves: fusing structural connectivity maps constructed using diffusion tensor imaging (DTI) and functional connectivity maps constructed using functional magnetic resonance imaging (fMRI) to build a brain network; using regions of interest (ROIs) on the brain map as nodes in this brain network; extracting node features based on the data modalities (functional or structural) of the constructed brain network; and inputting these node features into a classification model to obtain the predicted brain diseases output by the classification model.

[0004] In the multimodal fusion process described above, simple integration is only performed at the feature layer, such as by splicing or weighting. This inefficient fusion method mostly focuses only on feature fusion and ignores the structural relationships between multimodal data. It cannot fully learn the complex coupling relationships between multimodal data, resulting in information loss during fusion and low prediction accuracy of the classification model. Summary of the Invention

[0005] The purpose of this invention is to provide a method and device for predicting brain diseases based on multimodal collaborative representation of coupled networks. It uses a diffusion model to couple functional and structural modes to form a high-order coupled brain network, and further extracts features based on the heterogeneous graph formed by the high-order coupled brain network and motifs for disease prediction. This solves the problem of low prediction accuracy when using feature vectors with inefficient feature fusion for disease prediction.

[0006] This invention is achieved through the following technical solution:

[0007] The first aspect of this application provides a method for predicting brain diseases based on multimodal collaborative representation of coupled networks, including:

[0008] Obtain the functional connectivity graph of the object under test based on functional magnetic resonance imaging data, and the structural connectivity graph based on diffusion tensor imaging data;

[0009] Gaussian noise and a coarse fusion map are both input into a trained diffusion model to obtain a high-order coupled brain network of the test subject output by the diffusion model. The Gaussian noise is used as the noise input of the diffusion model, and the coarse fusion map is used as the conditional input of the diffusion model. The diffusion model performs inverse denoising on the noise map under the constraint of the coarse fusion map to generate a high-order coupled brain network. The coarse fusion map is determined by lightweight fusion of functional connectivity maps and structural connectivity maps.

[0010] Based on the functional connectivity graph and structural connectivity graph of the test object, a motif set of the test object is generated; each motif in the motif set is a subgraph in the brain map representing a disease-related region; the brain map includes a functional connectivity graph and a structural connectivity graph.

[0011] A heterogeneous graph is constructed for the test object; the heterogeneous graph uses the higher-order coupled brain network of the test object and each motif in the motif set as nodes, and constructs motif-coupled graph edges based on the matching results of each motif in the higher-order coupled brain network, and constructs motif-motif edges based on the brain graph nodes where each motif is located.

[0012] Based on the heterogeneous graph, a feature extraction model capable of extracting node and edge information from the heterogeneous graph is used to generate the feature vector of the object to be tested.

[0013] The feature vector is input into the trained classification model to obtain the prediction results of the brain disease of the test subject output by the classification model.

[0014] In one feasible implementation, the method further includes:

[0015] For the high-order coupled brain network of the subject to be tested, determine the motif matching frequency of each edge in the high-order coupled brain network under a certain disease type;

[0016] Based on the motif matching frequency of the test subject under different disease types, the characteristic structural pattern of the disease type group possessed by the test subject is determined.

[0017] In one feasible implementation, determining the characteristic structural pattern of the disease type group possessed by the test object based on the motif matching frequency of the test object under different disease types specifically includes:

[0018] Based on the motif matching frequency of the subject under the first disease and the motif matching frequency under the second disease, the structural offset score of the edge of the high-order coupled brain network of the subject under the first disease relative to the second disease is determined.

[0019] If the structural offset score exceeds the first threshold, then the subject under test possesses the characteristic structural pattern of the first type of disease group;

[0020] If the structural offset score is lower than the second threshold, then the subject of the test possesses the characteristic structural pattern of the second type of disease group.

[0021] In one feasible implementation, the loss function of the diffusion model during the training phase is:

[0022] ;

[0023] In the formula, is the loss function of the denoising network in the diffusion model, which characterizes the deviation between the predicted noise of the denoising network and the real noise injected by the diffusion network; To control the strength of the consistency constraints on the structure of the control graph; The structural loss is determined by the similarity between the adjacency matrix of the high-order coupling graph output during diffusion model training and the adjacency matrix of the functional connectivity graph and the structural connectivity graph of the same sample.

[0024] In one feasible implementation, the method further includes: performing motif mining on the functional connectivity graph and the structural connectivity graph respectively to generate motif sets corresponding to different types of brain diseases; specifically including:

[0025] Obtain the functional connectivity and structural connectivity maps of each sample in a brain network dataset; the dataset contains samples of different types of brain diseases.

[0026] The farthest point sampling method is used to sample subgraphs from the functional connection diagram and the structural connection diagram respectively, and to determine the sampling subgraphs of the functional mode and the sampling subgraphs of the structural mode respectively.

[0027] The importance score for each subgraph is determined using a term frequency-inverse document frequency model.

[0028] Based on the importance score of each subgraph, the motif set corresponding to different types of brain diseases is determined.

[0029] In one feasible implementation, the method further includes:

[0030] The functional connectivity graph and structural connectivity graph of the object under test are respectively input into the trained autoencoder to obtain the latent representation of the functional connectivity graph and the latent representation of the structural connectivity graph output by the autoencoder in the latent space.

[0031] During the training phase, the autoencoder introduces positional encoding for nodes in the mind map, extracts local features of nodes in the mind map through a local message passing network, and extracts long-range dependency features between any two nodes in the mind map through a global attention mechanism.

[0032] In one feasible implementation, the loss function of the autoencoder during training includes the modality alignment loss of the encoder output and the difference loss of the decoder output.

[0033] The difference loss is characterized by the difference between the latent representation of the mode output by the decoder and the latent representation of the corresponding mode input to the autoencoder; the mode alignment loss is characterized by the maximum mean difference on the bimodal graph-level latent representation output by the encoder, wherein the bimodality includes structural mode and functional mode.

[0034] A second aspect of this application provides a brain disease prediction device based on multimodal collaborative representation of coupled networks, comprising:

[0035] The brain map acquisition unit is used to acquire the functional connectivity map of the subject based on functional magnetic resonance imaging data and the structural connectivity map based on diffusion tensor imaging data.

[0036] A coupled brain network generation unit is used to input both Gaussian noise and a coarse fusion map into a trained diffusion model to obtain a high-order coupled brain network of the test subject output by the diffusion model. The Gaussian noise serves as the noise input to the diffusion model, and the coarse fusion map serves as the conditional input to the diffusion model. The diffusion model performs inverse denoising on the noise map under the constraints of the coarse fusion map to generate a high-order coupled brain network. The coarse fusion map is determined by lightweight fusion of functional connectivity graphs and structural connectivity graphs.

[0037] A motif set generation unit generates a motif set for the test object based on its functional connectivity graph and structural connectivity graph; each motif in the motif set is a subgraph representing a disease-related region in a brain map; the brain map includes a functional connectivity graph and a structural connectivity graph.

[0038] Heterogeneous graph construction unit is used to construct a heterogeneous graph for the test object; the heterogeneous graph uses each motif in the higher-order coupled brain network and motif set of the test object as nodes, and constructs motif-coupled graph edges based on the matching results of each motif in the higher-order coupled brain network. If there are the same nodes between two motifs, then a motif-motif edge is constructed between the two motifs.

[0039] The feature extraction unit, based on the heterogeneous graph, uses a feature extraction model capable of extracting node and edge information from the heterogeneous graph to generate a feature vector of the object to be tested.

[0040] The prediction unit is used to input the feature vector into the trained classification model to obtain the prediction result of the brain disease of the test subject output by the classification model.

[0041] A third aspect of this application provides an electronic device, including: a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the above-described method.

[0042] A fourth aspect of this application provides a storage medium, comprising: storing a program or instructions on the storage medium, wherein the program or instructions, when executed by a processor, implement the steps of the above-described method.

[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0044] This embodiment of the application uses a diffusion model to fuse the functional connectivity graph and structural connectivity graph of the test object to form a high-order coupled brain network, achieving the fusion of functional and structural modalities. Furthermore, based on the heterogeneous graph constructed from the high-order coupled brain network and motifs, feature vectors are generated. Because the motifs in the heterogeneous graph capture the local pathological associations of the two modalities, the feature vectors are more focused on the core disease patterns. Therefore, the accuracy of disease prediction based on the feature vectors of this embodiment is improved. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0046] Figure 1 A flowchart illustrating a brain disease prediction method based on multimodal collaborative representation of coupled networks, provided in an embodiment of this application;

[0047] Figure 2(a) is a schematic diagram of the ring structure of the NC class subject's landmark connection structure provided in the embodiment of this application;

[0048] Figure 2(b) is a schematic diagram of the ring structure of the MCI class subject's signature connection structure provided in the embodiment of this application;

[0049] Figure 3 A schematic diagram of the overall framework of a brain disease prediction method based on multimodal collaborative representation of coupled networks provided in this application embodiment;

[0050] Figure 4A schematic diagram of the structure of the brain disease prediction device based on multimodal collaborative representation of coupled networks provided in the embodiments of this application;

[0051] Figure 5 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are for explanation only and are not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments in this application without creative effort are within the scope of protection of this application.

[0053] As will be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0054] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, product, or apparatus.

[0055] Example 1

[0056] Embodiment 1 of this application provides a brain disease prediction method based on multimodal collaborative representation of coupled networks, which solves the problem of low prediction accuracy of feature vectors used for disease prediction with inefficient feature fusion.

[0057] The subject executing this method can be any computing device capable of implementing the method, such as a server, mobile phone, personal computer, smart wearable device, smart robot, etc.

[0058] Furthermore, the embodiments of this application do not limit the execution order of different steps. When using the method provided in the embodiments of this application, the execution order of different steps can be adjusted according to actual needs.

[0059] For ease of description, the following uses a brain disease prediction method device based on multimodal collaborative representation of coupled networks as the execution subject of this method to provide a detailed description of the method provided in the embodiments of this application.

[0060] The Multimodal Collaborative Representation Framework (MCRF) proposed in this embodiment takes the functional / structural brain network constructed by fMRI and DTI as input and simultaneously models multimodal coupled topology and discriminative representation in a unified latent space. On the one hand, it adaptively generates coupled brain maps consistent with the function-structure relationship through graph autoencoding and conditional diffusion process of the latent space. On the other hand, it introduces population-level motif priors into the generated coupled network to construct heterogeneous graph neural networks to extract motif-level interpretable representations, which are ultimately used for the prediction and classification of brain diseases.

[0061] like Figure 1 The diagram shown is a flowchart illustrating the specific implementation of a brain disease prediction method based on multimodal collaborative representation of coupled networks, according to an embodiment of this application. The method includes the following steps 11-16:

[0062] Step 11: Obtain the functional connectivity diagram of the object under test based on functional magnetic resonance imaging data, and the structural connectivity diagram based on diffusion tensor imaging data.

[0063] Functional magnetic resonance imaging (fMRI) is a functional modal imaging technique based on the principle of magnetic resonance. Its core is to indirectly capture the activity state of neurons in the brain.

[0064] Functional Connectivity (FC) is a functional brain network constructed based on fMRI data. It depicts the "dynamic functional connections" between brain regions, rather than physical anatomical connections.

[0065] Based on BOLD (Blood Oxygenation Level Dependent) signals acquired by fMRI, functional connectivity (FC) calculates the correlation (e.g., Pearson correlation coefficient) of temporal signals between different brain regions (ROIs). If the BOLD signals of two brain regions fluctuate synchronously (high correlation coefficient), they are considered to have strong functional connectivity; otherwise, the connectivity is considered weak. This ultimately forms an N×N "functional connectivity matrix" (N being the number of brain regions), where the matrix elements represent the strength of functional associations between brain regions.

[0066] Diffusion Tensor Imaging (DTI) is also based on the principle of magnetic resonance, but it belongs to the structural modality imaging technology. Its core is to detect the diffusion movement of water molecules in the brain and reflect the anatomical structure and integrity of the white matter fiber bundles.

[0067] Structural Connectivity (SC) is a structural brain network constructed based on DTI data, which depicts the "physical anatomical connections" between brain regions, that is, the integrity of the connections between white matter fiber bundles.

[0068] Based on diffusion characteristics indicators from DTI (such as fractional anisotropy (FA) values ​​and fiber tracing results), SC calculates the "connectivity strength" of white matter fibers between different brain regions. For example, by counting the number of fibers from brain region A to brain region B, or calculating the average FA value of these fibers (reflecting fiber integrity), an N×N "structural connectivity matrix" is formed. The matrix elements represent the anatomical connectivity strength between brain regions.

[0069] Step 12: Input both Gaussian noise and coarse fusion map into the trained diffusion model to obtain the high-order coupled brain network of the test subject output by the diffusion model.

[0070] In this model, the Gaussian noise is used as the noise input of the diffusion model, and the coarse fusion graph is used as the conditional input of the diffusion model. The diffusion model performs inverse denoising on the noise graph under the constraints of the coarse fusion graph to generate a high-order coupled brain network. The coarse fusion graph is determined by lightweight fusion of the functional connectivity graph and the structural connectivity graph.

[0071] The fusion of the coarse fusion graph includes node feature fusion based on attention mechanisms and weighted fusion of the adjacency matrix, specifically:

[0072] Node features are processed using a Graph Convolutional Network (GCN):

[0073] , ;

[0074] in, and These represent node features from two different modalities (e.g., DTI and fMRI). and This represents the adjacency matrix for two modes. and This represents the node feature matrix under two modes.

[0075] Using an attention mechanism to fuse node features:

[0076] ;

[0077] Where Attention(·) represents the attention fusion function, softmax(·) represents the activation function, and d is the scaling factor. To query the weight matrix, The key weight matrix is... The value weight matrix.

[0078] For structural information, perform simple weighted fusion:

[0079] ;

[0080] in, and These are the normalized adjacency matrices of the two modes, and β is a weighting factor that takes a value between 0 and 1, used to control the fusion ratio of the two modes.

[0081] In one feasible implementation, the method of this embodiment further includes: inputting the functional connectivity graph and structural connectivity graph of the object under test into the trained autoencoder, respectively, to obtain the latent representation of the functional connectivity graph and the latent representation of the structural connectivity graph output by the autoencoder in the latent space.

[0082] The core objective of an autoencoder (AE) is to map a multimodal brain network (or brain map, i.e., functional connectivity map and structural connectivity map) to a reconstructable, alignable unified latent space, and then use a decoder to approximately restore the original map domain from that space.

[0083] During the pre-training phase of the autoencoder, positional encoding is introduced for nodes in the mind map, and local features of nodes in the mind map are extracted through a local message passing network, while long-range dependency features between any two nodes in the mind map are extracted through a global attention mechanism.

[0084] In the pre-training phase, SC and FC are used as encoders based on the GraphGPS concept, combining message-passing-based GCN and a global self-attention mechanism (Transformer). GCN helps the model capture the topological information of the local neighborhood, while Transformer models long-range dependencies between arbitrary node pairs through global attention. To further enhance the representation of node positional information, we introduce positional encoding (PE) for each node.

[0085] To simultaneously capture regional features and connectivity patterns in brain networks, we jointly model nodes and edges in the encoder. Let the input graph (including structural and functional connectivity graphs) be (X,A), and the encoder... Map it to the node latent representation Z and the edge latent representation W:

[0086] ;

[0087] in, Indicates potential node embedding. This represents the latent edge embedding. The update of node features involves two parts: local message passing and global attention.

[0088] ;

[0089] in, This represents the latent embedding vector of ROI node i in the l-th layer encoder. The location encoding vector for the ROI node. Let represent the updated latent embedding vector of node i in the (l+1)th layer encoder.

[0090] The representation of an edge is determined by both the node pair features and the edge weight:

[0091] ;

[0092] in, Let represent the updated latent embedding vector of the edge between ROI nodes i and j in the (l+1)th layer encoder. In the MLP(·) multilayer perceptron, [·, ·, ·] denotes the vector concatenation operation. Let be the original edge weights between nodes i and j in the input graph.

[0093] The final latent representation of the graph It serves as input to the latent spatial diffusion model during the training phase, used to model the joint distribution of multimodal brain networks.

[0094] After learning the latent representation and aligning its modalities, mapping it to specific graph structures and features in the original graph space is a crucial step in generating multimodal brain networks. Experiments show that complex decoders (such as GAT-based decoders) do not significantly improve reconstruction quality; therefore, we only set up a single linear layer as the decoder for nodes and edges. It is responsible for mapping it back to the original graph structure and feature space to generate the reconstruction result. ,in, The reconstructed node feature matrix is ​​the output of the decoder. The reconstructed adjacency matrix output by the decoder.

[0095] During the training phase of the autoencoder, to ensure that the decoder can recover node features and edge structure, we define:

[0096] ;

[0097] in, Indicates the losses incurred during reconstruction. (·) represents the expectation operation. This represents the squared L2 norm (the square of the Euclidean distance).

[0098] Meanwhile, to ensure that the structural modes (SC) and functional modes (FC) have a consistent statistical structure in the latent space, this embodiment applies a maximum mean discrepancy (MMD) regularization term to the graph-level latent representation output by the encoder.

[0099] Let the latent representations of FC and SC be respectively and The graph-level embedding is obtained through the graph readout operator:

[0100] ,

[0101] ;

[0102] Record and( The induced empirical distributions are respectively and Then the alignment loss is defined as:

[0103] ;

[0104] By minimizing The encoder is explicitly constrained to map the multimodal brain network to a unified and alignable latent space, gradually aligning the latent distributions of different modalities and providing a stable representational basis for subsequent coupling graph generation. The training objective of the autoencoder is to minimize:

[0105] ;

[0106] This represents the loss function of the autoencoder during training, which includes the modality alignment loss of the encoder output. The difference loss between the decoder output and the output .

[0107] The difference loss is characterized by the difference between the latent representation of the mode output by the decoder and the latent representation of the corresponding mode input to the autoencoder; the mode alignment loss is characterized by the maximum mean difference on the bimodal graph-level latent representation output by the encoder, wherein the bimodality includes structural mode and functional mode.

[0108] In this embodiment, a diffusion model is used to learn the dynamic evolution process of the latent data distribution by gradually adding and removing noise to the representations of nodes and edges in the latent space.

[0109] During the training phase of the diffusion model, the autoencoder is a pre-trained model with fixed parameters. An autoencoder consists of an encoder and a decoder.

[0110] The encoder output serves as the initial latent representation. Noise is gradually injected during the forward pass to simulate random perturbations:

[0111] ;

[0112] in, Let represent the noisy latent representation in the t-th step of the diffusion process. Indicates the noise scheduling parameters Defined cumulative product, , This represents the noise intensity parameter at step s. This represents the noise scheduling parameters at step t. This represents a Gaussian noise vector that follows a standard normal distribution. ; This represents a standard normal distribution with a mean of 0 and a covariance matrix of identity matrix I.

[0113] The goal of the reverse process is to start from the noisy state. Gradually restore the original latent representation .

[0114] During the graph structure generation stage, noise is applied to all edges, making edge indices indistinguishable in the latent space. This embodiment employs a Graph Transformer, which is independent of edge indices, as the denoising network. This structure can compute global attention between any pair of nodes, thus avoiding the limitations of traditional adjacency-index-based GNNs.

[0115] To achieve controllable generation of high-order coupled brain networks, this embodiment further models the conditional distribution p(h|c), where h represents the latent graph representation and c is the conditional graph corresponding to the coarse fusion graph. This embodiment introduces a cross-attention mechanism into the diffusion network, enabling the generation of high-order coupled brain networks. Conditional denoising network. (h,t,c) uses the conditional graph c as an additional input to guide the reconstruction process of the latent representation.

[0116] To achieve this controllable generation process, we define a conditional diffusion loss function:

[0117] ;

[0118] in, The objective function represents the conditional embedding obtained through the conditional graphical encoder. This objective function enables the model to learn the generative distribution under conditional constraints in the latent space, achieving controllable generation of brain networks.

[0119] Noise-dependent prediction loss can only be constrained at the latent space level. The consistency with Gaussian noise does not explicitly characterize the relationship between the generated brain network and the original FC / SC in the graph structure space. To address this, we further introduce a cosine similarity-based structure consistency regularization term in the graph domain to encourage the generated high-order coupled brain networks to maintain topological consistency with both the functional connectivity graph and the structural connectivity graph.

[0120] Specifically: Let the generative brain network obtained through the denoising network and the decoder be denoised as... The functional connectivity graph and structural connectivity graph of the corresponding subjects in the dataset are denoted as follows: and We flatten the adjacency matrix into a vector. =vec( ), =vec( ), =vec( And define a structural loss based on cosine similarity:

[0121] ;

[0122] Where, cos(u, v) = , and The weighting coefficients control the proximity of FC / SC connections. This regularization term does not depend on the absolute scale of the edge weights but focuses on the directional consistency of connection patterns, thus making it more suitable for characterizing the similarity of relative connection patterns in brain networks. Finally, the optimization objective (loss function) for the conditional diffusion stage can be written as:

[0123] ;

[0124] In the formula, is the loss function of the denoising network in the diffusion model, which characterizes the deviation between the predicted noise of the denoising network and the real noise injected by the diffusion network; To control the strength of the consistency constraints on the structure of the control graph; This is the structural loss based on the similarity between the adjacency matrix of the high-order coupled graph output during diffusion model training and the adjacency matrices of the functional connectivity graph and the structural connectivity graph of the same sample. By simultaneously imposing constraints in the latent space and the graph structure space, the model can learn a high-order multimodal collaborative brain network that conforms to the global generative distribution and maintains consistency with the original structural / functional connectivity patterns under multimodal conditions.

[0125] Step 13: Based on the functional connectivity graph and structural connectivity graph of the test object, generate a motif set for the test object; each motif in the motif set is a subgraph representing a disease-related region in the brain map; the brain map includes a functional connectivity graph and a structural connectivity graph.

[0126] In complex network research, small subgraph structures that appear frequently and are significantly more frequent than random graphs are called motifs. Based on this, this embodiment interprets motifs in brain networks as special structural patterns that characterize local interactions in disease-related brain regions, introducing motif-based structural priors for representation learning. We construct a heterogeneous graph for each sample, consisting of motif nodes and coupled graph nodes, enabling the model to capture motif-level discriminative patterns in coupled brain networks, thereby obtaining more interpretable representations.

[0127] In one feasible implementation, this embodiment includes motif mining of both the functional connectivity graph and the structural connectivity graph to generate motif sets corresponding to different brain disease types. Specifically, this includes: acquiring the functional connectivity graph and structural connectivity graph of each sample in a brain network dataset; the dataset containing samples of different brain disease types; using the furthest point sampling method to sample subgraphs from the functional connectivity graph and the structural connectivity graph respectively, determining the sampled subgraphs of the functional modality and the sampled subgraphs of the structural modality respectively; using a term frequency-inverse document frequency model to determine the importance score of each subgraph; and based on the importance score of each subgraph, determining the motif sets corresponding to different brain disease types respectively.

[0128] Specifically, the motif set in this embodiment is composed of recurring local connectivity patterns from the original functional connectivity graph (FC, fMRI) and structural connectivity graph (SC, DTI), representing the most discriminative motif-level structural units in multimodal brain networks. By independently extracting motifs from the two modalities and merging them into a unified motif set, it can be used to examine whether the generated coupling graph contains these key connectivity patterns at the function-structure level, thereby indirectly evaluating whether the model captures cross-modal dependent structures and using them for subsequent representation learning.

[0129] For a brain network dataset containing M subjects, including healthy controls with normal cognitive impairment (NC) and individuals with mild cognitive impairment (MCI), we also have functional connectivity maps for each subject. and structural connection diagram Therefore, a multimodal dataset can be denoted as:

[0130] ;

[0131] The mind map for each modality is defined as follows:

[0132] ;

[0133] Where V is a set of common nodes containing N brain regions (ROIs), Let m be the set of edges for mode m. For the connection strength matrix, This is the node feature matrix.

[0134] To extract high-frequency local connectivity structures from the two modalities, we first perform subgraph sampling for each modality in the brain map of each subject, denoted as:

[0135] ;

[0136] To obtain a local connectivity pattern that uniformly covers the entire brain map, such that:

[0137] .

[0138] Since brain networks are high-dimensional and structurally complex in both modalities, directly enumerating patterns in the entire graph is too costly. Therefore, we use farthest-point sampling (FSS) to ensure the diversity and representativeness of the sampled subgraphs.

[0139] Subsequently, significantly recurring motifs were identified in the sampled subgraphs of FC and SC, respectively. To systematically measure the frequency of these motifs and their ability to distinguish between groups, we used the term frequency–inverse document frequency (TF-IDF) model to evaluate their importance. Based on the importance TF-IDF scores, we obtained the motif sets for the NC and MCI groups, respectively.

[0140] ;

[0141] .

[0142] Will and These were considered as motif-level biomarkers for healthy controls and MCI patients, used for motif matching in the coupling graph and subsequent representation learning.

[0143] Step 14: Construct a heterogeneous graph for the test object; the heterogeneous graph uses the higher-order coupled brain network of the test object and each motif in the motif set as nodes, and constructs motif-coupled graph edges based on the matching results of each motif in the higher-order coupled brain network. If there are identical nodes between two motifs, then construct motif-motif edges between the two motifs.

[0144] The heterogeneous graph in this embodiment contains two types of nodes: one high-order coupled brain network node and K motif instance nodes. Based on the motif prior structure in FC and SC, the features of the high-order coupled brain network at the motif layer can be obtained through the heterogeneous graph.

[0145] One high-order coupled brain network node represents the generative coupling graph of the current subject. Its initial feature is the global representation after the graph is read out.

[0146] K motif instance nodes: in generating the coupling graph All actually matched The vocabulary contains radix instances (each instance is a node). For radix instance nodes, one-hot encoding is used to generate node features. Specifically, if the size of the radix vocabulary is |V|, then each radix node i corresponds to a feature vector Xi of length |V|. When the index corresponds to the radix, Xi[i] = 1, and the rest are 0.

[0147] Edges connecting different nodes in a heterogeneous graph include baso-coupled graph edges (connecting baso-nodes and coupled graph nodes) and baso-baso-node edges (connecting different baso-nodes).

[0148] Among them, motif-coupled graph edges are determined by the matching results of each motif in the higher-order coupled brain network: if the motif instance In coupling diagram If a match is successfully found, then an edge-coupled graph is established to connect the motifs. With brain (higher-order coupled brain network) nodes; edge weights are defined as motifs. The population-level TF-IDF score multiplied by the matching frequency in the current sample. This preserves population discriminative power while reflecting individual specificity.

[0149] Motif-motif edges: determined by the brain map nodes where each motif is located: if two motif instances share at least one brain region, then a motif-motif edge is established to connect the two motif nodes; the edge weight is the Jaccard coefficient (number of shared brain regions / number of union brain regions).

[0150] Step 15: Based on the heterogeneous graph, a feature extraction model capable of extracting node and edge information from the heterogeneous graph is used to generate the feature vector of the object to be tested.

[0151] Feature extraction models can be Relation Graph Convolutional Network (RGCN) or Heterogeneous Graph Transformer (HGT). The heterogeneous graph is then subjected to 3–4 layers of message passing through the feature extraction model, ultimately yielding: the final embedding hbrain of the high-order coupled brain network nodes; and hmotif obtained by mean + max pooling of the embeddings of all motif instance nodes.

[0152] We utilize heterogeneous graph neural networks to learn the feature embeddings at each motif level. Furthermore, we input a separate GCN layer into each higher-order coupled brain network to obtain another dimension of feature embeddings. Finally, we concatenate the embedding vectors from the motif level and the node level to obtain the feature vector of the test object.

[0153] Step 16: Input the feature vector into the trained classification model to obtain the prediction results of the brain disease of the test subject output by the classification model.

[0154] It should be noted that the output of the brain disease of the test subject in step 16 can be mild cognitive impairment or health (no brain disease).

[0155] The classification model can be a multilayer perceptron model.

[0156] In one feasible implementation, to achieve visualized biomarker discovery at the brain region-edge level, this method offers greater clinical interpretability. This embodiment also includes:

[0157] For the high-order coupled brain network of the test subject, determine the motif matching frequency of each edge in the high-order coupled brain network under a certain disease type; based on the motif matching frequency of the test subject under different disease types, determine the characteristic structural pattern of the disease type group possessed by the test subject.

[0158] The step of determining the characteristic structural pattern of the disease type group possessed by the test subject based on the motif matching frequency under different disease types specifically includes: determining the structural offset score of the edges of the test subject's higher-order coupled brain network under the first disease type relative to the second disease type based on the motif matching frequency under the first disease type and the motif matching frequency under the second disease type; if the structural offset score exceeds a first threshold, the test subject possesses the characteristic structural pattern of the first disease type group; if the structural offset score is lower than a second threshold, the test subject possesses the characteristic structural pattern of the second disease type group.

[0159] It should be noted that the "first" and "second" in the first and second categories of diseases mentioned above are used to distinguish different groups. The first and second categories of diseases can correspond to the mild cognitive impairment group and the healthy group (aencephaly). That is, in this embodiment, health is regarded as the aencephaly type in the classification of brain diseases.

[0160] A specific implementation could be:

[0161] In obtaining motif sets for both NC and MCI subjects and and the higher-order coupled brain output of the decoder Next, we aim to mine structural connectivity patterns among different population groups at the motif level. To this end, we construct intra-class edge frequency statistics based on motif matching results and define signature connectivity patterns for NC and MCI accordingly.

[0162] Let the label of the i-th subject be... The coupled brain diagram for ∈{NC,MCI} is denoted as:

[0163] ,

[0164] in, This is a set of nodes (corresponding to the original brain region ROI). Let be the set of edges. For any candidate edge (u,v), its basal order matching frequency in category c∈{NC,MCI} is defined as:

[0165] ,

[0166] in, Represents the radix set of category c; This indicates that motifs s are present in higher-order coupled brain networks. The set of edges contained in a certain matching instance; 1[·] is the indicator function: if edge (u,v) appears in the matching instance, the indicator function takes the value 1, otherwise it is 0.

[0167] motif matching frequency The frequency of edge (u,v) in the coupled brain network of class c and the edge associated with the motif of that class is characterized, reflecting the "frequency" of the edge in the motif patterns of that class.

[0168] Because the sample sizes of different categories may vary, the resulting motif matching frequencies across different categories lack comparability. Therefore, we normalize the motif matching frequencies as follows to obtain the intra-class matching frequencies:

[0169] ,

[0170] in, Let be the set of all candidate edges.

[0171] To further determine the population type of the coupled graph containing edge (u,v), the structural offset score of edge (u,v) in MCI relative to NC is defined as follows:

[0172] ;

[0173] When the value of d(u,v) exceeds the set first threshold, it indicates that the frequency of this edge matching with the MCI motif in the coupling graph of MCI subjects is significantly higher than that of NC, and therefore it can be regarded as a characteristic structural pattern of the MCI population.

[0174] When the value of d(u,v) is less than the set second threshold, it indicates that the edge is more common in the NC population and can be regarded as a characteristic structural pattern of the healthy control group.

[0175] The first threshold and the second threshold can be determined experimentally. Usually, the values ​​of the first threshold and the second threshold are different, and the first threshold is greater than the second threshold.

[0176] Based on the above structural offset scores, select the highest-scoring edges (e.g., K edges) from both MCI and NC to form their signature connection sets:

[0177] ,

[0178] ;

[0179] Where TopK{·} represents selecting the K edges with the highest scores. In visualization, we will... and The brain regions were plotted as chord diagrams, with brain regions as nodes and the landmark edges connecting two nodes as arcs. The thickness or color of the arcs represented their scores or matching frequencies, as shown in Figures 2(a) and 2(b). These diagrams presented the landmark connection structures of NC and MCI subjects, respectively, and could intuitively show the structural differences in landmark connections (connections between nodes) among different populations.

[0180] like Figure 3The diagram illustrates the overall framework of this embodiment: Based on two modalities of brain maps (functional connectivity graph and structural connectivity graph), a coarse fused graph is obtained through lightweight fusion. Latent space representations are then performed on both modalities of the brain maps. These latent space representations are then processed through a diffusion process to generate a noisy representation H_T. This noisy representation H_T is then processed by a denoising network, and the coarse fused graph is added as a condition to generate a high-order coupled brain network. The high-order coupled brain network and motif set are used to generate a heterogeneous graph. Features are extracted from this heterogeneous graph, and another dimension of features is extracted from the high-order coupled brain network via a GCN. These two features are then fused to form a feature vector for disease prediction. This feature vector is then input into a multilayer perceptron (MLP) to complete the prediction task.

[0181] To verify the effectiveness of the prediction method proposed in the above embodiments, the method of this embodiment will be specifically implemented below:

[0182] Dataset preparation and preprocessing:

[0183] fMRI and DTI data of 402 subjects were selected from the multimodal brain imaging dataset of ADNI (Alzheimer's Disease Neuroimaging Initiative). Among them, there were 192 patients with mild cognitive impairment (MCI) and 210 normal controls (NC).

[0184] All selected image data underwent preprocessing, including slice time correction, motion correction, spatial normalization, and denoising, and were registered to the MNI standard space.

[0185] For the preprocessed image data, the AAL (Automated Anatomical Labeling) template was used to divide the whole brain into 90 ROI nodes to construct a unified graph structure.

[0186] For fMRI data in the image data, the average BOLD time series is extracted from each ROI, and after bandpass filtering and detrending processing, node-level functional features are formed; and the Pearson correlation coefficient between the time series of each ROI is calculated to generate a functional connectivity matrix, which serves as the edge features of the functional connectivity graph.

[0187] For DTI data in the imaging data, a structural connectivity matrix is ​​obtained based on fiber tracking. Its elements typically represent the number of fibers between ROIs or the average FA (Fractional Anisotropy) value, serving as the edge features of the structural connectivity graph. Since the structural graph itself does not contain native attributes at the node level, this embodiment constructs a one-hot encoding for each ROI as a node feature to ensure strict alignment with the fMRI data in the node space.

[0188] Experimental setup:

[0189] The MCRF (Multimodal Collaborative Representation Framework) framework of this embodiment was implemented using PyTorch, and experiments were conducted on an RTX3090 with 24GB of memory. We used five metrics to evaluate model performance, including ACC (Accuracy), F1 (F1-Score, the harmonic mean of combined precision and recall), AUC (Area Under ROC Curve), SEN (Sensitivity), and SPE (Specificity). The encoder and denoising network had 3 encoding layers, and the latent dimension d of nodes and edges in the latent space was set to 64. A masking operation was performed on nodes and edges with a masking rate of 0.2.

[0190] All subjects were randomly divided into five equal-sized subsets. One subset was used as the test set, and the union of the other four subsets was used as the training set. This process was repeated five times to eliminate random bias. The classification performance was evaluated using accuracy (ACC), sensitivity (SEN), specificity (SPE), and area under the ROC curve (AUC), and the standard deviation was recorded.

[0191] Model Comparison:

[0192] To verify the effectiveness of the method proposed in this embodiment, comparative experiments were conducted with several representative models. These representative models include unimodal brain network models and multimodal fusion methods proposed in recent years.

[0193] Table 1 presents the results of the comparative experiment. It can be seen that the method of this embodiment (Ours) outperforms the listed representative models in terms of ACC, SEN, SPE and AUC, demonstrating the advantages of the method of this embodiment in multimodal graph representation learning.

[0194]

[0195] Table 1

[0196] Ablation experiment:

[0197] To systematically evaluate the effectiveness of each module proposed in this embodiment, multiple ablation experiments were conducted based on the ADNI dataset. The optimizer used during model training was Adam (learning rate 1×). (batch size=32), 100 training rounds, and the random seed (42) remains consistent. To ensure comparability, only one component is removed or replaced in each experiment, while the rest of the settings remain completely consistent.

[0198] To analyze the impact of different encoder structures on model performance, we replaced the original GraphGPS encoder in the MCRF framework proposed in this embodiment, and compared the results using GCN, GAT, GraphSAGE, and Transformer-only structures. Meanwhile, to verify the contribution of higher-order representation modules, we list the results of removing the motif-graph co-embedding module (w / oMGCE) in the same table. All experiments maintained consistency in latent space dimensions, optimizer, training epochs, and data partitioning.

[0199] The results shown in Table 2 indicate that the model performance decreased to varying degrees after replacing the encoder structure. The GCN encoder, by capturing only local neighborhood information, lacked the ability to model cross-modal global dependencies. GAT and GraphSAGE performed slightly better than the other replacement encoders, but still could not simultaneously capture both global and local features. Using only the Transformer encoder (without the message passing mechanism) significantly reduced accuracy and stability, indicating that local topological information is also crucial in brain network modeling. Furthermore, removing the motif-graph co-embedding module (w / o MGCE) further reduced performance, demonstrating the importance of this module in capturing higher-order function-structure relationships. Therefore, this further confirms that the MCRF architecture in this embodiment is a superior implementation for feature representation.

[0200] The overall results show that the GraphGPS structure achieves the best balance in capturing local-global topological dependencies, while the motif-graph collaborative representation (MGCE) further enhances the discriminative power and interpretability of the feature layer.

[0201]

[0202] Table 2

[0203] This embodiment of the application uses a diffusion model to fuse the functional connectivity graph and structural connectivity graph of the test object to form a high-order coupled brain network, achieving the fusion of functional and structural modalities. Furthermore, based on the heterogeneous graph constructed from the high-order coupled brain network and motifs, feature vectors are generated. Because the motifs in the heterogeneous graph capture the local pathological associations of the two modalities, the feature vectors are more focused on the core disease patterns. Therefore, the accuracy of disease prediction based on the feature vectors of this embodiment is improved.

[0204] Example 2

[0205] To address the problem of low prediction accuracy in disease prediction using feature vectors obtained through inefficient feature fusion in existing technologies, and based on the same inventive concept as Embodiment 1, this application also provides a brain disease prediction device based on multimodal collaborative representation of coupled networks.

[0206] A schematic diagram of the specific structure of the device is shown below. Figure 4 As shown, it includes the following functional units 31 to 36:

[0207] The brain map acquisition unit 31 is used to acquire the functional connectivity map of the subject constructed based on functional magnetic resonance imaging data and the structural connectivity map constructed based on diffusion tensor imaging data.

[0208] The coupled brain network generation unit 32 is used to input both Gaussian noise and a coarse fusion map into a trained diffusion model to obtain a high-order coupled brain network of the test subject output by the diffusion model. The Gaussian noise is used as the noise input of the diffusion model, and the coarse fusion map is used as the condition input of the diffusion model. The diffusion model performs inverse denoising on the noise map under the constraint of the coarse fusion map to generate a high-order coupled brain network. The coarse fusion map is determined by lightweight fusion of the functional connectivity map and the structural connectivity map.

[0209] The coupled brain network generation unit is further configured to: input the functional connectivity graph and structural connectivity graph of the test object into the trained autoencoder, respectively, to obtain the latent representation of the functional connectivity graph and the latent representation of the structural connectivity graph output by the autoencoder in the latent space; during the training phase, the autoencoder introduces positional encoding for the nodes in the brain map, extracts local features of the nodes in the brain map through a local message passing network, and extracts long-range dependency features between any two nodes in the brain map through a global attention mechanism.

[0210] The loss function of the autoencoder during training includes the modality alignment loss of the encoder output and the difference loss of the decoder output; wherein, the difference loss is characterized by the difference between the latent representation of the modality output by the decoder and the latent representation of the corresponding modality input to the autoencoder; the modality alignment loss is characterized by the maximum mean difference on the bimodal graph-level latent representation output by the encoder, wherein the bimodality includes structural mode and functional mode.

[0211] The loss function of the diffusion model during the training phase is:

[0212] ;

[0213] In the formula, is the loss function of the denoising network in the diffusion model, which characterizes the deviation between the predicted noise of the denoising network and the real noise injected by the diffusion network; To control the strength of the consistency constraints on the structure of the control graph; The structural loss is determined by the similarity between the adjacency matrix of the high-order coupling graph output during diffusion model training and the adjacency matrix of the functional connectivity graph and the structural connectivity graph of the same sample.

[0214] The motif set generation unit 33 generates a motif set for the test object based on the functional connectivity graph and structural connectivity graph of the test object; each motif in the motif set is a subgraph representing a disease-related region in the brain map; the brain map includes a functional connectivity graph and a structural connectivity graph.

[0215] The motif set generating unit is specifically used for:

[0216] The functional connectivity graph and structural connectivity graph of each sample in the brain network dataset are obtained; the dataset contains samples of different brain disease types; the furthest point sampling method is used to sample subgraphs from the functional connectivity graph and structural connectivity graph respectively to determine the sampled subgraphs of functional modality and structural modality respectively; the word frequency-inverse document frequency model is used to determine the importance score of each subgraph; based on the importance score of each subgraph, the motif set corresponding to different brain disease types is determined respectively.

[0217] Heterogeneous graph construction unit 34 is used to construct a heterogeneous graph for the test object; the heterogeneous graph uses each motif in the higher-order coupled brain network and motif set of the test object as nodes, and constructs motif-coupled graph edges based on the matching results of each motif in the higher-order coupled brain network. If there are the same nodes between two motifs, then a motif-motif edge is constructed between the two motifs.

[0218] Feature extraction unit 35, based on the heterogeneous graph, uses a feature extraction model that can extract node information and edge information in the heterogeneous graph to generate the feature vector of the object to be tested;

[0219] The prediction unit 36 ​​is used to input the feature vector into the trained classification model to obtain the prediction result of the brain disease of the test subject output by the classification model.

[0220] The prediction device in this embodiment further includes a landmark structural pattern recognition unit, specifically used for: determining the motif matching frequency of each edge in the high-order coupled brain network of the test object under a certain disease type; and determining the landmark structural pattern of the disease type group possessed by the test object based on the motif matching frequency of the test object under different disease types.

[0221] Based on the motif matching frequency of the test subject under different disease types, the characteristic structural pattern of the disease type group possessed by the test subject is determined. Specifically, this includes: based on the motif matching frequency of the test subject under the first type of disease and the motif matching frequency under the second type of disease, determining the structural offset score of the edges of the higher-order coupled brain network of the test subject under the first type of disease relative to the second type of disease; if the structural offset score exceeds a first threshold, the test subject possesses the characteristic structural pattern of the first type of disease group; if the structural offset score is lower than a second threshold, the test subject possesses the characteristic structural pattern of the second type of disease group.

[0222] This embodiment of the application uses a diffusion model to fuse the functional connectivity graph and structural connectivity graph of the test object to form a high-order coupled brain network, achieving the fusion of functional and structural modalities. Furthermore, based on the heterogeneous graph constructed from the high-order coupled brain network and motifs, feature vectors are generated. Because the motifs in the heterogeneous graph capture the local pathological associations of the two modalities, the feature vectors are more focused on the core disease patterns. Therefore, the accuracy of disease prediction based on the feature vectors of this embodiment is improved.

[0223] Based on the same inventive concept as the foregoing embodiments of this application, this application also provides a computing device.

[0224] like Figure 5 As shown, the computing device includes a memory 41 and a processor 42. The memory 41 can be configured to store various other data to support operation on the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device. The memory 41 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0225] The processor 42, coupled to the memory 41, is used to execute the computer program stored in the memory 41 for performing the brain disease prediction method based on multimodal collaborative representation of coupled networks as described in the foregoing embodiments.

[0226] When processor 42 executes the computer program to perform a brain disease prediction method based on multimodal collaborative representation of coupled networks, it fuses the functional connectivity graph and structural connectivity graph of the test object through a diffusion model to form a high-order coupled brain network, thereby achieving the fusion of functional and structural modalities. Furthermore, based on the heterogeneous graph constructed from the high-order coupled brain network and motifs, feature vectors are generated. Since the motifs in the heterogeneous graph capture the local pathological associations of the two modalities, the feature vectors are more focused on the core disease patterns. Therefore, the accuracy of disease prediction based on the feature vectors of this embodiment is improved.

[0227] When the processor 42 executes the computer program in the memory 41, in addition to the functions described above, it can also perform other functions, as detailed in the descriptions of the preceding embodiments.

[0228] Furthermore, such as Figure 5 As shown, the computing device also includes other components such as a display 44, a communication component 43, a power supply component 45, and an audio component 46. Figure 5 The diagram only shows some components and does not mean that the computing device includes only these components. Figure 5 The components shown.

[0229] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement the methods provided in the above embodiments.

[0230] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0231] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0232] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting brain diseases based on multimodal collaborative representation of coupled networks, characterized in that, include: Obtain the functional connectivity graph of the object under test based on functional magnetic resonance imaging data, and the structural connectivity graph based on diffusion tensor imaging data; Gaussian noise and a coarse fusion map are both input into a trained diffusion model to obtain a high-order coupled brain network of the test subject output by the diffusion model. The Gaussian noise is used as the noise input of the diffusion model, and the coarse fusion map is used as the conditional input of the diffusion model. The diffusion model performs inverse denoising on the noise map under the constraint of the coarse fusion map to generate a high-order coupled brain network. The coarse fusion map is determined by lightweight fusion of functional connectivity maps and structural connectivity maps. Based on the functional connectivity graph and structural connectivity graph of the test object, a motif set of the test object is generated; each motif in the motif set is a subgraph in the brain map representing a disease-related region; the brain map includes a functional connectivity graph and a structural connectivity graph. Construct a heterogeneous graph for the object to be tested; The heterogeneous graph uses the higher-order coupled brain network of the test object and each motif in the motif set as nodes, and constructs motif-coupled graph edges based on the matching results of each motif in the higher-order coupled brain network; if there are identical nodes between two motifs, then a motif-motif edge is constructed between the two motifs. Based on the heterogeneous graph, a feature extraction model capable of extracting node and edge information from the heterogeneous graph is used to generate the feature vector of the object to be tested. The feature vector is input into the trained classification model to obtain the prediction results of the brain disease of the test subject output by the classification model.

2. The method according to claim 1, characterized in that, The method further includes: For the high-order coupled brain network of the subject to be tested, determine the motif matching frequency of each edge in the high-order coupled brain network under a certain disease type; Based on the motif matching frequency of the test subject under different disease types, the characteristic structural pattern of the disease type group possessed by the test subject is determined.

3. The method according to claim 2, characterized in that, The determination of the characteristic structural pattern of the disease type group possessed by the test subject based on the motif matching frequency under different disease types specifically includes: Based on the motif matching frequency of the subject under the first disease and the motif matching frequency under the second disease, the structural offset score of the edge of the high-order coupled brain network of the subject under the first disease relative to the second disease is determined. If the structural offset score exceeds the first threshold, then the subject under test possesses the characteristic structural pattern of the first type of disease group; If the structural offset score is lower than the second threshold, then the subject of the test possesses the characteristic structural pattern of the second type of disease group.

4. The method according to claim 1, characterized in that, The loss function of the diffusion model during the training phase is: ； In the formula, is the loss function of the denoising network in the diffusion model, which characterizes the deviation between the predicted noise of the denoising network and the real noise injected by the diffusion network; To control the strength of the consistency constraints on the structure of the control graph; The structural loss is determined by the similarity between the adjacency matrix of the high-order coupling graph output during diffusion model training and the adjacency matrix of the functional connectivity graph and the structural connectivity graph of the same sample.

5. The method according to claim 1, characterized in that, The method further includes: performing motif mining on the functional connectivity graph and the structural connectivity graph respectively to generate motif sets corresponding to different types of brain diseases; specifically including: Obtain the functional connectivity and structural connectivity maps of each sample in a brain network dataset; the dataset contains samples of different types of brain diseases. The farthest point sampling method is used to sample subgraphs from the functional connection diagram and the structural connection diagram respectively, and to determine the sampling subgraphs of the functional mode and the sampling subgraphs of the structural mode respectively. The importance score for each subgraph is determined using a term frequency-inverse document frequency model. Based on the importance score of each subgraph, the motif set corresponding to different types of brain diseases is determined.

6. The method according to claim 1, characterized in that, The method further includes: The functional connectivity graph and structural connectivity graph of the object under test are respectively input into the trained autoencoder to obtain the latent representation of the functional connectivity graph and the latent representation of the structural connectivity graph output by the autoencoder in the latent space. During the training phase, the autoencoder introduces positional encoding for nodes in the mind map, extracts local features of nodes in the mind map through a local message passing network, and extracts long-range dependency features between any two nodes in the mind map through a global attention mechanism.

7. The method according to claim 6, characterized in that, The loss function of the autoencoder during training includes the modality alignment loss of the encoder output and the difference loss of the decoder output. The difference loss is characterized by the difference between the latent representation of the mode output by the decoder and the latent representation of the corresponding mode input to the autoencoder; the mode alignment loss is characterized by the maximum mean difference on the bimodal graph-level latent representation output by the encoder, wherein the bimodality includes structural mode and functional mode.

8. A brain disease prediction device based on multimodal collaborative representation of coupled networks, characterized in that, include: The brain map acquisition unit is used to acquire the functional connectivity map of the subject based on functional magnetic resonance imaging data and the structural connectivity map based on diffusion tensor imaging data. A coupled brain network generation unit is used to input both Gaussian noise and a coarse fusion map into a trained diffusion model to obtain a high-order coupled brain network of the test subject output by the diffusion model. The Gaussian noise serves as the noise input to the diffusion model, and the coarse fusion map serves as the conditional input to the diffusion model. The diffusion model performs inverse denoising on the noise map under the constraints of the coarse fusion map to generate a high-order coupled brain network. The coarse fusion map is determined by lightweight fusion of functional connectivity graphs and structural connectivity graphs. A motif set generation unit generates a motif set for the test object based on its functional connectivity graph and structural connectivity graph; each motif in the motif set is a subgraph representing a disease-related region in a brain map; the brain map includes a functional connectivity graph and a structural connectivity graph. Heterogeneous graph construction unit is used to construct a heterogeneous graph for the test object; the heterogeneous graph uses each motif in the higher-order coupled brain network and motif set of the test object as nodes, and constructs motif-coupled graph edges based on the matching results of each motif in the higher-order coupled brain network. If there are the same nodes between two motifs, then a motif-motif edge is constructed between the two motifs. The feature extraction unit, based on the heterogeneous graph, uses a feature extraction model capable of extracting node and edge information from the heterogeneous graph to generate a feature vector of the object to be tested. The prediction unit is used to input the feature vector into the trained classification model to obtain the prediction result of the brain disease of the test subject output by the classification model.

9. An electronic device, characterized in that, include: A processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as claimed in any one of claims 1-7.

10. A storage medium, characterized in that, include: The storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-7.

Images