A diagnostic method for autism spectrum disorder based on adaptive fusion multigraph Transformer

Abstract

This invention proposes a diagnostic method for autism spectrum disorder (ASD) based on adaptive fusion of multiple graph Transformers, targeting resting-state functional magnetic resonance imaging (rs-fMRI) brain network analysis scenarios. This method constructs a hybrid GCN-Transformer architecture, simultaneously modeling local topological features and global long-range dependencies in the brain network, and introduces bidirectional attention and a two-stream adaptive fusion mechanism to dynamically integrate complementary information from multiple functional connectivity patterns, such as Pearson correlation, sparse representation, and Granger causality, achieving efficient identification of complex neuropathological features of ASD. Experimental results show that the method achieves a classification accuracy of 94.7% on the ABIDE public dataset, a 3.6% improvement over existing state-of-the-art methods. Its innovation lies in: the first application of hybrid GCN-Transformers to brain network analysis, the proposal of a data-driven dynamic fusion strategy to replace traditional static fusion, and the systematic integration of multiple brain connectivity patterns, providing a more comprehensive and effective solution for intelligent diagnosis of brain diseases.

Description

Technical Field

[0001] This invention relates to the field of computer-aided medical diagnostic technology, specifically to an intelligent diagnostic method for autism spectrum disorder (ASD) that utilizes deep learning models to analyze functional magnetic resonance imaging (fMRI) data. Technical Background

[0002] Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder whose clinical diagnosis relies primarily on subjective behavioral assessments, which often leads to diagnostic delays or inconsistencies, thus missing crucial opportunities for early intervention.

[0003] With the development of neuroimaging technology, resting-state functional magnetic resonance imaging (rs-fMRI) has made it possible to explore objective neural biomarkers for ASD. By constructing functional brain networks (FBNs), researchers can analyze connectivity patterns between brain regions. However, existing technologies have the following limitations:

[0004] Limitations of the single connectivity model: Most methods use only a single connectivity metric (such as the Pearson correlation coefficient), which can only capture linear, non-directional relationships and ignores the rich complementary information that can be provided by other metrics (such as Granger causality, sparse representation).

[0005] Insufficient ability to model long-range dependencies: Traditional graph convolutional networks (GCNs) excel at capturing local topology, but due to their shallow receptive fields, they struggle to effectively capture long-range neural dynamic interactions that are crucial to ASD pathology.

[0006] Inefficiency of static feature fusion: Existing multimodal data fusion strategies often employ simple concatenation or average pooling. This static approach fails to consider the relative diagnostic importance of different connection modes, which may lead to poor model performance and interpretability.

[0007] Therefore, there is an urgent need for an innovative method that can integrate multiple brain network connectivity patterns, capture both local and global neural dependencies, and adaptively fuse features to improve the accuracy and reliability of ASD diagnosis. Summary of the Invention

[0008] We describe the entire process and loss function of our proposed MGTAF (Multiplex GraphTransformer with Adaptive Fusion) method for the diagnosis of autism spectrum disorder (ASD). This method aims to overcome the limitations of traditional methods that rely on single brain network connectivity patterns and cannot simultaneously capture long- and short-range neural dependencies. Specifically, the MGTAF method mainly includes the following steps:

[0009] Step 1. Construction and Representation of Multifunctional Brain Networks

[0010] As a diagnostic method based on brain functional connectivity, this invention first requires converting the raw fMRI time-series signals into graph structure data. Unlike traditional methods that rely solely on a single connectivity pattern, this invention constructs functional brain networks using multiple computational methods to more comprehensively capture the complex linear and nonlinear, directed and undirected interactions between brain regions.

[0011] Specifically, we constructed three different functional brain networks for each participant. The first is a Pearson correlation (PC)-based network, which quantifies the linear correlation between signals from different brain regions and is the most widely used measure of undirected connectivity. The second is a sparse representation (SR)-based network, which estimates the partial correlation between brain regions by solving a regression problem regularized with L1 norm, resulting in a sparser and more interpretable undirected network. The third is a Granger causal mapping (GCM)-based network, which determines the directed causal influence of one brain region on another by analyzing the lag relationships in time series. These three networks depict the functional organization patterns of the brain from different perspectives, and their construction process can be represented by the following formula:

[0012]

[0013] in, Represents the adjacency matrix. It is the first The data for each brain region was time-series. In this way, the data for each subject was represented as a set of three different graphs, providing rich and complementary input information for subsequent hierarchical feature learning.

[0014] Step 2. Hierarchical feature extraction using hybrid GCN-Transformer

[0015] To simultaneously model local connectivity patterns and global integration properties in brain networks, this invention proposes a hybrid GCN-Transformer architecture. This architecture first leverages the powerful capabilities of Graph Convolutional Networks (GCNs) to aggregate information within the neighborhood of each brain region (node), thereby effectively extracting local topological features. The GCN update process can be represented as follows:

[0016]

[0017] in, It is the first The node feature matrix of the layer This is the normalized adjacency matrix. After capturing local features using GCN, we introduce an edge-enhanced graph Transformer to model long-range dependencies between nodes. We incorporate the graph connection weights into a self-attention mechanism, enabling the model to focus on biologically more important or more tightly connected distant brain regions. The attention score is calculated as follows:

[0018]

[0019] in, , It is a query and key vector. It is Hadamaji. Representative side The weight information is then used. In the final stage of feature extraction, we use a MixPool layer to aggregate node-level features into a graph-level representation. This layer adaptively balances average pooling and max pooling through a learnable parameter to retain the most discriminative information.

[0020] Step 3. Fusion of bidirectional attention and adaptive features

[0021] After extracting high-level features from different brain network maps, the key lies in how to effectively fuse this information from different perspectives. To this end, this invention designs an innovative mechanism consisting of bidirectional attention and dual-stream adaptive fusion.

[0022] (1) Two-way attention interaction

[0023] We first facilitate information interaction between features from different modes using a bidirectional attention mechanism. For any two graph-level feature vectors X and Y, this mechanism dynamically generates attention weights by calculating their differences, thereby mutually reinforcing each other's representations and capturing their complementarity. This process can be represented as:

[0024]

[0025] (2) Dual-stream adaptive fusion

[0026] The features obtained after the interaction are fed together with the original features into the Two-Stream Adaptive Fusion (DSAF) module. This module contains two parallel information streams: one that processes the original features (the main information stream). Another feature obtained after processing the interaction (interaction information flow) The outputs of these two streams are passed through a learnable gating parameter. Dynamic weighted summation is performed, allowing the model to autonomously determine the contribution ratio of original features and interaction features. The fusion process is represented as:

[0027]

[0028] Finally, the fused feature vector H is processed by an enhancement layer and then fed into the classifier for final diagnostic prediction.

[0029] Step 4. Loss Function Optimization

[0030] We directly use the standard binary cross-entropy loss function to perform end-to-end optimization of the entire model. For a given sample whose true label is y (0 or 1), the model predicts the probability that it is ASD as follows: The loss function is defined as:

[0031]

[0032] By minimizing this simple yet effective loss function, the distance between the predicted value and the true label can be effectively narrowed, thereby driving the model to learn the brain network feature patterns that can accurately distinguish between ASD patients and normal controls.

[0033] Innovation

[0034] The following are the innovative aspects of the autism spectrum disorder diagnostic method based on adaptive fusion multigraph Transformer:

[0035] 1. A novel hybrid GCN-Transformer architecture is proposed for brain network analysis. This architecture organically combines the advantages of GCN in capturing local topological structures with the powerful ability of Transformer in modeling long-range dependencies. This hierarchical processing paradigm enables the model to simultaneously decode local functional connectivity anomalies and brain-wide integration dysregulation present in ASD, establishing a new paradigm for brain network analysis.

[0036] 2. A bidirectional attention and dual-stream adaptive fusion mechanism is designed for the dynamic integration of multi-source graph information. Unlike traditional static feature concatenation or averaging, this method explicitly models the complementary relationships between different connection patterns through a bidirectional attention mechanism and adaptively balances the contributions of original features and interaction features through a learnable gating mechanism. This dynamic fusion strategy significantly improves the model's ability to extract discriminative features from multi-source information.

[0037] 3. By introducing multiple connectivity patterns into the deep learning framework, a more comprehensive representation of brain functional connectivity is achieved. By simultaneously utilizing multiple metrics such as Pearson correlation, sparse representation, and Granger causality, this method captures different aspects of brain functional organization. This multi-perspective methodology provides a new approach for systematically integrating multimodal or multi-view neuroimaging data into a unified diagnostic model and validates its significant potential for improving diagnostic accuracy.

[0038] Results Analysis

[0039] Quantitative results

[0040] To validate the quantitative performance of the proposed MGTAF model, we conducted experiments on the publicly available ABIDE dataset, which includes 92 participants (45 ASD patients and 47 healthy controls), and compared it with previous state-of-the-art methods such as MPGCN and GAT. The results are shown in Table 2. Specifically, we compared the MGTAF model with several baseline methods in the same experimental settings. The MGTAF model achieved a classification accuracy of 94.7%, an average improvement of 3.6% compared to the existing state-of-the-art GCN method (91.1%). Furthermore, the model achieved precision, recall (sensitivity), and F1 score of 97.1%, 93.8%, and 95.4%, respectively. These results, combined with its performance on the ROC curve (…), demonstrate the effectiveness of the MGTAF model. Figure 5 The AUC value of up to 97.9% indicates that our method can make a diagnosis with an extremely low false positive rate while maintaining high sensitivity, demonstrating strong competitiveness and clinical application potential in the accurate identification of ASD.

[0041] Quality Results

[0042] To validate the quality performance of the MGTAF model and explore its biological interpretability, we conducted a series of qualitative analyses, see [link to relevant documentation]. Figure 6 and Figure 7 In node sensitivity analysis ( Figure 7 In Table 3, we quantified the contribution of each brain region to the model's classification decision using a gradient-based method. The results showed that the right amygdala (AMYG.R), orbitofrontal cortex (ORBsupmed.R), and left fusiform gyrus (FFG.L) regions had the highest sensitivity scores. These regions are closely related to emotion processing, social cognition, and executive function, and their high contribution is highly consistent with known neurobiological theories of ASD, thus providing neuroanatomical support for the model's effectiveness. Furthermore, we analyzed the evolution of the adaptive mixing parameter α in the MixPool layer (…). Figure 6The parameter eventually converges to within the range of 0.35 to 0.65, indicating that both average pooling and max pooling strategies make meaningful contributions to the classification task. This result demonstrates that the MGTAF model can adaptively balance globally distributed patterns with locally salient features and fine-tune according to the input characteristics of different connectivity patterns.

[0043] ablation experiment

[0044] To evaluate the effectiveness of key components in the model, particularly the role of the Transformer module (1), we conducted ablation experiments, and the results are presented in Table 2. The results in Table 2 show that removing the Transformer module from the MGTAF architecture significantly reduced all performance metrics. Specifically, the accuracy decreased from 94.7% to 84.2%, the F1 score from 95.4% to 78.2%, and the AUC also decreased substantially. These results demonstrate that the Transformer module is crucial for capturing long-range dependencies in brain networks; its absence prevents the model from effectively integrating global information, leading to a significant performance degradation. This also indicates that the hybrid architecture combining local feature extraction from GCNs with global dependency modeling from Transformers is key to achieving high model performance. Attached Figure Description

[0045] (1) Figure 1 This is the overall framework diagram of the MGTAF model. Compared with traditional methods, our method processes multiple connectivity patterns through parallel GCN-Transformer pathways and employs bidirectional attention and adaptive fusion mechanisms for information integration.

[0046] (2) Figure 2 This is a structural diagram of the hybrid GCN-Transformer module, showing the concatenated design of GCN layers for local feature extraction and edge enhancement, and Transformer layers for global dependency modeling.

[0047] (3) Figure 3 This is a schematic diagram of a multi-head self-attention mechanism that incorporates edge information, illustrating how the connection weights of the brain network can be integrated into attention calculation through the Hadamard product to achieve global dependency modeling guided by biological information.

[0048] (4) Figure 4 This is an architecture diagram of the Dual-Stream Adaptive Fusion (DSAF) module, depicting the parallel processing and adaptive fusion process of the main information stream (processing raw features) and the interactive information stream (processing interactive features).

[0049] (5) Figure 5The graph shows a comparison of the ROC curves of the MGTAF model with the baselines of each single connectivity mode. The results indicate that the AUC value of MGTAF is significantly better than any single mode, demonstrating the effectiveness of multi-mode fusion.

[0050] (6) Figure 6 The curve shows the evolution of the adaptive mixing parameter α in the MixPool layer during training, illustrating its convergence process for automatically learning balancing strategies for different connection modes.

[0051] (7) Figure 7 The results are visualizations of the node sensitivity analysis, showing the top 10 brain regions that contribute the most to ASD classification in the form of a 3D brain map, where the node size represents its sensitivity score.

Claims

1. A diagnostic method for autism spectrum disorder based on adaptive fusion of multiple graph Transformers, characterized in that, Includes the following steps: We acquired functional magnetic resonance imaging (fMRI) data from the subjects and constructed three types of functional brain networks: Pearson correlation network, sparse representation network, and Granger causal network. Each type of functional brain network is input into a graph convolutional network (GCN) to extract local topological features; The GCN output is input to the edge-enhanced graph Transformer, and long-range dependency features are modeled by introducing connection strength weights. For each connection pattern, the graph-level features are adaptively aggregated using a MixPool pooling layer to obtain the corresponding graph-level representation. A bidirectional attention mechanism is employed to enhance the interaction between features of different connection modes; The original features and interactive features are respectively input into the dual-stream adaptive fusion (DSAF) module, and adaptive feature fusion is completed through learnable gating parameters; The fused unified features are input into the classifier, which outputs the ASD diagnosis result. Construct a binary cross-entropy loss function and jointly optimize model parameters in an end-to-end manner.

2. The method according to claim 1, wherein, The three types of functional brain networks include: undirected networks based on Pearson correlation, sparse undirected networks based on sparse representation, and directed networks based on Granger causal mapping.

3. The method according to claim 1, wherein, The sparse representation network is constructed using the following optimization model: Where X is the brain region time series matrix and W is the sparse weight matrix.

4. The method according to claim 1, wherein, The attention calculation formula for the graph Transformer is as follows: in The connection weights between brain regions, For Hadama accumulation.

5. The method according to claim 1, wherein, The MixPool layer employs the following fusion strategy: in, These are learnable parameters.

6. The method according to claim 1, wherein, The bidirectional attention mechanism is based on the following calculation process: Used to generate interactive enhancement features between different connection modes.

7. The method according to claim 1, wherein, The dual-stream adaptive fusion module includes a main information stream and an interactive information stream, and fuses them in the following way: in These are learnable gating parameters.

8. The method according to claim 1, wherein, The classifier is a fully connected layer, a support vector machine (SVM), or another binary classification model.

9. A diagnostic system for autism spectrum disorder based on adaptive fusion multigraph Transformer, characterized in that, include: The brain network construction module is used to generate three types of functional brain networks: PC, SR, and GCM. The local feature extraction module is used to extract local connectivity features using a graph convolutional network. The global dependency modeling module captures long-range dependencies based on the edge-enhanced graph Transformer. An adaptive pooling module is used to generate graph-level feature representations using MixPool; The modal interaction module enables interaction based on three types of connection pattern features through a bidirectional attention mechanism; The adaptive fusion module fuses multi-source features based on learnable weights. The classification module is used to output ASD diagnostic results; An optimization module is used to train models based on binary cross-entropy loss.

10. The system according to claim 10, wherein, The system can be deployed on medical image analysis servers, edge computing devices, or cloud diagnostic platforms.

Images