Alzheimer disease classification method and system

Abstract

The invention discloses a method and a system for classifying Alzheimer's disease. The method comprises the following steps of: firstly, respectively preprocessing original data of three-dimensional structural magnetic resonance imaging (sMRI) and four-dimensional resting state functional magnetic resonance imaging (rs-fMRI); then, brain structure features are extracted through a three-dimensional visual Transform model fusing the dynamic combinable multi-head attention mechanism and a multi-layer feature fusion module, and space-time function features are extracted based on a SwiFT model; secondly, performing intra-modal sparse screening and inter-modal bidirectional interaction on the two features through a bidirectional sparse cross attention mechanism to obtain a fusion feature; and finally, outputting a classification probability result of the Alzheimer's disease patient, the mild cognitive impairment person or the healthy person based on the fusion features. According to the method, the accuracy and reliability of classification are improved through an advanced deep learning model and an efficient multi-modal feature fusion strategy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence classification technology, specifically to a method and system for classifying Alzheimer's disease. Background Technology

[0002] Alzheimer's disease (AD) is a common neurodegenerative disease characterized by cognitive decline, memory impairment, and structural abnormalities in the brain. Classification of AD is crucial for slowing disease progression and guiding individualized treatment. However, due to its complex and insidious pathological process, clinical diagnosis still relies on a comprehensive assessment of imaging, neuropsychological evaluation, and biomarkers. With the development of neuroimaging technology, magnetic resonance imaging (sMRI) has been widely used in AD research. Structural MRI reflects changes in brain region morphology and volume, while resting-state functional MRI (rs-fMRI) reveals the dynamic characteristics of brain functional networks. These two modalities describe brain lesions from structural and functional perspectives, respectively, providing complementary information for classification.

[0003] While both sMRI and fMRI have achieved significant results in AD research, a single modality cannot fully reflect the multi-layered pathological mechanisms of AD. sMRI only provides static anatomical information and struggles to capture dynamic interactions between brain regions; fMRI can reflect the temporal characteristics of neural activity, but it suffers from low signal-to-noise ratios and complex time series. More importantly, most current fMRI studies focus on indirect modeling based on time-series statistical indicators (such as functional connectivity, regional consistency, and low-frequency amplitude), while research directly utilizing raw four-dimensional fMRI volume data for end-to-end feature learning is relatively limited. This results in models failing to fully explore the spatiotemporal dynamic patterns of the whole brain and providing insufficient representation of early neural functional abnormalities in AD. Therefore, achieving deep integration of structural and functional information within a unified learning framework has become an important research direction for intelligent classification of AD.

[0004] In recent years, deep learning-based multimodal fusion methods have brought new opportunities for AD classification. Multimodal models can simultaneously utilize the complementary features of structural MRI and functional fMRI to achieve a global representation of brain abnormalities. However, existing research still faces the following three challenges: First, the feature distribution differences between modalities are significant, and direct fusion can easily lead to semantic imbalance and feature degradation. MRI and fMRI differ fundamentally in spatial resolution, temporal dimension, and signal characteristics. MRI emphasizes static structural morphology, while fMRI reflects temporal neural activity patterns. Traditional fusion strategies are mostly based on low-dimensional statistical indicators or feature splicing methods, lacking a unified modeling framework for high-dimensional image data. In particular, end-to-end deep fusion research on 3D MRI and 4D fMRI volume data is still relatively scarce, which limits the model's in-depth exploration of structure-function synergy. Second, attention modeling suffers from redundancy and inefficiency, making it difficult to accurately capture key brain region dependencies. Existing multi-head self-attention mechanisms often introduce excessive redundant computation in high-dimensional medical images, and the lack of dynamic coordination between attention heads makes it difficult for the model to focus on key regions and pathways related to AD. This not only increases computational overhead but also weakens the model's discriminative ability in structural feature modeling. Finally, modal interaction mechanisms are generally unidirectional and coarse-grained, making it difficult to achieve deep collaboration at the structure-function level. Most multimodal methods only achieve information transfer from one modality to another, lacking bidirectional, fine-grained feature interaction modeling. This makes it difficult for structural and functional features to be fully integrated in a common semantic space, failing to effectively characterize the intrinsic link between brain structural atrophy and neurological dysfunction in AD.

[0005] Therefore, providing a classification method and system for Alzheimer's disease to solve the above problems is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] To address the aforementioned technical problems, the present invention aims to provide an Alzheimer's disease classification method. This method designs a three-dimensional visual transformer model that combines a dynamic composable multi-head attention mechanism with a multi-layer feature fusion module. This model can significantly reduce the redundancy of attention calculations while ensuring global modeling capabilities. Furthermore, it enhances the information transmission between features at different levels through a multi-layer feature interaction mechanism, thereby improving the discriminativeness of feature representation. Simultaneously, a bidirectional sparse cross-attention mechanism is proposed to effectively improve the relevance and robustness of multimodal feature fusion.

[0007] The technical solution provided by this invention is as follows: A method for classifying Alzheimer's disease includes the following steps: The raw data of three-dimensional structural magnetic resonance imaging and the raw data of four-dimensional resting-state functional magnetic resonance imaging were preprocessed to obtain preprocessed three-dimensional structural magnetic resonance imaging data and preprocessed four-dimensional resting-state functional magnetic resonance imaging data. Brain structural features in the preprocessed three-dimensional structural magnetic resonance imaging data are extracted based on a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module. Spatiotemporal functional features were extracted from the preprocessed four-dimensional resting-state functional magnetic resonance imaging data based on the SwiFT model. Based on the bidirectional sparse cross-attention mechanism, intramodal sparse feature screening and intermodal bidirectional interaction are performed on the brain structural features and the spatiotemporal functional features to obtain fused features. Based on the fusion features, a classification result is output, which is a numerical feature vector of Alzheimer's disease probability, a numerical feature vector of mild cognitive impairment probability, or a numerical feature vector of health probability.

[0008] Preferably, the preprocessing of the raw three-dimensional structural magnetic resonance imaging data includes the following steps: Data processing tools were used to remove the skull structure from the brain image in the original three-dimensional structural magnetic resonance imaging data to obtain a skull-removed T1-weighted structural image; Register the T1-weighted structural images obtained from the craniotomy process to the corresponding functional image space; Within the functional image space, the craniotomized T1-weighted structural image is segmented into images of three tissue components: gray matter, white matter, and cerebrospinal fluid. The images of each segmented component are spatially normalized to MNI space to obtain standard data for three-dimensional structural magnetic resonance imaging. The standard three-dimensional structural magnetic resonance imaging data is resampled to a fixed voxel spacing and cropped to a fixed three-dimensional size to obtain the preprocessed three-dimensional structural magnetic resonance imaging data.

[0009] Preferably, the preprocessing of the raw data from four-dimensional resting-state functional magnetic resonance imaging includes the following steps: Data processing tools are used to remove image data at preset time points during the initial scanning phase of the original four-dimensional resting-state functional magnetic resonance imaging data to obtain denoised four-dimensional resting-state functional magnetic resonance imaging data. The four-dimensional resting-state functional magnetic resonance imaging (fMRI) denoised data were sequentially processed with time-slice correction, head motion correction, and head motion parameter regression to obtain four-dimensional resting-state functional magnetic resonance imaging (fMRI) corrected data. The four-dimensional resting-state functional magnetic resonance imaging correction data are spatially normalized to MNI space and then spatially smoothed to obtain four-dimensional resting-state functional magnetic resonance imaging standard data. Global normalization is performed on the non-background regions in the standard four-dimensional resting-state functional magnetic resonance imaging data, and the background regions are filled with preset voxel intensity values. The data is then cropped to a fixed three-dimensional size to obtain the preprocessed four-dimensional resting-state functional magnetic resonance imaging data.

[0010] Preferably, the step of extracting brain structural features from the preprocessed three-dimensional structural magnetic resonance imaging data based on a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module includes the following steps: The preprocessed 3D structural magnetic resonance imaging data is divided into patches by a 3D convolutional embedding layer, mapped to a high-dimensional feature space, and appended with learnable class label vectors to form an initial token sequence. The initial token sequence is input into a 3D vision transformer encoder consisting of several stacked encoding blocks; each encoding block includes layer normalization, a dynamically composable multi-head attention module, and a multilayer perceptron module; the dynamically composable multi-head attention module dynamically generates a combination coefficient matrix based on the input features to adaptively adjust the interaction relationship between multiple attention heads; Shallow, mid-level, and deep features are extracted from different levels of the 3D vision transformer encoder. The shallow and mid-level features are convolved, concatenated, and weighted and then fused with the deep features through the multi-layer feature fusion module to obtain the brain structure features.

[0011] Preferably, the step of dynamically generating a combination coefficient matrix based on input features to adaptively adjust the interaction relationship between multiple attention heads includes the following steps: A linear transformation is performed on the input features of the current layer of the encoded block to generate a query matrix, a key matrix, and a value matrix; The combination coefficient matrix is ​​calculated based on the preset dynamic generation network; Before Softmax normalization, the attention score matrix calculated from the query matrix and the key matrix is ​​pre-modulated using the combined coefficient matrix. After Softmax normalization, the normalized attention weight matrix is ​​modulated again using the combined coefficient matrix. The Concat function multiplies the attention weight matrix (after two modulations) with the value matrix and then concatenates it with a preset output projection matrix to output the result of the dynamically composable multi-head attention module.

[0012] Preferably, the combined coefficient matrix is ​​generated by a five-branch weighted combination structure, which includes: a static basic projection matrix branch, a low-rank mapping matrix branch generated from query features, a low-rank mapping matrix branch generated from key features, a rank-constrained dimension branch, and a dynamic gating weight branch.

[0013] Preferably, the extraction of spatiotemporal functional features from the preprocessed four-dimensional resting-state functional magnetic resonance imaging data based on the SwiFT model includes the following steps: The preprocessed four-dimensional resting-state functional magnetic resonance imaging data is divided into multiple fixed-length subsequences along the time dimension using a sliding window; Multiple subsequences are input as input features to the SwiFT model, which uses a four-stage hierarchical architecture for feature extraction and outputs the spatiotemporal functional features. The feature extraction is performed using a four-stage hierarchical architecture, specifically as follows: In the first stage, initial spatiotemporal features are obtained by performing tile embedding processing on the input features, and local spatiotemporal dependency modeling is performed through at least two four-dimensional Swing Transformer blocks; wherein, within each of the four-dimensional Swing Transformer blocks, the input features are updated alternately using a four-dimensional window multi-head self-attention mechanism and a four-dimensional shift window multi-head self-attention mechanism to achieve local window partitioning and cross-window information interaction. In the second and third stages, spatial resolution is reduced and the number of feature channels is increased by tile embedding and merging, respectively. Different numbers of four-dimensional Swin Transformer blocks are stacked in different stages to enhance representation capabilities. In the fourth stage, after obtaining deep spatiotemporal features, a global attention mechanism is introduced to integrate the contextual information of all spatiotemporal locations.

[0014] Preferably, the alternating use of a four-dimensional window multi-head self-attention mechanism and a four-dimensional shift window multi-head self-attention mechanism to update the input features includes the following steps: After performing layer normalization on the input features, the four-dimensional window multi-head self-attention is calculated using the first formula or the four-dimensional shift window multi-head self-attention is calculated using the second formula, and then residual connection is performed with the input features to obtain intermediate features; After performing layer normalization on the intermediate features, they are processed by a multilayer perceptron and then residually connected with the intermediate features to obtain the output features. The output feature is the updated input feature; The first formula is as follows: ; The second formula is as follows: ; The formula for calculating the output features is: ; or, ; in, For the first Output features of a layered four-dimensional Swing Transformer block For the first The intermediate representation of a layered four-dimensional SwinTransformer block. For four-dimensional window multi-head self-attention, For four-dimensional shift window multi-head self-attention, For layer normalization, It is a multilayer perceptron module.

[0015] Preferably, the step of performing intra-modal sparse feature screening and inter-modal bidirectional interaction on the brain structural features and the spatiotemporal functional features based on a bidirectional sparse cross-attention mechanism to obtain fused features includes the following steps: The brain structural features and the spatiotemporal multifunctional features are respectively mapped to the same fusion feature space through a learnable linear projection matrix to obtain aligned structural feature representations and functional feature representations; The aligned structural and functional feature representations are then processed using a dynamic sparse attention mechanism to filter key features within the modality, resulting in filtered structural and functional features. The dynamic sparse attention mechanism calculates a standard dense attention distribution and a distribution based on... The function generates a sparse distribution and adaptively weights and fuses the standard dense attention distribution and the sparse distribution with learnable sparsity weights to achieve a balance between redundancy suppression and information preservation. Based on the filtered structural features and filtered functional features, a bidirectional cross-attention mechanism is used for bidirectional interaction between modalities. Specifically, the filtered functional features are used as query vectors, and the filtered structural features are used as key vectors and value vectors to calculate the first-direction cross-attention, resulting in the structurally modulated functional features. The filtered structural features are used as query vectors, and the filtered functional features are used as key vectors and value vectors to calculate the second-direction cross-attention, resulting in the functionally modulated structural features. Pooling operations are performed on the functional features after structural modulation and the structural features after functional modulation to obtain pooled functional vectors and pooled structural vectors. The pooled functional vectors and pooled structural vectors are then concatenated to obtain the fused features.

[0016] An Alzheimer's disease classification system includes: The preprocessing module is used to preprocess the raw data of three-dimensional structural magnetic resonance imaging and the raw data of four-dimensional resting-state functional magnetic resonance imaging, respectively, to obtain preprocessed three-dimensional structural magnetic resonance imaging data and preprocessed four-dimensional resting-state functional magnetic resonance imaging data. The brain structure feature extraction module is used to extract brain structure features from the preprocessed three-dimensional structural magnetic resonance imaging data based on a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module. The spatiotemporal functional feature extraction module is used to extract spatiotemporal functional features from the preprocessed four-dimensional resting-state functional magnetic resonance imaging data based on the SwiFT model. The feature fusion module is used to perform intramodal sparse feature screening and intermodal bidirectional interaction on the brain structural features and the spatiotemporal functional features based on a bidirectional sparse cross-attention mechanism to obtain fused features. The result output module is used to output a classification result based on the fusion features. The classification result is a numerical feature vector of Alzheimer's disease probability, a numerical feature vector of mild cognitive impairment probability, or a numerical feature vector of health probability.

[0017] This invention discloses a classification method for Alzheimer's disease. It involves preprocessing raw data from three-dimensional structural magnetic resonance imaging (sMRI) and four-dimensional resting-state functional magnetic resonance imaging (fMRI); constructing a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module; extracting brain structural features from sMRI using this model; extracting spatiotemporal functional features from fMRI using a SwiFT model; performing feature selection and fusion of brain structural and spatiotemporal functional features based on a bidirectional sparse cross-attention mechanism to obtain fused features; and outputting classification results using these fused features.

[0018] The Alzheimer's disease classification method disclosed in this invention has the following beneficial effects: Higher classification accuracy and robustness: By deeply integrating the two most complementary neuroimaging modalities and optimizing their respective features before fusion, it is expected to significantly outperform single-modal methods and traditional multimodal fusion baseline models in the three-class classification task of AD, mild cognitive impairment (MCI), and normal control NC.

[0019] Enhanced classification capabilities: MCI is an early stage of AD, characterized by subtle changes. In this scheme, the Dynamically Composable Multi-Head Attention (DCMHA) mechanism sensitively captures subtle structural changes, SwiFT uncovers early functional network disorder, and the Bidirectional Sparse Cross-Attention Fusion (BSCAF) module strengthens weak cross-modal associations, collectively improving the model's sensitivity to MCI recognition.

[0020] Enhanced model interpretability: DCMHA's attention weights, BSCAF's sparse filtering, and cross-attention maps can visualize the key brain regions (structures) and spatiotemporal patterns (functions) that the model focuses on when making decisions, thus enhancing the transparency and credibility of AI classification.

[0021] This invention also discloses an Alzheimer's disease classification system, which, since it belongs to the same technical concept and solves the same technical problem as this method, should have the same beneficial effects, and will not be described in detail here. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an Alzheimer's disease classification method provided in an embodiment of the present invention; Figure 2 This is a flowchart of preprocessing raw data from three-dimensional structural magnetic resonance imaging provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the preprocessing of raw data from four-dimensional resting-state functional magnetic resonance imaging, as provided in an embodiment of the present invention. Figure 4 This is a flowchart of step S2 provided in an embodiment of the present invention; Figure 5 This is a flowchart illustrating the dynamic combination and modulation of dynamically composable multi-head attention provided in an embodiment of the present invention; Figure 6 This is a flowchart of step S4 provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an Alzheimer's disease classification system provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] The embodiments of this invention are written in a progressive manner.

[0026] This invention provides a method and system for classifying Alzheimer's disease. It primarily addresses the technical problems in existing technologies, such as the inability of a single modality to comprehensively characterize the structural and functional abnormalities of AD, and the shortcomings of existing multimodal fusion methods including differences in feature distribution, attention redundancy, and insufficient modal interaction.

[0027] like Figure 1 As shown, an Alzheimer's disease classification method includes the following steps: S1. Preprocess the raw data of three-dimensional structural magnetic resonance imaging and the raw data of four-dimensional resting-state functional magnetic resonance imaging respectively to obtain preprocessed three-dimensional structural magnetic resonance imaging data and preprocessed four-dimensional resting-state functional magnetic resonance imaging data. S2. Brain structural features in preprocessed three-dimensional structural magnetic resonance imaging data are extracted based on a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module. S3. Spatiotemporal functional features are extracted from preprocessed four-dimensional resting-state functional magnetic resonance imaging data based on the SwiFT model; S4. Based on the bidirectional sparse cross-attention mechanism, intramodal sparse feature screening and intermodal bidirectional interaction are performed on brain structural features and spatiotemporal functional features to obtain fused features; S5. Based on the fusion features, output the classification results, which are numerical feature vectors of Alzheimer's disease probability, numerical feature vectors of mild cognitive impairment probability, or numerical feature vectors of health probability.

[0028] Step S1 involves three-dimensional structural magnetic resonance imaging (fMRI), a medical imaging technique based on the principles of nuclear magnetic resonance (NMR) that non-invasively acquires high-resolution three-dimensional anatomical images of human tissues such as the brain. It generates static three-dimensional volumetric data with excellent soft tissue contrast by measuring differences in physical properties such as the relaxation time of hydrogen protons in tissues. This data is primarily used to accurately display and quantify changes in the morphology, volume, and tissue composition (such as gray and white matter) of brain regions, making it a key tool for the diagnosis and research of neurodegenerative diseases. Four-dimensional resting-state functional magnetic resonance imaging (fMRI) is a functional imaging technique used non-invasively to detect dynamic changes in spontaneous neural activity in the brain. It continuously acquires three-dimensional images of the brain under resting conditions, generated by the blood oxygen level-dependent effect, forming four-dimensional volumetric data of "three-dimensional space + one-dimensional time," thereby revealing the synchronization and fluctuation characteristics of the whole brain functional network in the time dimension, providing key dynamic functional information for understanding the mechanisms of brain diseases; Preprocessing refers to a series of standardized calculations and corrections performed on the original image data (such as MRI, fMRI) before formal analysis or modeling, in order to eliminate or reduce non-biological variations, technical artifacts, and data inconsistencies, and to standardize and make them comparable. The dynamic composable multi-head attention mechanism in step S2 is an improved attention computation module designed to optimize the standard multi-head self-attention mechanism. It breaks down the independence between multiple attention heads, reduces computational redundancy, and enhances the model's ability to model complex dependencies by dynamically generating and combining the weight distributions of different attention heads based on the current input features. The multi-layer feature fusion module is a feature integration mechanism for deep learning models, designed to systematically aggregate and fuse features extracted from different depth levels (such as shallow, medium, and deep layers) of the neural network, which have different semantic granularities and receptive fields. The 3D vision Transformer model is a deep learning model based on the Transformer architecture, specifically designed for processing 3D volumetric data (such as medical images and 3D scans). It extends and adapts successful self-attention mechanisms in natural language processing and 2D images to 3D space to capture long-distance, global spatial dependencies in volumetric data. Brain structural features refer to computable measures extracted from structural neuroimaging (mainly structural magnetic resonance imaging, sMRI) used to quantify, describe, and characterize the brain's anatomical morphology, tissue composition, and volumetric spatial properties. The SwiFT model in step S3 is a deep learning model specifically designed for four-dimensional functional magnetic resonance imaging (fMRI) data; its full name is Swin 4D fMRI Transformer. The core innovation of this model lies in extending the SwiFT architecture to the four-dimensional spatiotemporal domain, enabling end-to-end spatiotemporal joint feature learning directly from the raw fMRI volume data. Spatiotemporal functional features refer to quantitative information extracted from four-dimensional functional neuroimaging data that simultaneously characterizes the spatial distribution patterns and temporal evolution of brain neural activity. The bidirectional sparse cross-attention mechanism in step S4 is an attention architecture for deep fusion of multimodal data. Its core lies in achieving precise and efficient complementary fusion of features from different modalities through a combination of sparse constraints and bidirectional interaction, including intra-modal sparse definition and inter-modal bidirectional cross-attention. Steps S1 to S5 detail the implementation of the Alzheimer's disease classification method. In the 3D structural magnetic resonance imaging (sMRI) branch, a 3D Vision Transformer is constructed, combining a Dynamic Composable Multi-Head Attention (DCMHA) mechanism with a Multi-level Feature Fusion Module (MFFM). The DCMHA mechanism reduces redundancy in multi-head attention and enhances inter-head interaction, thereby improving the representational ability of structural features. The MFFM module strengthens the consistency between local structure and global semantics through multi-level feature aggregation. The fMRI branch uses a SwiFT model to directly model the spatiotemporal dependence of 4D resting-state functional magnetic resonance imaging (fMRI) to fully explore the dynamic patterns of neural function. Furthermore, a Bidirectional Sparse Cross-Attention Fusion (BSCAF) module is designed to achieve intra-modal sparse feature selection and bidirectional inter-modal interaction. This module can highlight disease-related dynamic connectivity features in the functional modality and enhance the spatial representation of lesion-sensitive areas in the structural modality, thereby achieving complementary fusion of information at the structure-function level. Finally, the fused multimodal features are input into the classifier to achieve the discrimination results of AD, mild cognitive impairment (MCI), and normal controls (NC) as classification reference content.

[0029] The proposed scheme employs a 3DVision Transformer encoder that combines a dynamic, composable multi-head attention mechanism with a multi-layer feature fusion module to efficiently model the spatial structure and spatiotemporal dependencies in MRI and fMRI volumetric data. This structure significantly reduces the redundancy of attention computation while maintaining global modeling capabilities, and enhances information transfer between features at different levels through a multi-layer feature interaction mechanism, thereby improving the discriminativeness of feature representation. To fully exploit the complementary information between MRI and fMRI modalities, a bidirectional sparse cross-attention mechanism is proposed. Within a modality, sparse attention enables feature selection and redundancy suppression, while between modalities, a bidirectional interaction mechanism enables collaborative modeling of structural and functional features, effectively improving the relevance and robustness of multimodal feature fusion and providing a more discriminative fusion representation for subsequent classification tasks. Based on these modules, an end-to-end 3D sMRI–4D fMRI joint analysis framework is constructed, achieving efficient fusion and joint optimization of multimodal features on the ADNI dataset. This framework can simultaneously model brain structural and functional features, effectively improving the classification performance of Alzheimer's disease (AD) and providing a new scalable solution for intelligent classification of multimodal neuroimaging.

[0030] In one embodiment, at the input level, sMRI uses T1-weighted (T1w) structural images, while fMRI uses raw 4D high-dimensional data with dimensions (H, W, D, T), where T represents the time frame number and (H, W, D) is the three-dimensional spatial resolution. This 4D data can simultaneously provide spatial structural and temporal dynamic information of brain regions. During the feature extraction stage, the sMRI channel uses a 3DVision Transformer as the basic encoder and introduces a dynamically composable multi-head attention mechanism and a multi-layer feature fusion module to enhance the modeling ability of key structural regions. The fMRI channel uses a SwiFT model, directly applied to the 4D high-dimensional data input to learn the spatiotemporal representation of the whole-brain functional network. Notably, the fMRI encoder maintains updatable parameters during training to ensure sufficient adaptation of temporal features. Subsequently, the features from the two modalities interact through a designed sparse cross-attention fusion module, achieving complementarity at the structure-function level. The fused features are finally input to a fully connected classifier, achieving AD classification through a two-layer perceptron and Dropout regularization.

[0031] like Figure 2 As shown, preferably, the preprocessing of the raw three-dimensional structural magnetic resonance imaging data includes the following steps: A1. Using data processing tools, the skull structure of the brain image in the original three-dimensional structural magnetic resonance imaging data was removed to obtain a skull-removed T1-weighted structural image; A2. Register the craniotomized T1-weighted structural images to the corresponding functional image space; A3. In the functional image space, the craniotomized T1-weighted structural image is segmented into images of three tissue components: gray matter, white matter, and cerebrospinal fluid; A4. The images of each segmented component are spatially normalized to MNI space to obtain standard data for three-dimensional structural magnetic resonance imaging; A5. Resample the standard data of three-dimensional structural magnetic resonance imaging to a fixed voxel spacing and crop it to a fixed three-dimensional size to obtain the preprocessed three-dimensional structural magnetic resonance imaging data.

[0032] The T1-weighted structural image in step A1 is a high-contrast anatomical image in magnetic resonance imaging (MRI) that primarily reflects differences in T1 relaxation time among tissues through specific scanning sequence parameter settings. Its core feature is the ability to clearly distinguish major brain tissue components such as gray matter, white matter, and cerebrospinal fluid. In neuroimaging analysis, T1-weighted images are considered the "gold standard" structural reference image because they provide the highest resolution static anatomical detail and are commonly used for precise brain region segmentation, volume measurement, cortical thickness analysis, and as an anatomical template for spatial registration with other functional imaging modalities (such as fMRI). Registration in step A2 refers to the process of geometrically aligning two or more medical images acquired at different times, with different devices, or under different modalities through spatial transformation. Its core objective is to achieve optimal spatial matching of corresponding anatomical structures or functional regions within the images. The essence of registration is finding an optimal spatial transformation model that maps the coordinate system of one image (called the floating image) to the coordinate system of another image (called the reference image), thereby eliminating spatial inconsistencies caused by factors such as scanning posture, device differences, or individual anatomical variations. Functional image space refers to the inherent, original three-dimensional spatial coordinate system of functional magnetic resonance imaging (fMRI) data, particularly blood oxygen level dependent (BOLD) signals. This coordinate system is directly defined by the fMRI scanner during acquisition, and its coordinate axes are aligned with the scanner's own physical orientation (e.g., left-right, front-back, head-to-toe). Spatial normalization in step A4 is a crucial medical image preprocessing technique. Specifically, it refers to the process of registering and transforming brain images (structural or functional) from different individuals into a standard, universal three-dimensional brain template space (such as the MNI template space of the Montreal Neuroscience Institute) using a nonlinear spatial transformation algorithm. Its core purpose is to eliminate anatomical differences in shape, size, and location between individuals' brains, enabling image data from different subjects and scanning sites to be compared, analyzed, and statistically analyzed within the same spatial coordinate framework. The MNI space is an internationally recognized standard three-dimensional brain coordinate system and its corresponding average brain anatomical template, established and maintained by the Montreal Neuroscience Institute. It is the most fundamental spatial reference framework in neuroimaging research. Resampling in step A5 refers to the process of changing the spatial resolution or voxel spacing of medical image data. That is, by using an interpolation algorithm to estimate new voxel values ​​on the discrete voxel grid of the original image, a new image with a different spatial sampling frequency is generated. Steps A1 to A5 are the specific implementation details of the preprocessing of the raw three-dimensional structural magnetic resonance imaging (sMRI) data in step S1. This involves using data processing tools to remove the skull structure from the brain image in the raw sMRI data, obtaining a skull-removed T1-weighted structural image. This T1-weighted structural image is then registered to the functional image space, where it is segmented into images of three tissue components: gray matter, white matter, and cerebrospinal fluid. Each tissue component image is then spatially normalized to the MNI space to obtain standard sMRI data. This standard sMRI data is then resampled to a fixed voxel spacing and cropped to a fixed three-dimensional size, resulting in preprocessed sMRI data.

[0033] Specifically: Step A1 removes non-brain tissues (skull, scalp, dura mater) to obtain clean brain tissue images, eliminating the interference of high-intensity signals from non-brain tissues on image analysis and model learning; Step A2 aligns structural images (sMRI) and functional images (fMRI) in geometric space, establishing a precise anatomical-functional correspondence for subsequent multimodal fusion, ensuring that structural and functional features extracted from the same brain region are spatially perfectly matched; Step A3, based on image signal intensity, classifies brain tissue into gray matter, white matter, and cerebrospinal fluid, obtaining tissue-specific images. Alzheimer's disease primarily affects gray matter, and this step can initially focus the model's attention on relevant regions; Step A4 nonlinearly deforms individual brains to a standard template to eliminate anatomical variations in brain shape, size, and location between individuals, allowing comparison of all subjects' brains in the same standard coordinate system; Step A5 unifies the spatial resolution and size of all data, obtaining input tensors with completely consistent dimensions. The above scheme systematically transforms the raw, heterogeneous, and noisy three-dimensional structural magnetic resonance imaging data into a standardized three-dimensional data set with uniform specifications, precise anatomical location correspondence, pure tissue composition, and removal of non-brain interference.

[0034] In one embodiment, step A1 involves using the DPABI tool for data preprocessing. DPABI is an integrated, MATLAB-based toolbox for processing and analyzing neuroimaging data. This tool aims to provide researchers with a one-stop, standardized workflow to automate or semi-automatically process multimodal brain imaging data (especially functional magnetic resonance imaging, fMRI) from preprocessing to statistical analysis. In step A5, the standard sMRI data is resampled to the same voxel spacing (1.75×1.75×1.75), then cropped, and the 3D volume size is adjusted to 128×128×128 to obtain the preprocessed sMRI.

[0035] like Figure 3 As shown, preferably, the preprocessing of the raw data from four-dimensional resting-state functional magnetic resonance imaging includes the following steps: B1. Data processing tools are used to remove image data at preset time points in the initial scanning stage of the original data of four-dimensional resting-state functional magnetic resonance imaging to obtain denoised four-dimensional resting-state functional magnetic resonance imaging data. B2. The four-dimensional resting-state functional magnetic resonance imaging (fMRI) denoised data were sequentially processed with time-slice correction, head motion correction, and head motion parameter regression to obtain the four-dimensional resting-state functional magnetic resonance imaging (fMRI) corrected data. B3. Spatially normalize the four-dimensional resting-state functional magnetic resonance imaging correction data to MNI space and perform spatial smoothing to obtain four-dimensional resting-state functional magnetic resonance imaging standard data. B4. Globally normalize the non-background regions in the standard four-dimensional resting-state functional magnetic resonance imaging data, fill the background regions with preset voxel intensity values, and crop them to a fixed three-dimensional size to obtain the preprocessed four-dimensional resting-state functional magnetic resonance imaging data.

[0036] Step B2, time-slice correction, is a crucial step in fMRI data preprocessing, specifically designed to compensate for minor temporal misalignments in images from different layers due to layered acquisition by the MRI equipment. It aligns signals from different layers at the same time point in the temporal dimension, ensuring that data from all brain layers represent the neural activity state at the same instant, thereby eliminating temporal artifacts caused by the acquisition order and ensuring the accuracy of subsequent temporal analyses (such as functional connectivity calculations). Head motion correction is a key technique in fMRI data preprocessing for eliminating head motion artifacts. This process calculates and compensates for the three-dimensional translation and rotation parameters of each time-point image relative to a reference image (usually the first time point), realigning all time-series images to the same spatial location. This ensures that changes in the BOLD signal of neural activity reflect real physiological processes, rather than spatial misalignments caused by head movements. Head motion parameter regression processing is a statistical step in fMRI data preprocessing, designed to directly remove the signal components represented by translation and rotation parameters estimated by head movements from the time series of BOLD signals. This process further suppresses the residual effects of head movements on functional connectivity and activation analysis at the signal level by performing linear regression on the head motion parameters (typically including 6: 3 translations and 3 rotations) of the time series of each voxel and retaining the residual signal as the “cleaned” neural activity signal. The spatial smoothing process in step B3 is a spatial filtering operation for three-dimensional medical images (especially fMRI and PET). It usually uses a three-dimensional Gaussian kernel to convolve the image, aiming to achieve the following effects by averaging the signals of neighboring voxels: reducing high-frequency random noise in the image, improving the signal-to-noise ratio, compensating for spatial mismatch caused by subtle anatomical differences between individuals, improving the statistical power of population analysis, and making the data distribution more consistent with the premise assumptions of certain statistical inference methods (such as random field theory). Global normalization (or global normalization) in step B4 is a standardization technique in medical image analysis, which refers to mapping the signal intensity of all non-background voxels (i.e., effective tissue areas) in the entire three-dimensional or four-dimensional image data to a unified and comparable numerical range. Steps B1 to B4 are the specific implementation details of the preprocessing of the raw four-dimensional resting-state functional magnetic resonance imaging (fMRI) data in step S1. This involves removing noise from the initial scanning stage of the raw fMRI data using data processing tools. The denoised fMRI data is then subjected to preprocessing operations such as time-slice correction, head motion correction, and head motion parameter regression to obtain corrected fMRI data. Similarly, the corrected fMRI data is spatially standardized to the MNI space and spatially smoothed to obtain standard fMRI data. Global normalization is then performed on the area outside the background region of the standard fMRI data, and the background region is filled with preset voxel intensity values. Finally, the data is cropped to a fixed three-dimensional size to obtain the preprocessed fMRI data.

[0037] Specifically, the process involves: discarding images from the initial scan period when the gradient magnetic field is unstable and the subject has not yet adapted, based on step B1, to eliminate high-intensity transient noise at the beginning of the sequence and obtain a more stable initial time series with a higher signal-to-noise ratio; compensating for time differences in layered acquisition based on step B2 time-slice correction; head movement correction to spatially align images misaligned due to head movement; and head movement parameter regression to remove motion-related components from the signal, thus ensuring spatiotemporal consistency: signals from all brain regions are comparable at the same time (time-slice correction); the anatomical position of the brain is fixed at all time points (head movement correction); and motion-related artifacts in the BOLD signal are removed to the greatest extent possible (head movement regression); based on step B3 spatial... Inter-regional standardization registers individual brains to a standard template; spatial smoothing uses a Gaussian kernel to blur images, improving group comparability and signal-to-noise ratio; standardization ensures that the same functional brain regions (such as the posterior cingulate cortex) in different subjects are located consistently in the MNI space; smoothing suppresses subtle differences in individual anatomy and random noise, improving signal stability; global normalization based on step B4 standardizes the whole-brain signal intensity to a uniform scale; background filling marks non-brain regions with fixed values ​​(such as minimum intensity values); cropping unifies spatial dimensions, achieving a uniform data scale and format; eliminating overall signal intensity differences caused by scanning gain, etc.; clearly distinguishing brain regions from non-brain regions; generating 4D input tensors with completely consistent dimensions.

[0038] The above scheme gradually transforms the original 4D fMRI time-series data into a standardized signal that is spatiotemporally aligned, effectively suppresses motion artifacts and noise, and allows for direct comparison among the functions of all subjects, truly reflecting the spatiotemporal patterns of spontaneous neural activity in the brain.

[0039] In one embodiment, the DPABI tool is also used for data preprocessing in step B1. Since the gradient magnetic field needs time to stabilize and the subject needs time to adapt during MRI scanning, the images at the first few time points have more noise. Therefore, the data from the first 10 time points of the original fMRI data are removed. In step B4, for each 4D fMRI standard data, the brain image is globally normalized in four dimensions except for the background region. Then, the background is filled with the minimum voxel intensity value. The 3D volume is adjusted to 96×96×96 by cropping and filling the background to obtain the preprocessed fMRI data.

[0040] like Figure 4 As shown, preferably, the brain structural features in the preprocessed three-dimensional structural magnetic resonance imaging data are extracted based on a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module, including the following steps: C1. The preprocessed 3D structural magnetic resonance imaging data is divided into patches by a 3D convolutional embedding layer and mapped to a high-dimensional feature space, and a learnable class label vector is attached to form an initial token sequence; C2. Input the initial token sequence into a 3D vision transformer encoder consisting of several stacked encoding blocks; wherein, each encoding block includes layer normalization, a dynamically composable multi-head attention module and a multilayer perceptron module; the dynamically composable multi-head attention module dynamically generates a combination coefficient matrix according to the input features to adaptively adjust the interaction relationship between multiple attention heads; C3. Shallow, mid-level and deep features are extracted from different levels of the 3D vision transformer encoder. Through the multi-layer feature fusion module, the shallow and mid-level features are convolved, concatenated and weighted and then fused. Finally, residual connections are made with the deep features to obtain brain structure features.

[0041] The initial token sequence in step C1 refers to the first-stage feature representation obtained after the original input image is initially encoded and serialized when processing visual data using a Transformer-based model (such as Vision Transformer). The 3D vision Transformer encoder in step C2 is a core neural network module that is based on the standard Transformer architecture with dimensional extensions and is specifically designed to process 3D volumetric data (such as 3D medical images and 3D point clouds). Convolution in step C3 is a core mathematical operation in signal processing and deep learning. In deep learning, it specifically refers to the convolution operation, which involves sliding a filter across the input data and calculating the dot product (weighted sum) between the filter and a local region of the input data at each position to extract local features and generate new feature representations. Residual connections are a key structural design in deep neural networks. The core idea is to directly bypass the layer (or some of its internal operations) and add the input of a layer to the output of that layer. Steps C1 to C3 are the specific implementation details of step S2. The preprocessed sMRI is divided into patches by a three-dimensional convolutional embedding layer and embedded and mapped to a high-dimensional feature space. Learnable class label vectors are attached to form an initial token sequence. The initial token sequence is input into a three-dimensional visual transformer encoder composed of several stacked coding blocks to extract shallow, medium and deep features. The shallow and medium features are convolved, concatenated and weighted by a multi-layer feature fusion module and then residually connected with the deep features to obtain brain structure features.

[0042] Specifically: Based on step C1, 3D volumetric data is transformed into a sequence form that Transformer can process, and learnable classification labels are introduced to establish a serialized representation: unstructured voxel grids are transformed into structured token sequences, laying the foundation for global attention modeling. Among them, the category label serves as a summary of the entire sequence and is ultimately used for classification. Based on step C2, global contextual dependencies are learned through stacked encoding blocks using an improved self-attention mechanism to achieve global relation modeling: any two brain region map embeddings (regardless of distance) can interact directly, capturing structural covariance patterns across the entire brain and dynamically reducing redundancy: DCMHA suppresses repetitive attention patterns that are useless for AD discrimination by dynamically combining attention heads, allowing model resources to focus more on the dependencies between key pathology-related regions. This improves computational efficiency and discriminative density of features. Based on step C3, features extracted from different depths of the encoder with different semantic levels are fused to achieve multi-scale information complementarity: shallow / medium-level features contain rich spatial details and local morphological information, while deep features contain high-level global semantics and contextual information. By fusing them through convolution, attention, and residual connections, the resulting features retain the subtle anatomical details required for classification and are constrained and interpreted by high-level semantics, avoiding the loss of spatial information caused by excessive abstraction of deep features.

[0043] The above-mentioned approach involves a series of in-depth customizations tailored to 3D brain structural imaging and the characteristics of Alzheimer's disease. It achieves efficient and accurate global modeling through DCMHA, and then elegantly synthesizes multi-level information through MFFM, ultimately outputting a brain structural feature with high discriminative power and high information density.

[0044] like Figure 5 As shown, preferably, the combination coefficient matrix is ​​dynamically generated based on the input features to adaptively adjust the interaction relationship between multiple attention heads, including the following steps: D1. Perform a linear transformation on the input features of the current layer of the coded block to generate a query matrix, a key matrix, and a value matrix; D2. Calculate the combination coefficient matrix based on the preset dynamic generation network; D3. Before Softmax normalization, the attention score matrix calculated from the query matrix and the key matrix is ​​pre-modulated by combining the coefficient matrix; D4. After Softmax normalization, the normalized attention weight matrix is ​​modulated again by combining the coefficient matrix. D5. The Concat function multiplies the attention weight matrix and value matrix after two modulations and then connects them with the preset output projection matrix to output the result of the dynamically combinable multi-head attention module.

[0045] The combination coefficient matrix in step D2 is the core dynamic parameter component of the dynamic composable multi-head attention mechanism in this invention. Its function is to adaptively calculate a set of weights based on the current input features, so as to modulate and combine the outputs of multiple attention heads. The attention score matrix in step D3 is an intermediate result matrix used in the self-attention or cross-attention mechanism calculation process to quantify the strength of all pairing associations between the query vector sequence and the key vector sequence. The attention weight matrix in step D4 is the core computational matrix in the attention mechanism, representing the normalized attention level of each element in the query sequence to all elements in the key-value sequence. This matrix is ​​obtained by row-normalizing the attention score matrix using the Softmax function. Each element's value is between 0 and 1, and the sum of the elements in each row is 1. Therefore, it can be directly used as a weighting coefficient to perform a weighted summation on the value vector sequence to generate the final attention output. Steps D1 to D5 are the specific implementation details of the dynamic combination and modulation of the dynamic composable multi-head attention in step C2. It involves performing a linear transformation on the input features in the current layer of the coding block to generate a query matrix, a key matrix, and a value matrix, and calculating the combination coefficient matrix based on the dynamically generated network. Before softmax normalization, the attention score matrix is ​​pre-modulated, and after softmax normalization, the attention weight matrix is ​​post-modulated. The Concat function is used to multiply the attention weights after the two modulations with the value matrix and then concatenate them with the output projection matrix to output the result of the dynamic composable multi-head attention module.

[0046] Specifically: Step D1 projects the input features into three subspaces: query, key, and value, preparing for attention computation. This is a standard pre-operation for all attention mechanisms, decomposing high-dimensional features into components for matching (Q / K) and aggregation (V). Step D2 calculates the weights for modulating multi-head attention in real time using the current input features, achieving the core of "dynamic" and "composable" characteristics. This coefficient matrix is ​​not a fixed parameter but is content-adaptive. Step D3, before Softmax normalization, performs cross-head adjustment on the original association strength (score) to exert influence before the formation of the attention distribution. This is equivalent to directly adjusting the original association probability between different brain region patch embeddings. Step D4, after Softmax normalization, readjusts the formed, probabilistic attention distribution to fine-tune and recalibrate the focused attention. Even after pre-modulation and Softmax, the final attention distribution may still have slight deviations. Post-modulation provides a second opportunity to correct these distributions; based on step D5, the modulated attention weights are applied to the value matrix, and the final result is output through projection, i.e., generating contextual features that have been dynamically combined and modulated.

[0047] In the above scheme, the DCMHA module breaks through the limitations of independent attention heads and computational redundancy in standard multi-head attention by using a "two-time dynamic modulation" mechanism. It achieves adaptive reshaping of attention patterns based on the input brain image content, thereby generating more focused, efficient, and discriminative feature representations.

[0048] Preferably, the combination coefficient matrix is ​​generated by a five-branch weighted combination structure, which includes: a static basic projection matrix branch, a low-rank mapping matrix branch generated from query features, a low-rank mapping matrix branch generated from key features, a rank-constrained dimension branch, and a dynamic gating weight branch.

[0049] In practical applications, the combination coefficient matrix includes: a static basic projection matrix branch: a learnable but fixed weight matrix that does not change with input, providing a stable attention modulation baseline. It encodes the universally applicable cross-head combination prior knowledge learned by the model from training data, preventing drastic fluctuations in the dynamic generation process due to input noise; a low-rank mapping matrix branch generated from query features: dynamically generating a low-rank matrix from query (Q) features, providing a modulation signal from the perspective of the "information seeker". Since Q represents the current feature location to be updated (such as a brain region), this branch allows the combination coefficients to reflect the needs of the "current focus", and the low-rank design significantly reduces parameters; a low-rank mapping matrix branch generated from key features: dynamically generating a low-rank matrix from key (K) features, providing a modulation signal from the perspective of the "retrieved information source". Since K represents all associatable features (other areas of the whole brain), this branch allows the combination coefficients to reflect the supply of the "overall context"; and a rank-constrained dimension branch: the rank of the above low-rank mapping ( Explicit control is implemented to strictly limit the expressive power and number of parameters of dynamic branches. This is achieved by limiting the size of the rank ( <<Feature Dimensions>> force the model to learn the most core and compact modulation patterns, avoiding overfitting and greatly reducing computational overhead; Dynamically Gated Weight Branches: scalar or vector weights generated from the network's output act as global or fine-grained regulators, adaptively weighting or gates the contributions of other branches. It determines whether, given the current input, it relies more on static priors or dynamic information from Q or K; The five-branch structure is not a simple fully connected layer, but a carefully designed, modular feature fusion architecture. It generates the final modulation coefficients by combining different types of information sources, ensuring the coefficient matrix is ​​both stable and reliable, flexible and adaptable, and computationally efficient, thus providing high-quality control signals for the two dynamic modulations of DCMHA.

[0050] In one embodiment, the 3D vision transformer model consists of a 3D patch embedding layer, several stacked DCMHA Transformer coding blocks, and a multi-layer feature fusion module. Suppose the input 3D MRI volume data is ,in, These represent the dimensions of the volume data in three spatial dimensions, with one channel. First, the model divides the input voxels into volume patches and maps them to a high-dimensional feature space using a 3D convolutional embedding layer (Patch Embedding), obtaining an initial token sequence: ; in, For the initial token sequence, This indicates the side length of the patch, with the kernel size and stride both set to [value]. , The embedding dimension for each patch. To preserve spatial location information, the model incorporates learnable location encodings into the embedding features. Furthermore, a learnable class token vector is appended to the beginning of the sequence. The initial input representation is obtained as follows: ; The sequence undergoes feature extraction via multi-layer Transformer encoder blocks. Each encoder block consists of layer normalization, a DCMHA attention sublayer, and a multilayer perceptron (MLP) sublayer, and includes residual connections. Its computational form is as follows: ; in, Indicates the first The output features of the layer For layer normalization operation, It is a multilayer sensor module; In each DCMHA module, the input features are transformed linearly to obtain the query matrix, key matrix, and value matrix. When there are n attention heads in a multi-head attention process, for the ... The attention score matrix for each attention head is calculated as follows: ; in, = ; Traditional multi-head attention computes scores independently from different attention heads and then concatenates them, while DCMHA breaks down head independence by dynamically combining scores across heads at the attention matrix level. Specifically, it performs a score matrix output by each attention head. Define dynamic combination functions Linear mixing is performed under input-dependent weight control: ; in, For the first Attention vectors of attention heads, combined coefficient matrix The query and key features are adaptively calculated by a dynamic weight generator to reflect the correlation between different attention heads. To improve parameter efficiency and enhance expressive power, The weighted combination form decomposed into five branches: ; in, For the updated attention vector, For attention vectors, For static basic projection matrix; and This is a low-rank mapping matrix generated from the query and the key. For rank-constrained dimensions; These are dynamically gating weights used to adjust the effectiveness of different attention heads. All of the above weights are generated by a dynamically generated network. It is calculated from the input features, that is: ; DCMHA applies this dynamic combination mechanism in both stages of attention: first, before Softmax normalization, it optimizes the attention score matrix... First, pre-modulation is performed; second, the attention weight matrix is ​​adjusted after Softmax normalization. Post-modulation is performed to obtain the final multi-head attention output, which can be expressed as follows: ; ; ; in, This is the output projection matrix. Through this input-dependent cross-head combination mechanism, the model can adaptively adjust the interaction between attention heads, thereby achieving more flexible and higher-rank feature dependency modeling in 3D space.

[0051] After stacking several Transformer blocks, the latent features extracted by the model from different levels are denoted as follows: , and Shallow features contain more spatial details, while deep features contain richer semantic context. To fully utilize spatial information at different levels while maintaining global semantic consistency, a multi-layer feature fusion module is further introduced to integrate multi-scale features across the layers of the 3D Vision Transformer.

[0052] Let the number of tokens output at each level be... , To determine the batch size, it can be divided into... Remapping back to a 3D patch feature map: ; Represents a 3D Patch feature map. The spatial dimensions of the restored 3D feature map are defined. After dimensionality reduction via 1×1×1 convolution, the multi-layer feature fusion module first fuses low- and mid-level features, and then the 3D convolutional attention module CBAM3D weights the channels and spatial dimensions to obtain attention-enhanced features. ; in This indicates feature concatenation; the MLP layer includes batch normalization and the ReLU activation function. Then, the middle layer features... with deep features The layers are fused and mapped back to the Transformer embedding dimension via a 1×1×1 convolution:

[0053] in, For fused features, the plus sign indicates residual connections, used to preserve the stability of deep semantics. The final fused features will then be... The 3D spatial feature map is reorganized into a token sequence and used as the global semantic representation of the MRI modality, denoted as . .

[0054] Preferably, the spatiotemporal functional features of the preprocessed four-dimensional resting-state functional magnetic resonance imaging data are extracted based on the SwiFT model, including the following steps: The preprocessed four-dimensional resting-state functional magnetic resonance imaging data is divided into multiple fixed-length subsequences along the time dimension using a sliding window; Multiple subsequences are input features into the SwiFT model. The SwiFT model uses a four-stage hierarchical architecture for feature extraction and outputs spatiotemporal functional features. A four-stage hierarchical architecture is used for feature extraction, specifically: In the first stage, initial spatiotemporal features are obtained by performing tile embedding on the input features, and local spatiotemporal dependency modeling is performed through at least two four-dimensional Swing Transformer blocks. Within each four-dimensional Swing Transformer block, the input features are updated alternately using a four-dimensional window multi-head self-attention mechanism and a four-dimensional shift window multi-head self-attention mechanism to achieve local window partitioning and cross-window information interaction. In the second and third stages, spatial resolution is reduced and the number of feature channels is increased by tile embedding and merging, respectively. Different numbers of four-dimensional Swin Transformer blocks are stacked in different stages to enhance representation capabilities. In the fourth stage, after obtaining deep spatiotemporal features, a global attention mechanism is introduced to integrate the contextual information of all spatiotemporal locations.

[0055] Among them, initial spatiotemporal features refer to intermediate feature representations in the four-dimensional fMRI data processing workflow that have undergone preliminary transformations but have not yet undergone deep hierarchical modeling, while retaining the basic spatial structure and temporal dynamic information of the original data; local spatiotemporal dependency modeling refers to the process of learning and characterizing the correlation patterns of signal changes at nearby time points in four-dimensional data (such as fMRI and video); the four-dimensional window multi-head self-attention mechanism is an attention computation method for efficiently processing 4D spatiotemporal data (such as fMRI and video). Its core is to divide the complete 4D data into local four-dimensional windows and perform standard multi-head self-attention computation only within each window, thereby reducing the quadratic complexity of global attention to the linear complexity within the window, achieving refined modeling of feature interactions within local spatiotemporal regions; the four-dimensional shift window multi-head self-attention mechanism is an efficient attention strategy for processing 4D spatiotemporal data, aiming to overcome the locality limitations of four-dimensional window multi-head self-attention. Its core is to alternately use four-dimensional window partitioning schemes with different offsets (i.e., "shifting" windows) in two or more consecutive attention modules, so that voxels between different windows can establish interaction in the next layer, ultimately achieving cross-window, near-global spatiotemporal dependency modeling, while maintaining the high efficiency advantage brought by window computation. The above scheme is a specific implementation detail of step S3. It involves dividing the preprocessed four-dimensional fMRI data into several fixed-length subsequences along the time dimension using a sliding window. These subsequences are then used as input features into the SwiFT model. Based on this model, feature extraction is performed in four stages. In the first stage, tile embedding is used to obtain initial spatiotemporal features for local spatiotemporal dependency modeling. In each four-dimensional Swing Transformer block, the input features are updated alternately using a four-dimensional window multi-head self-attention mechanism and a four-dimensional shift window multi-head self-attention mechanism. In the second and third stages, tile embedding and merging are used to reduce spatial resolution and increase the number of feature channels, respectively. Different numbers of four-dimensional Swing Transformer blocks are stacked in different stages. After obtaining deep spatiotemporal features in the fourth stage, a global attention mechanism is introduced to integrate the contextual information of all spatiotemporal locations, and finally, spatiotemporal functional features are output. Specifically, the process involves: segmenting long-term fMRI data into fixed-length subsequences to adapt to the model input; standardizing scan data of arbitrary length into fixed-dimensional input; capturing dynamic characteristics: each subsequence captures the dynamics of neural activity over a short period, allowing the model to learn patterns within these segments separately, and then integrating them through ensemble or subsequent processing to model the time-varying nature of functional connectivity; and based on the first stage, performing shallow feature extraction through tile embedding and several 4D Swin Transformer blocks to achieve basic feature extraction: tile embedding converts the raw signal into an initial spatiotemporal token sequence for efficient local spatiotemporal modeling; 4D Swin Transformer blocks efficiently learn the spatiotemporal relationships between voxels within a window through 4D window multi-head self-attention (4D-WMSA), introducing cross-window interaction: 4D shift window multi-head self-attention (4D-SWMSA). Breaking down window isolation allows information to be transferred between adjacent windows, achieving a controlled, progressively expanding receptive field. Based on the second and third stages, spatial resolution is reduced and the number of channels increased through "tile embedding and merging," and more Transformer blocks are stacked to achieve spatial abstraction and semantic enhancement: the "tile embedding and merging" operation (similar to pooling) progressively expands the spatial receptive field represented by each feature point, while increasing feature dimensions to encode more complex information, allowing features to evolve from representing local voxel activity to representing activity patterns of brain regions or networks, enhancing representational capabilities; stacking more blocks enables the model to learn more complex, higher-order spatiotemporal nonlinear relationships. Based on the fourth stage, after obtaining deep features, a global attention mechanism is introduced to achieve the ultimate fusion of the whole-brain spatiotemporal context. After undergoing local hierarchical abstraction, this step allows any location in the feature sequence to interact directly with all other locations. This enables modeling long-range dependencies between any two high-level functional units across the whole brain.

[0056] The above scheme transforms the preprocessed fMRI signal into a multi-scale, high-level spatiotemporal dynamic representation. Based on efficient computation, through its hierarchical, attention-driven architecture, it can learn functional abnormalities at various levels, from local brain region coordination to whole-brain network dynamics.

[0057] Preferably, the input features are updated by alternately using a four-dimensional window multi-head self-attention mechanism and a four-dimensional shift window multi-head self-attention mechanism, including the following steps: After layer normalization of the input features, the four-dimensional window multi-head self-attention is calculated using the first formula or the four-dimensional shift window multi-head self-attention is calculated using the second formula, and then residual connection is performed with the input features to obtain intermediate features; After layer normalization of the intermediate features, they are processed by a multilayer perceptron and then residually connected with the intermediate features to obtain the output features. The output features are the updated input features; The first formula is as follows: ; The second formula is as follows: ; The formula for calculating the output features is: ; or, ; in, For the first Output features of a layered four-dimensional Swing Transformer block For the first The intermediate representation of a layered four-dimensional SwinTransformer block. For four-dimensional window multi-head self-attention, For four-dimensional shift window multi-head self-attention, For layer normalization, It is a multilayer perceptron module.

[0058] The above steps are the specific implementation details of updating the input features within each four-dimensional Swing Transformer block. After layer normalization of the input features, the four-dimensional window multi-head self-attention is calculated using the first formula or the four-dimensional shift window multi-head self-attention is calculated using the second formula. The input features are then residually connected to obtain intermediate features. After layer normalization of the intermediate features again, they are processed by a multilayer perceptron and residually connected to the intermediate features to obtain the output features.

[0059] Specifically: Based on the first layer normalization, the distribution of input features is standardized before attention calculation or MLP processing to stabilize the training process: the mean and variance of features are normalized to a stable range, reducing internal covariate shifts, making subsequent attention weight calculations and MLP activation values ​​more controllable, accelerating model convergence and improving training stability; based on 4D-WMSA / 4D-SWMSA calculations, core spatiotemporal dependency modeling is performed to inject interactive information. 4D-WMSA: learns the dependencies between features in local spatiotemporal regions within a window; 4D-SWMSA: establishes connections between adjacent windows through window shifting, gradually expanding the receptive field and learning spatiotemporal correlations across regions; based on the first residual connection, the output of the attention module is added to the original input of the module to retain the original identity information and prevent information loss. The attention mechanism may "reshape" or "filter" features. The residual connection ensures that no matter what the attention calculation is, the key information components of the original input can be passed to the next layer without loss, alleviating gradient vanishing. Based on the second layer normalization, the distribution of intermediate features is standardized again before MLP processing. This is to prepare conditions for nonlinear transformation. MLP (usually containing activation functions) is sensitive to the input scale. The second normalization ensures that the features input to the MLP have a stable distribution, making the nonlinear transformation more effective and preventing gradient explosion or vanishing. Based on multilayer perceptron (MLP) processing, the features at each spatiotemporal location are subjected to independent, nonlinear, high-dimensional transformation to enhance the expressive power of each feature point. MLP is a powerful nonlinear function approximator that can nonlinearly fuse and upscale the contextual information after attention aggregation, uncover more complex feature interactions, and increase the expressive power of the model. Based on the second residual connection, the output of MLP is added to the input of MLP (i.e., intermediate features) to ensure smooth information flow again and prevent the loss of important information during the nonlinear transformation of MLP. The features after attention modulation and MLP enhancement are combined with the features before modulation to form a new feature with richer information and more robustness.

[0060] The above scheme constructs a robust feature transformation unit that emphasizes both information enhancement and information preservation. While actively introducing cross-spatial interaction and nonlinearity, it ensures that the original key information is not lost or degraded in deep networks through residual connections. This allows the model to be stacked very deep to learn highly abstract features, while avoiding gradient vanishing and network degradation problems.

[0061] In one embodiment, SwiFT is an end-to-end Transformer architecture specifically designed for 4D fMRI data, capable of jointly learning spatiotemporal representations of intrinsic brain activity and effectively capturing complex neurodynamic patterns. At the input layer, the fMRI data is formally represented as: , in For batch size, For the number of channels, =96 represents the spatial dimension. The number of time frames is the input number to the model. It was fixed at 20. This was to convert the original long-term fMRI data ( >20) Adapt to the model input, using a sliding window with a step size of 20 to segment the time series: ; in, The time frame number represents the input raw fMRI data, and S represents the set of fixed-length subsequences obtained after time frame segmentation. This transforms each subject's scan into... / 20 The model uses multiple sub-sequences to ensure it covers the entire neural dynamics. In terms of structural design, SwiFT employs a four-stage hierarchical architecture. The input first obtains an initial feature representation through patchbedding, and then models local spatiotemporal dependencies using two 4D SwinTransformer blocks. Within each 4D SwinTransformer block, SwiFT alternately uses four-dimensional window multi-head self-attention (4D-WMSA) and four-dimensional shifted window multi-head self-attention (4D-SWMSA). 4D-WMSA divides the input into local windows, while 4D-SWMSA achieves cross-window information interaction through window displacement. For any input features... The update process is as follows: ; ; ; or, ; in, For the first Output features of the 4D SwinTransformer block This serves as an intermediate representation. Subsequently, in the second and third stages, patch merging is used to halve the spatial resolution and double the number of channels. Simultaneously, different numbers of 4D SwiFT blocks are stacked in each stage to continuously enhance the representation capability. Finally, after obtaining deep features in the fourth stage, a global attention mechanism is introduced to integrate all spatiotemporal context information. Ultimately, after the above four-stage structure, the features output by SwiFT are: This is used for subsequent cross-modal fusion.

[0062] like Figure 6As shown, preferably, based on the bidirectional sparse cross-attention mechanism, intramodal sparse feature screening and intermodal bidirectional interaction are performed on brain structural features and spatiotemporal functional features to obtain fused features, including the following steps: E1. Brain structural features and spatiotemporal multifunctional features are mapped to the same fusion feature space through learnable linear projection matrices to obtain aligned structural feature representations and functional feature representations; E2. For the aligned structural feature representation and functional feature representation respectively, a dynamic sparse attention mechanism is used to filter key features within the modality, resulting in the filtered structural features and filtered functional features. The dynamic sparse attention mechanism calculates a standard dense attention distribution and a sparse distribution generated based on the ReLU² function, and adaptively weights and fuses the standard dense attention distribution and the sparse distribution through learnable sparsity weights to achieve a balance between redundancy suppression and information preservation. E3. Based on the filtered structural features and filtered functional features, a bidirectional cross-attention mechanism is used to perform bidirectional interaction between modalities; wherein, the filtered functional features are used as query vectors, and the filtered structural features are used as key vectors and value vectors to calculate the first-direction cross-attention to obtain the structurally modulated functional features; the filtered structural features are used as query vectors, and the filtered functional features are used as key vectors and value vectors to calculate the second-direction cross-attention to obtain the functionally modulated structural features. E4. Perform pooling operations on the functional features after structural modulation and the structural features after functional modulation respectively to obtain the pooled functional vector and the pooled structural vector. Concatenate the pooled functional vector and the pooled structural vector to obtain the fused feature.

[0063] The linear projection matrix in step E1 is a set of learnable parameters used in neural networks (especially in attention mechanisms and multimodal fusion) to perform linear transformations. Its core function is to map input features from one vector space to another new vector space with the same or different dimensions. The dynamic sparse attention mechanism in step E2 is an improved attention computation method. Its core lies in dynamically and adaptively generating a sparse attention weight distribution based on the current input content, rather than calculating dense attention for all input positions. This mechanism aims to reduce computational overhead, decrease noise interference, and enhance the model's ability to focus on key information by suppressing most low-association attention connections. A standard dense attention distribution refers to the attention weight vector / matrix obtained after normalization using the Softmax function in classic attention mechanisms (such as scaled dot product attention). Its core characteristic is that for a given query, the sum of attention weights for all keys is 1, and each key receives a non-zero attention weight. When constructing the sparse attention distribution, the ReLU² function is often used as the activation function to generate the sparse mask. Its squared property significantly amplifies positive attention scores while completely suppressing negative scores to zero. This helps generate a sharper, more discriminative sparse attention pattern, thus more effectively highlighting a few key feature interactions and forcing the model to ignore a large number of unimportant associations. The bidirectional cross-attention mechanism in step E3 is a multimodal feature fusion strategy. Its core is to allow feature sequences from two different modalities to interact in a symmetrical and inverse manner. Steps E1 to E4 are the specific implementation details of step S4. They involve mapping brain structural features and spatiotemporal functional features to the fusion feature space through a linear projection matrix, aligning the two to obtain aligned structural and functional feature representations. Based on the aligned structural and functional feature representations, key features are selected within the modality using a dynamic sparse attention mechanism, and bidirectional interaction is performed between modalities using a bidirectional cross-attention mechanism to obtain structurally modulated functional features and functionally modulated structural features. These are then pooled and concatenated to obtain the fusion feature.

[0064] Specifically, step E1 maps features from different encoders to the same semantic space to establish a common platform for modal dialogue. Since sMRI and fMRI features have different distributions in the original space, this step unifies and aligns the feature spaces, making their feature vectors comparable and operable. Step E2 adaptively filters redundancy within each modality, highlighting key features to achieve feature purification and denoising. For AD classification: on the structural side, it suppresses a large number of normal brain region features unrelated to AD, focusing on atrophy-sensitive areas; on the functional side, it suppresses a large amount of background neural activity and noise throughout the brain, focusing on key brain networks with abnormal functional connectivity. The dynamism is reflected in the sparsity being determined by the input content, and the balance is reflected in the fusion of dense and sparse distributions through learnable weights to avoid excessive information loss. Step E3 establishes two-directional, asymmetric information modulation channels to achieve deep "structure-function" mutual interpretation and enhancement: structure → function modulation: functional features are updated under the "key-value" pair constraints of structural features. This is equivalent to providing precise anatomical anchors for dynamic functional activities. For example, the functional anomaly of "weakened activity in the posterior nodes of the default network" is more explicitly associated with the structural anomaly of "atrophy of the posterior cingulate cortex." Functional → structural modulation: Structural features are updated under the modulation of the "key-value" pairs of functional features. This is equivalent to re-evaluating and weighting the importance of structural features from the perspective of functional activity. Based on step E4, the high-dimensional sequence features after bidirectional interaction are aggregated into fixed-length vectors and merged to generate the final fused feature vector. Pooling (such as average pooling and max pooling) summarizes the sequence information of each modality into a global feature vector, retaining the core information of the modality after being modulated by the other party. Concatenation merges the two vectors to form a composite feature vector that simultaneously contains "structural information from the functional perspective" and "functional information from the structural perspective."

[0065] In one embodiment, the feature fusion process fully leverages the brain anatomical features provided by structural magnetic resonance imaging (sMRI) and the dynamic features of neural function reflected by resting-state functional magnetic resonance imaging (fMRI), achieving deep fusion within and between modalities through sparse attention and bidirectional cross-attention mechanisms. Specifically, representations are first extracted from both modalities; the features extracted from the fMRI encoder model are represented as follows: in Indicates batch size, This indicates the number of patches / tokens after fMRI feature expansion. This represents the feature dimension of fMRI. Similarly, the features extracted from the sMRI model are represented as... ,in, This indicates the number of patches / tokens after the sMRI feature is unfolded. This represents the feature dimension of sMRI. To unify the feature dimension, linear projection is performed using a learnable projection matrix. It maps to the same fusion space dimension : ; ; In the intramodal representation phase, MRI is decomposed into a series of 3D patch embeddings after encoder processing. While these patch embeddings comprehensively characterize brain structure, they contain numerous anatomical regions irrelevant to Alzheimer's disease. Distributing attention equally to all patch embeddings inevitably introduces redundant features and may obscure key regions highly correlated with Alzheimer's. Sparse attention mechanisms enhance the discriminative power of MRI features by adaptively filtering key regions and highlighting the representation of lesion-sensitive areas. In contrast, fMRI features represent complex spatiotemporal sequences, containing rich dynamic functional information but also accompanied by temporal noise and non-specific connections. Directly employing dense attention modeling may lead to model interference from irrelevant activity patterns, weakening the ability to capture disease-related functional pathways. Sparse attention, by suppressing redundant brain activity in the spatiotemporal dimension, highlights functional connections closely related to AD progression, thereby enhancing the sensitivity and specificity of disease representation at the functional level. To achieve this goal, a dynamic sparse attention mechanism is introduced, which combines a standard dense attention distribution with a sparse distribution obtained through ReLU² activation using learnable sparsity weights. An adaptive weighted fusion is performed to achieve a balance between redundancy suppression and information preservation. The sparse attention weights are adaptively fused from dense and sparse distributions. ; Where A represents the final generated hybrid attention weight matrix, and the more discriminative intra-modal features are represented as follows: .

[0066] In the intermodal interaction phase, a bidirectional cross-attention mechanism is used to achieve deep fusion of structural and functional information. This mechanism allows the dynamic functional features of fMRI to acquire supplementary information from the anatomical representation of sMRI, while the structural features of sMRI can also be modulated by the functional activity patterns of fMRI. In Alzheimer's disease, anatomical atrophy reflected on MRI and functional abnormalities shown on fMRI are complementary. Therefore, bidirectional cross-attention is not only a modal fusion strategy but can also be regarded as a modeling of disease mechanisms. When fMRI features are used as the query and sMRI features as the key and value, cross-attention updates the functional representation of fMRI by weighted aggregation of MRI anatomical features, thereby allowing dynamic temporal features to be constrained by anatomical localization. Conversely, when sMRI features are used as the query and fMRI features as the key and value, MRI anatomical features can acquire supplementary information from functional activation, thereby highlighting disease-related functional connectivity abnormalities. ; ; in, These are the functional characteristics after structural modulation. The structural features are those resulting from functional modulation; During the bidirectional interaction, the temporal dynamics of fMRI features, under structural constraints, more readily reveal abnormal activation states; simultaneously, sMRI features, under functional modulation, better highlight atrophic regions associated with Alzheimer's disease. Through this complementary effect, the model not only further suppresses redundant features in the unimodal model, but also strengthens multimodal features closely related to the pathology of Alzheimer's disease through the interaction of functional and structural features.

[0067] Finally, the model respectively... and Pooling is performed, and the two are concatenated to obtain the dual-modal fused feature. ; Step S5: Input the fused features into the classification layer and calculate the classification probability to obtain the AD classification result, as shown in the following formula: .

[0068] To further verify the technical effects of the present invention, an experiment was designed, and the specific experimental procedure is as follows: The data used in this study came from the ADNI dataset, and its use was approved by the relevant organizations. The study included 89 AD patients, 142 MCI patients, and 220 normal controls (NC) as the experimental sample. Because each sample required available fMRI and MRI paired data, many ADNI samples only contained one type of data and were therefore excluded from the study. For both MRI and fMRI modalities, the study uniformly used the DPABI tool for data preprocessing. For MRI data, scalp structures were first removed, then T1 structural images were fitted to the functional image space, and the structural images were segmented into gray matter, white matter, and cerebrospinal fluid. Finally, the data was normalized to the MNI space. Because the gradient magnetic field requires time to stabilize during MRI scans, and subjects also need time to adapt, the images at the first few time points had more noise. Therefore, the first 10 time points of fMRI data were removed, and then the fMRI data underwent time-film correction, head motion correction, head motion parameter regression, spatial normalization, and spatial smoothing preprocessing. Before inputting bimodal data into the model, the preprocessed brain MRI images need to be resampled to the same voxel spacing (1.75×1.75×1.75), and then cropped to adjust the 3D volume to 128×128×128. For each 4D fMRI data point, the brain image is globally normalized in four dimensions excluding the background region, and then the background is filled with the minimum voxel intensity value. Cropping and filling on the background adjusts the 3D volume to 96×96×96.

[0069] All experiments were implemented using a deep learning framework based on PyTorch 2.0.0 + CUDA 11.8. Model training employed the AdamW optimizer at 5×10-1 times. -6The initial learning rate was set using binary cross-entropy (BCE) as the loss function. The training batch size was set to 32, and the maximum number of training epochs was determined using an adaptive early stopping strategy. For data partitioning, the training and test sets were randomly divided according to participant IDs at a ratio of 8:2 to ensure no data leakage between different participants. Model performance was evaluated using a comprehensive analysis of multiple metrics, including Balanced Classification Accuracy (BCA), Area Under Curve (AUC), F1 score, Specificity (SPE), Sensitivity (SEN), and Precision (PR). All experiments were performed on an NVIDIA A800 GPU (80 GB VRAM) to fully support joint modeling and feature fusion training of high-dimensional 3D sMRI and 4D fMRI modalities.

[0070] The experimental results are shown in Table 1: Table 1. Classification results of the proposed Alzheimer's disease classification method compared with other methods.

[0071] As can be seen from the comparison results in Table 1, the method proposed in this invention achieves optimal performance on the three classification tasks of AD / NC, MCI / NC, and AD / MCI, and its overall performance is significantly better than existing state-of-the-art single-modal and multi-modal methods. Compared with traditional 3D CNNs (such as 3D ResNet) and single-modal methods based on Transformer deep models (such as Swing UNETR and SAM-Med3D), multi-modal methods have stronger modeling capabilities for high-dimensional image data, more comprehensive characterization of AD brain symptom information, and more accurate classification results. In terms of multi-modal fusion models, although methods such as OLFG, MDL-Net, and D-MAFF have achieved certain improvements through mechanisms such as latent space constraints, cross-modal attention, or multi-scale fusion, their fusion structures mostly still rely on unidirectional interaction or dense attention patterns, which are insufficient in suppressing intramodal redundant features and do not adequately model the fine-grained correspondence between structure and function. HybridCA-Net shares some similarities with our proposed method, both employing a cross-attention-based cross-modal interaction approach and feature selection to highlight key features. However, when processing 4D fMRI data, HybridCA-Net and BF still use a method of dividing brain regions and calculating functional connectivity, while our study directly uses raw data for end-to-end feature learning. This results in a more comprehensive model that retains more information, leading to a significant improvement in classification performance after multimodal fusion. Our invention achieves accuracy (ACC) of 97.67%, 92.23%, and 95.13% and AUC of 99.71%, 97.70%, and 98.26% on the three tasks, significantly outperforming existing methods.

[0072] like Figure 7 As shown, an Alzheimer's disease classification system includes: The preprocessing module is used to preprocess the raw data of three-dimensional structural magnetic resonance imaging and the raw data of four-dimensional resting-state functional magnetic resonance imaging, respectively, to obtain preprocessed three-dimensional structural magnetic resonance imaging data and preprocessed four-dimensional resting-state functional magnetic resonance imaging data. The brain structure feature extraction module is used to extract brain structure features from preprocessed three-dimensional structural magnetic resonance imaging data based on a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module. The spatiotemporal functional feature extraction module is used to extract spatiotemporal functional features from preprocessed four-dimensional resting-state functional magnetic resonance imaging data based on the SwiFT model. The feature fusion module is used to perform intramodal sparse feature screening and intermodal bidirectional interaction on brain structural features and spatiotemporal functional features based on the bidirectional sparse cross-attention mechanism to obtain fused features. The results output module outputs classification results based on fusion features. The classification results are either numerical feature vectors of Alzheimer's disease probability, numerical feature vectors of mild cognitive impairment probability, or numerical feature vectors of health probability.

[0073] This invention also discloses an Alzheimer's disease classification system, aiming to achieve Alzheimer's disease classification in a modular manner. Specifically, a preprocessing module preprocesses raw data from three-dimensional structural magnetic resonance imaging (3D SMILE) and four-dimensional resting-state functional magnetic resonance imaging (4D FMRI) to obtain preprocessed 3D SMILE and 4D FMRI data, respectively. A brain structure feature extraction module extracts brain structure features from the preprocessed 3D SMILE data. A spatiotemporal functional feature extraction module extracts spatiotemporal functional features from the preprocessed 4D FMRI data. A feature fusion module fuses the brain structure features and spatiotemporal functional features. A result output module takes the fused features as input and outputs classification results: probability numerical feature vectors of Alzheimer's patients, probability numerical feature vectors of mild cognitive impairment, or probability numerical feature vectors of healthy individuals.

[0074] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

[0075] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0076] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0077] The above provides a detailed description of an Alzheimer's disease classification method and system provided in this application. The above description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for classifying Alzheimer's disease, characterized in that, Includes the following steps: The raw data of three-dimensional structural magnetic resonance imaging and the raw data of four-dimensional resting-state functional magnetic resonance imaging were preprocessed to obtain preprocessed three-dimensional structural magnetic resonance imaging data and preprocessed four-dimensional resting-state functional magnetic resonance imaging data. Brain structural features in the preprocessed three-dimensional structural magnetic resonance imaging data are extracted based on a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module. Spatiotemporal functional features were extracted from the preprocessed four-dimensional resting-state functional magnetic resonance imaging data based on the SwiFT model. Based on the bidirectional sparse cross-attention mechanism, intramodal sparse feature screening and intermodal bidirectional interaction are performed on the brain structural features and the spatiotemporal functional features to obtain fused features. Based on the fusion features, a classification result is output, which is a numerical feature vector of Alzheimer's disease probability, a numerical feature vector of mild cognitive impairment probability, or a numerical feature vector of health probability.

2. The Alzheimer's disease classification method as described in claim 1, characterized in that, The preprocessing of the raw three-dimensional structural magnetic resonance imaging data includes the following steps: Data processing tools were used to remove the skull structure from the brain image in the original three-dimensional structural magnetic resonance imaging data to obtain a skull-removed T1-weighted structural image; Register the T1-weighted structural images obtained from the craniotomy process to the corresponding functional image space; Within the functional image space, the craniotomized T1-weighted structural image is segmented into images of three tissue components: gray matter, white matter, and cerebrospinal fluid. The images of each segmented component are spatially normalized to MNI space to obtain standard data for three-dimensional structural magnetic resonance imaging. The standard three-dimensional structural magnetic resonance imaging data is resampled to a fixed voxel spacing and cropped to a fixed three-dimensional size to obtain the preprocessed three-dimensional structural magnetic resonance imaging data.

3. The Alzheimer's disease classification method as described in claim 1, characterized in that, The preprocessing of raw data from four-dimensional resting-state functional magnetic resonance imaging includes the following steps: Data processing tools are used to remove image data at preset time points during the initial scanning phase of the original four-dimensional resting-state functional magnetic resonance imaging data to obtain denoised four-dimensional resting-state functional magnetic resonance imaging data. The four-dimensional resting-state functional magnetic resonance imaging (fMRI) denoised data were sequentially processed with time-slice correction, head motion correction, and head motion parameter regression to obtain four-dimensional resting-state functional magnetic resonance imaging (fMRI) corrected data. The four-dimensional resting-state functional magnetic resonance imaging correction data are spatially normalized to MNI space and then spatially smoothed to obtain four-dimensional resting-state functional magnetic resonance imaging standard data. Global normalization is performed on the non-background regions in the standard four-dimensional resting-state functional magnetic resonance imaging data, and the background regions are filled with preset voxel intensity values. The data is then cropped to a fixed three-dimensional size to obtain the preprocessed four-dimensional resting-state functional magnetic resonance imaging data.

4. The Alzheimer's disease classification method as described in claim 1, characterized in that, The extraction of brain structural features from the preprocessed three-dimensional structural magnetic resonance imaging data using a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module includes the following steps: The preprocessed 3D structural magnetic resonance imaging data is divided into patches by a 3D convolutional embedding layer, mapped to a high-dimensional feature space, and appended with learnable class label vectors to form an initial token sequence. The initial token sequence is input into a 3D vision transformer encoder consisting of several stacked encoding blocks; each encoding block includes layer normalization, a dynamically composable multi-head attention module, and a multilayer perceptron module; the dynamically composable multi-head attention module dynamically generates a combination coefficient matrix based on the input features to adaptively adjust the interaction relationship between multiple attention heads; Shallow, mid-level, and deep features are extracted from different levels of the 3D vision transformer encoder. The shallow and mid-level features are convolved, concatenated, and weighted and then fused with the deep features through the multi-layer feature fusion module to obtain the brain structure features.

5. The Alzheimer's disease classification method as described in claim 4, characterized in that, The step of dynamically generating a combination coefficient matrix based on input features to adaptively adjust the interaction relationship between multiple attention heads includes the following steps: A linear transformation is performed on the input features of the current layer of the encoded block to generate a query matrix, a key matrix, and a value matrix; The combination coefficient matrix is ​​calculated based on the preset dynamic generation network; Before Softmax normalization, the attention score matrix calculated from the query matrix and the key matrix is ​​pre-modulated using the combined coefficient matrix. After Softmax normalization, the normalized attention weight matrix is ​​modulated again using the combined coefficient matrix. The Concat function multiplies the attention weight matrix (after two modulations) with the value matrix and then concatenates it with a preset output projection matrix to output the result of the dynamically composable multi-head attention module.

6. The Alzheimer's disease classification method as described in claim 4, characterized in that, The combined coefficient matrix is ​​generated by a five-branch weighted combination structure, which includes: a static basic projection matrix branch, a low-rank mapping matrix branch generated from query features, a low-rank mapping matrix branch generated from key features, a rank-constrained dimension branch, and a dynamic gating weight branch.

7. The Alzheimer's disease classification method as described in claim 1, characterized in that, The extraction of spatiotemporal functional features from the preprocessed four-dimensional resting-state functional magnetic resonance imaging data based on the SwiFT model includes the following steps: The preprocessed four-dimensional resting-state functional magnetic resonance imaging data is divided into multiple fixed-length subsequences along the time dimension using a sliding window; Multiple subsequences are input as input features to the SwiFT model, which uses a four-stage hierarchical architecture for feature extraction and outputs the spatiotemporal functional features. The feature extraction is performed using a four-stage hierarchical architecture, specifically as follows: In the first stage, initial spatiotemporal features are obtained by performing tile embedding processing on the input features, and local spatiotemporal dependency modeling is performed through at least two four-dimensional Swing Transformer blocks; wherein, within each of the four-dimensional Swing Transformer blocks, the input features are updated alternately using a four-dimensional window multi-head self-attention mechanism and a four-dimensional shift window multi-head self-attention mechanism to achieve local window partitioning and cross-window information interaction. In the second and third stages, spatial resolution is reduced and the number of feature channels is increased by tile embedding and merging, respectively. Different numbers of four-dimensional Swin Transformer blocks are stacked in different stages to enhance representation capabilities. In the fourth stage, after obtaining deep spatiotemporal features, a global attention mechanism is introduced to integrate the contextual information of all spatiotemporal locations.

8. The Alzheimer's disease classification method as described in claim 7, characterized in that, The method of alternately using a four-dimensional window multi-head self-attention mechanism and a four-dimensional shift window multi-head self-attention mechanism to update the input features includes the following steps: After performing layer normalization on the input features, the four-dimensional window multi-head self-attention is calculated using the first formula or the four-dimensional shift window multi-head self-attention is calculated using the second formula, and then residual connection is performed with the input features to obtain intermediate features; After performing layer normalization on the intermediate features, they are processed by a multilayer perceptron and then residually connected with the intermediate features to obtain the output features. The output feature is the updated input feature; The first formula is as follows: ； The second formula is as follows: ； The formula for calculating the output features is: ； or, ; in, For the first Output features of a layered four-dimensional Swing Transformer block For the first The intermediate representation of a layered four-dimensional SwinTransformer block. For four-dimensional window multi-head self-attention, For four-dimensional shift window multi-head self-attention, For layer normalization, It is a multilayer perceptron module.

9. The Alzheimer's disease classification method as described in claim 1, characterized in that, The method of performing intra-modal sparse feature screening and inter-modal bidirectional interaction on the brain structural features and spatiotemporal functional features based on a bidirectional sparse cross-attention mechanism to obtain fused features includes the following steps: The brain structural features and the spatiotemporal multifunctional features are respectively mapped to the same fusion feature space through a learnable linear projection matrix to obtain aligned structural feature representations and functional feature representations; The aligned structural and functional feature representations are then processed using a dynamic sparse attention mechanism to filter key features within the modality, resulting in filtered structural and functional features. The dynamic sparse attention mechanism calculates a standard dense attention distribution and a distribution based on... The function generates a sparse distribution and adaptively weights and fuses the standard dense attention distribution and the sparse distribution with learnable sparsity weights to achieve a balance between redundancy suppression and information preservation. Based on the filtered structural features and filtered functional features, a bidirectional cross-attention mechanism is used for bidirectional interaction between modalities. Specifically, the filtered functional features are used as query vectors, and the filtered structural features are used as key vectors and value vectors to calculate the first-direction cross-attention, resulting in the structurally modulated functional features. The filtered structural features are used as query vectors, and the filtered functional features are used as key vectors and value vectors to calculate the second-direction cross-attention, resulting in the functionally modulated structural features. Pooling operations are performed on the functional features after structural modulation and the structural features after functional modulation to obtain pooled functional vectors and pooled structural vectors. The pooled functional vectors and pooled structural vectors are then concatenated to obtain the fused features.

10. An Alzheimer's disease classification system, characterized in that, include: The preprocessing module is used to preprocess the raw data of three-dimensional structural magnetic resonance imaging and the raw data of four-dimensional resting-state functional magnetic resonance imaging, respectively, to obtain preprocessed three-dimensional structural magnetic resonance imaging data and preprocessed four-dimensional resting-state functional magnetic resonance imaging data. The brain structure feature extraction module is used to extract brain structure features from the preprocessed three-dimensional structural magnetic resonance imaging data based on a three-dimensional visual transformer model that integrates a dynamic composable multi-head attention mechanism and a multi-layer feature fusion module. The spatiotemporal functional feature extraction module is used to extract spatiotemporal functional features from the preprocessed four-dimensional resting-state functional magnetic resonance imaging data based on the SwiFT model. The feature fusion module is used to perform intramodal sparse feature screening and intermodal bidirectional interaction on the brain structural features and the spatiotemporal functional features based on a bidirectional sparse cross-attention mechanism to obtain fused features. The result output module is used to output a classification result based on the fusion features. The classification result is a numerical feature vector of Alzheimer's disease probability, a numerical feature vector of mild cognitive impairment probability, or a numerical feature vector of health probability.

Images