Prediction method for progressive mild cognitive impairment in Alzheimer's disease

Abstract

The invention relates to the field of computer-aided diagnosis, cross-modal fusion, image morphological operation and feature extraction related to computer vision and artificial intelligence, in particular to a method for predicting mild cognitive impairment in the progress stage of Alzheimer's disease, which comprises the following steps: S1, respectively inputting MRI (Magnetic Resonance Imaging), Clinical Table and PET (Positron Emission Tomography) data into respective encoders, and extracting initial image features and table features; s2, for the MRI branch and the PET branch, inputting respective image features and table features into a mixed channel attention module together, and calculating a mixed channel attention feature; s3, performing element-by-element multiplication on the mixed channel attention features obtained in the step S2 and original image features of corresponding branches to realize feature enhancement of channel dimensions; and S4, inputting the feature enhanced in the step S3 into a space attention module, and calculating a space attention feature. Structural MRI, functional PET and clinical table data are integrated for predicting the conversion from MCI to AD.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, computer-aided diagnosis involved in artificial intelligence, cross-modal fusion, image morphological operation, feature extraction, and particularly relates to a method for predicting progression mild cognitive impairment in Alzheimer's disease. BACKGROUND

[0002] Alzheimer's disease (AD) is a progressive neurodegenerative disorder characterized by irreversible cognitive decline, severely affecting patients' daily functioning. The Alzheimer's Association reports that more than 50 million people worldwide are affected by AD, and the number of patients in the United States is expected to double by 2050. As AD progresses, patients typically experience severe deterioration in cognitive function, including memory loss, language impairment, and impaired reasoning. The prodromal stage of AD is classified according to the severity of cognitive decline, including subjective cognitive decline (SCD) and mild cognitive impairment (MCI). Patients in the MCI stage have already shown significant cognitive deficits. Currently, there is no effective treatment that can reverse AD or MCI, and only limited drugs can alleviate symptoms. In clinical practice, progressive mild cognitive impairment (pMCI) refers to patients who are likely to develop AD within about three years, while stable MCI (sMCI) indicates patients whose cognitive status remains unchanged during the same period. Accurate prediction of pMCI is crucial for early intervention, which can delay the onset of AD.

[0003] Advances in neuroimaging technology have significantly enhanced our ability to collect detailed anatomical and functional information about the brain using techniques such as magnetic resonance imaging (MRI) and positron emission tomography (PET). MRI provides high-resolution structural details, distinguishing between gray and white matter, while PET detects functional changes in brain metabolism, providing insights into neurodegenerative processes. Using this data, computer-aided diagnosis (CAD) systems have been developed to help distinguish pMCI and sMCI early. In addition to imaging data, clinical information, including demographic details, laboratory test results, and neurological assessments, also provides valuable insights into AD. Based on this wealth of information, machine learning (ML) techniques have been widely applied to analyze clinical data related to AD. For example, Moradi et al. used a random forest (RF) classifier to predict early conversion from MCI to AD. Similarly, Mathew et al. integrated principal component analysis, discrete wavelet transform, and support vector machines for AD classification. While these traditional ML methods have shown promise, they are heavily dependent on handcrafted features, which require extensive domain expertise and are often subject to subjective interpretation. This limitation highlights the need for more automated and robust methods to analyze and predict AD progression.

[0004] Recently, deep learning techniques such as ResNet, EfficientNet, and Vision Transformer (ViT) have been increasingly applied to AD diagnosis tasks. Wang et al. introduced the HOPE framework, which leverages MRI features from different disease stages to predict the conversion from MCI to AD, and achieved promising results. Zhang et al. developed a spatio-temporal Transformer-based method for constructing asynchronous functional brain networks. Some other studies have also made significant progress in MRI-based AD diagnosis. However, relying solely on MRI cannot capture key metabolic information in the brain, highlighting the limitations of single-modality approaches. To address these shortcomings, Kang et al. introduced visual attribute prompt learning (VAPL) to integrate MRI and clinical table data, while Duenias et al. proposed the hyperfusion framework, which combines medical images and clinical features. The fusion of MRI and PET has also become a common multi-modal strategy, providing a comprehensive understanding of brain pathology through structural and functional imaging. Li et al. proposed the ViT-based Diamond framework, which uses a dual attention mechanism to model inter-modality similarity. Other methods such as MMGPL and MDL have also achieved competitive performance.

[0005] Currently, many models face challenges in effectively integrating two or more modalities, making comprehensive diagnosis that requires structural imaging, functional imaging, and clinical data complex. To address this issue, Li et al. introduced the IMF framework, which enhances inter-modality interaction through a two-stage fusion design. The Modality-Flexible Framework leverages diverse clinical data to provide adaptive diagnosis, while the longitudinal prediction method incorporates modality uncertainty to enhance robustness. Innovative methods such as adversarial learning and flexible Mixture-of-Experts architecture have also emerged.

[0006] Despite these advances, current multi-modal fusion methods often heavily rely on deep convolutional layers and repetitive attention mechanisms, leading to high computational overhead and increased model complexity. Simplifying these processes remains a key research area for improving efficiency without sacrificing diagnostic accuracy. SUMMARY

[0007] Based on the technical problems existing in the background art, the present application proposes a method for predicting mild cognitive impairment in the progression stage of Alzheimer's disease.

[0008] The present invention proposes a method for predicting mild cognitive impairment in the progressive stage of Alzheimer's disease, comprising a lightweight trimodal fusion network model structure, which includes a hybrid block attention module and a multimodal cascade attention module, wherein the multimodal cascade attention module is connected to a classification module; wherein the hybrid block attention module consists of two submodules: a hybrid channel attention module, which is responsible for adaptively assigning weights to channel-level interactions between table and image features; and a spatial attention module, which captures and enhances local spatial dependencies, and comprises the following steps:

[0009] S1: MRI, Clinical Table, and PET data are input into their respective encoders to extract initial image features and table features;

[0010] S2: For MRI and PET branches, their respective image features and table features are input into the mixed channel attention module to calculate the mixed channel attention features;

[0011] S3: Multiply the mixed channel attention features obtained in step S2 by the original image features of the corresponding branch element by element to achieve feature enhancement in the channel dimension;

[0012] S4: The features enhanced in step S3 are input into the spatial attention module to calculate the spatial attention features;

[0013] S5: Multiply the spatial attention feature obtained in step S4 by the output result of step S3 again element by element to obtain the refined fusion feature F mri&cli and F pet&cli ;

[0014] S6: F after fusion mri&cli and F pet&cli The features are input into the multimodal cascade attention module, and deep cross-modal information interaction is performed through multiple groups of cross attention;

[0015] S7: The two feature streams output from step S6 are concatenated and then fed into the classification module to obtain the final pMCI and sMCI prediction results.

[0016] Preferably, the hybrid block attention module is used to integrate multimodal inputs, the hybrid block attention module processes image and table features, and refines the feature tensor extracted from the image encoder by utilizing complementary information from the vectorized representation of clinical table data.

[0017] Preferably, a hybrid channel attention module is used to merge information from images and clinical features at the channel level; initially, the image feature tensor F is processed img and clinical feature vector F cli ; Clinical features are embedded into a format compatible with image features through a linear embedding layer:

[0018] E cli = Embed(F cli ),

[0019] where Embed(·) is a linear layer that projects F cli from to

[0020] Subsequently, global average pooling and global max pooling are applied to the image feature tensor to produce two vectors that capture different aspects of the image features:

[0021]

[0022] These pooled vectors are passed through a shared MLP to generate channel attention weights:

[0023] M avg = MLP(F avg ), M max = MLP(F max ),

[0024] M avg , M max and E cli are combined through element-wise addition and normalization to produce a mixed channel attention map M hc :

[0025] M hc = σ(M avg + M max + E cli ),

[0026] where σ is the sigmoid function;

[0027] Finally, the mixed channel attention map M hc is applied to the original image feature tensor F img through channel-wise multiplication:

[0028]

[0029] where denotes channel-wise multiplication.

[0030] Preferably, the spatial attention feature M s ∈ R 1×D×H×W is created by concatenating the channel-wise average pooling and max pooling features, which are then processed by a convolutional layer:

[0031] M s = σ(f conv (Concat(GAP(Fhc ), GMP(F hc )))),

[0032] Where σ is the sigmoid function; the final output feature map is calculated as follows:

[0033] .

[0034] Preferably, the feature output F from the hybrid block attention module mri&cli and F pet&cli It is processed by a multimodal cascade attention module to produce two modality-enhanced outputs; these outputs are then concatenated and passed through a classification head for the final prediction; given two input modalities X and Y, each of shape R B×L×C , where B is the batch size, L is the sequence length, and L is the number of channels. The multimodal cascade attention module first reshapes these inputs into R B×C×L , and divide them into N groups, corresponding to the number of attention groups; for each group i in {1, ..., N}, a cascade computation occurs, where the feature representation of the current group is built on the features of the previous group, thereby hierarchically refining the attention.

[0035] Preferably, for each group, three projections are derived from the grouping features of the two modalities; let Represents the query, key and value of modality X, let represents the query, key, and value of modality Y; cross attention is performed bidirectionally, where features from one modality attend to the keys and values ​​of the other modality:

[0036]

[0037] The attention operation contains a learnable position-aware bias term b i ∈R L×L To enhance spatial sensitivity; the attention score is calculated as follows:

[0038]

[0039] in is the dimension of the key vector; specifically, Softmax transforms the attention log value into a probability distribution across all input positions, defined as:

[0040]

[0041] where z i represents the logarithmic value of attention corresponding to position i, and L is the total number of positions;

[0042] The outputs from all groups are concatenated along the channel dimension and projected to fuse the attended features:

[0043]

[0044] where Z X , Z Y ∈R B×L×C are the fused representations of modalities X and Y respectively;

[0045] Finally, Z X and Z Y are concatenated along the last dimension to form the final fused representation Z final ∈R B×L×2C :

[0046] Z final =Concat(Z X ,Z Y )。

[0047] Preferably, the three projections are query, key, value.

[0048] The beneficial effects of the present application are:

[0049] A lightweight tri-modal fusion network (TriLightNet) is introduced in the present application, which is an innovative lightweight tri-modal fusion network designed to integrate structural MRI, functional PET and clinical table data for predicting MCI to AD conversion. TriLightNet contains a hybrid backbone network that combines Kolmogorov-Arnold Networks and PoolFormer to achieve efficient feature extraction. In addition, it introduces a hybrid block attention module (Hybrid Block Attention Module) to capture subtle interactions between image and clinical features, and adopts multi-modal cascaded attention (MultiModal Cascaded Attention) to achieve progressive and efficient fusion across modalities. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 Fig. 1 is a schematic diagram of the overall architecture of the TriLightNet of the Alzheimer's disease progression mild cognitive impairment prediction method proposed in the present application;

[0051] Figure 2 Fig. 2 is a schematic diagram of the specific architecture of the multi-modal cascaded attention of the Alzheimer's disease progression mild cognitive impairment prediction method proposed in the present application. The inputs from the two modes are first divided into N groups for independent cross-attention. Then, the attention outputs of all groups are connected and projected. The two projected features are finally connected to form the final representation. DETAILED DESCRIPTION

[0052] The application will be further described below in connection with specific embodiments.

[0053] Embodiments

[0054] With reference to Figure 1 and Figure 2 , the application proposes the architecture of TriLightNet, which is a lightweight three-modal fusion network for predicting MCI to AD progression. The model input includes brain MRI, PET images and clinical table data.

[0055] The implementation steps of the Alzheimer's disease progression prediction method for mild cognitive impairment are as follows:

[0056] Step 1: MRI, Clinical Table and PET data are respectively input into the respective encoders to extract initial image features and table features.

[0057] Step 2: For the MRI and PET branches, the respective image features and table features are jointly input into the "mixed channel attention module" to calculate the mixed channel attention features.

[0058] Step 3: The mixed channel attention features obtained in step 2 are element-wise multiplied with the original image features of the corresponding branch to realize feature enhancement in the channel dimension.

[0059] Step 4: The enhanced features in step 3 are input into the "spatial attention module" to calculate the spatial attention features.

[0060] Step 5: The spatial attention features obtained in step 4 are again element-wise multiplied with the output results of step 3 to obtain refined fusion features F mri&cli and F pet&cli .

[0061] Step 6: The fused F mri&cli and F pet&cli features are input into the "multimodal cascading attention module" to perform deep cross-modal information interaction through multiple groups of cross-attention.

[0062] Step 7: The two feature streams output in step 6 are spliced and then sent into the "classification" module to obtain the final pMCI and sMCI prediction results.

[0063] The model structure is as follows:

[0064] The TriLightNet model structure diagram is shown in Figure 1, including (A) a hybrid block attention module and (D) a multi-modal cascaded attention. Wherein the hybrid block attention module consists of two sub-modules: (B) a hybrid channel attention module, responsible for adaptively assigning weights for channel-level interaction between table and image features; and (C) a spatial attention module, which captures and enhances local spatial dependencies.

[0065] (1) Hybrid Block Attention Module

[0066] A hybrid block attention module (HBAM) is introduced to effectively integrate multi-modal inputs, especially image and clinical features. As shown in Figure 1 (A), HBAM processes image and table features. Let denote the feature tensor extracted from the image encoder, and let denote the vectorized representation of the clinical table data. The goal is to refine F cli by leveraging complementary information from F img , thus enhancing the overall feature representation.

[0067] Hybrid Channel Attention Module (HCAM).

[0068] HCAM is an important component of HBAM, specifically designed to merge information from image and clinical features at the channel level. As shown in Figure 1 (B), HCAM refines the feature representation by leveraging the complementary data from the two modalities.

[0069] Initially, the image feature tensor F img and the clinical feature vector F cli are processed. Clinical features are embedded into a compatible format with image features through a linear embedding layer:

[0070] E cli = Embed(F cli ),

[0071] where Embed(·) is a linear layer that projects F cli from to

[0072] Subsequently, global average pooling (GAP) and global max pooling (GMP) are applied to the image feature tensor to produce two vectors that capture different aspects of the image features:

[0073]

[0074] These pooled vectors are passed through a shared MLP to generate channel attention weights:

[0075] M avg = MLP(F avg ), M max = MLP(F max ).

[0076] M avg , M max and E cli are combined through element-wise addition and normalization to produce the mixed channel attention map M hc :

[0077] M hc = σ(M avg + M max + E cli ),

[0078] where σ is the sigmoid function.

[0079] Finally, the mixed channel attention map M hc is applied to the original image feature tensor F img through channel-wise multiplication:

[0080]

[0081] where denotes channel-wise multiplication.

[0082] This design enables HCAM to effectively fuse information from image and table features, enhancing the feature representation by highlighting informative channels and suppressing less useful ones. The refined feature tensor F hc is then fed forward to the spatial attention module for further enhancement.

[0083] Spatial Attention Module.

[0084] After refinement at the channel level, we employ a spatial attention module to further emphasize important spatial regions, as shown in Figure 1 (C). The spatial attention feature M s ∈ R 1×D×H×W is created by concatenating the channel-level average pooling and max pooling features, which are then processed through a convolutional layer:

[0085] M s = σ(f conv (Concat(GAP(F hc ), GMP(F hc ))),

[0086] where σ is the sigmoid function. The final output feature map is computed as follows:

[0087]

[0088] HBAM provides a structured way to integrate tabular data into the attention mechanism, thus enhancing the robustness and semantic relevance of feature refinement in the mixed modality context.

[0089] (2) MultiModal Cascaded Attention

[0090] To capture the complex hierarchical dependency between modalities in multi-modal learning, a MultiModal Cascaded Attention (MMCA) module is designed, as shown in Figure 1 (D). The feature outputs F mri&cli and F pet&cli from the HBAM module are processed by the MMCA module to produce two modality-enhanced outputs. These outputs are then concatenated and passed through a classification head for the final prediction. Figure 2 The internal workflow and structure of MMCA are further disclosed.

[0091] Given two input modalities X and Y, each of shape B x L x C, where B is the batch size, L is the sequence length, and C is the number of channels, MMCA first reshapes these inputs to B x (L x C) and splits them into N groups, corresponding to the number of attention groups. For each group i in {1,..., N}, a cascaded computation occurs, where the feature representation of the current group is built upon the features of the previous groups, thus refining attention hierarchically. B×L×C B×C×L

[0092] For each group, three projections (query, key, value) are derived from the grouped features of both modalities. Let denote the query, key, and value for modality X, and let denote the query, key, and value for modality Y. Cross-attention is performed bidirectionally, where the features of one modality attend to the keys and values of the other modality:

[0093]

[0094] The attention operation contains a learnable position-aware bias term b i ∈ R L×L to enhance spatial sensitivity. The attention score is computed as follows:

[0095]

[0096] where​​ is the dimension of the key vector. Specifically, Softmax transforms the attention log value into a probability distribution across all input positions, defined as:

[0097]

[0098] where z i represents the logarithmic value of attention corresponding to position i, and L is the total number of positions.

[0099] The outputs from all groups are concatenated along the channel dimension and projected to fuse the attended features:

[0100]

[0101] where Z X , Z Y ∈R B×L×C are the fused representations of modalities X and Y respectively.

[0102] Finally, Z X and Z Y Concatenate along the last dimension to form the final fused representation Z final ∈R B×L×2C :

[0103] Z final =Concat(Z X , Z Y ).

[0104] The MMCA module offers significant advantages. Its cascaded structure facilitates progressive cross-modal feature fusion, with early attention groups guiding subsequent attention groups. Furthermore, its bidirectional design ensures that both modalities are symmetrically enhanced by information from the other, promoting balanced and robust fusion in multimodal settings.

[0105] (3) Loss function

[0106] In order to solve the problem of class imbalance in the dataset, we use the focal loss function. In the classification task, the cross entropy (CE) loss is traditionally used, which is defined as:

[0107] L ce =-(y log(p)+(1-y)log(1-p)),

[0108] Where y∈{0, 1} represents the true label and p∈[0, 1] represents the probability of predicting the positive class. This traditional approach assigns equal weight to positive and negative samples. Therefore, when faced with class imbalance, the model often favors the majority class.

[0109] To alleviate this problem, focal loss is adopted, which improves cross-entropy loss by introducing a dynamic modulation factor. This factor reduces the weight of easy-to-classify samples and focuses learning on challenging, misclassified instances. Focal loss L focal is expressed as:

[0110] L focal = -α(1-p) γ (ylog(p) + (1-y)log(1-p)),

[0111] where α ∈ [0, 1] is used to balance the importance between positive and negative samples, and γ > 0 as a focusing parameter, adjusts the degree of weight reduction for easy-to-classify samples.

[0112] (4) Results

[0113] Tables 1 and 2 show the results of the experiment.

[0114] (5) Conclusion

[0115] In the field of Alzheimer's disease cognitive impairment conversion prediction, existing three-modal fusion models often face problems such as high computational complexity and low performance. The present invention introduces TriLightNet, a novel and efficient three-modal fusion network, which integrates MRI, PET and clinical table data to predict cognitive decline in AD. Its key contributions include a hybrid KAN-PoolFormer backbone network for efficient table feature extraction, HBAM for enhancing imaging-clinical data interaction, and MMCA for progressive cross-modal fusion. Extensive experiments on the ADNI dataset demonstrate the superior classification performance of TriLightNet relative to the most advanced multi-modal methods, while significantly reducing the number of parameters and computational cost.

[0116] Experimental setup

[0117] 1. Dataset

[0118] This study utilized data from the Alzheimer's Disease Neuroimaging Initiative (ADNI), specifically extracted from the ADNI-1 and ADNI-2 datasets. To avoid duplication, subjects who were present in both datasets were excluded from ADNI-2. We used T1-weighted sMRI, FDG-PET images, and clinical data to divide the subjects into pMCI and sMCI groups.

[0119] Given the missing PET data and the class imbalance between pMCI and sMCI in both datasets, we focused on subjects with complete multi-modal data (including MRI, PET, and clinical features). Therefore, we combined ADNI-1 and ADNI-2 cohorts into a single dataset. The final dataset contains 512 subjects: 149 pMCI and 363 sMCI. This dataset was randomly split into training and testing sets with a 4:1 ratio. The training set includes 119 pMCI and 290 sMCI. The testing set contains 30 pMCI and 73 sMCI.

[0120] All MRI images were pre-processed, including intensity normalization, skull stripping, and normalization to Montreal Neurological Institute (MNI) space. FDG-PET images were also similarly processed by intensity normalization, normalization to MNI space, and registration with MRI images. For clinical data, seven key attributes were selected: age, gender, education, ApoE4 status, phosphorylated tau (P-tau 181), total tau (T-tau), and summary measures from 18F-FDG PET images. Finally, all images were resized to 96x128x96 resolution.

[0121] 2、Baseline and evaluation metrics

[0122] For evaluation, eight metrics were adopted to evaluate the classification performance and model efficiency: accuracy (ACC), sensitivity (SEN), precision (PRE), area under the receiver operating characteristic curve (AUROC), F1 score (F1), balanced accuracy (BACC), number of parameters (Params), and floating-point operations (FLOPs). Among them, balanced accuracy can provide a more fair evaluation for unbalanced positive and negative samples.

[0123] 3、Implementation details

[0124] All experiments were conducted using PyTorch version 2.6.0 and CUDA 11.8, running on a single NVIDIA V100 32GB GPU. A 5-fold cross-validation method was used to ensure robust evaluation of model performance. The model was trained for 200 epochs with a batch size of 8 for efficient data management. To optimize model parameters, the Adam optimizer was used with a fixed learning rate of 0.0001 for precise adjustments during training. In addition, a cosine learning rate scheduler (Cosine Learning Rate Scheduler) was implemented with hyperparameter T max set to 50. This setting allows dynamic adjustment of the learning rate throughout the training process, enhancing the model's adaptability.

[0125] 4. Experimental results

[0126] 4.1. Comparative experiments

[0127] Our framework was comprehensively evaluated against representative multi-modal methods in three fusion scenarios. For MRI and PET bimodal fusion, we compared with ResNet+Concat, ViT+Concat, nnMamba+Concat, Diamond, and MDL. For MRI and clinical data fusion, methods include VAPL and HyperFusionNet. In the three-modal setting that integrates MRI, PET, and clinical data, we compared with IMF, HFBSurv, ITCFN, and MultimodalADNet. Detailed comparison metrics are shown in Table 1.

[0128] TriLightNet achieved superior performance in multiple evaluation metrics, with an ACC of 81.25%, a PRE of 65.38%, an AUROC of 0.8146, and an F1 of 69.39%, all exceeding the baseline models. Although ResNet achieved the highest SEN of 75.04%, its ACC and PRE were lower, indicating its overall poor performance. In contrast, TriLightNet ranked second in sensitivity while maintaining the highest balanced accuracy, demonstrating its strong ability to effectively integrate multi-modal information and provide more reliable prediction results.

[0129] In terms of computational efficiency, TriLightNet has 17.405 million parameters and 10.517 billion FLOPs. Although models like MDL and MultimodalADNet have fewer parameters, their prediction performance is significantly worse. Among the three-modal methods, TriLightNet consistently outperforms competing methods, including HFBSurv (34.123 million parameters and 141.849 billion FLOPs) and IMF (67.843 million parameters and 70.925 billion FLOPs). This efficiency highlights that TriLightNet not only provides superior accuracy but also exhibits enhanced computational resource efficiency, making it particularly suitable for practical applications that require accurate and efficient prediction of cognitive impairment.

[0130] 4.2. Ablation study

[0131] An ablation study was conducted to evaluate the individual contributions of the HBAM and MMCA modules to the overall performance of the TriLightNet model. The results are summarized in Table 2, comparing the performance of the model under different configurations of these modules.

[0132] Initially, when the model was tested without the HBAM and MMCA modules, performance was relatively low: ACC of 73.50%, SEN of 48.37%, AUROC of 0.6266, and F1 of 49.13%. Next, we introduced only the HBAM module and removed the MMCA module. This configuration significantly enhanced model performance, raising ACC to 74.00%, SEN to 75.40%, AUROC to 0.7619, and F1 to 63.04%, demonstrating the effectiveness of HBAM in integrating image and clinical data. Subsequently, we removed the HBAM module and added the MMCA module. Results showed further improvement, with ACC reaching 78.50%, SEN of 72.94%, AUROC rising to 0.8011, and F1 rising to 67.07%. These findings demonstrate the superior capabilities of the MMCA module for multimodal fusion. Finally, when the HBAM and MMCA modules are combined, the model achieves the highest performance on all five metrics: ACC of 81.25%, PRE of 65.38%, and AUROC of 0.8146. F1 increases to 69.39%, and BACC reaches 79.06%, marking the best overall performance.

[0133] This indicates that the synergy between the two modules significantly enhances the classification ability of the model, highlighting the importance of incorporating HBAM and MMCA modules into the TriLightNet model.

[0134] Table 1: Comparative experimental results

[0135]

[0136]

[0137] Table 2: Ablation experiment results

[0138]

[0139] The meaning of proper nouns and English abbreviations

[0140]

[0141] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A method for predicting mild cognitive impairment in the progressive stage of Alzheimer's disease, comprising a lightweight trimodal fusion network model structure, wherein the lightweight trimodal fusion network model structure includes a hybrid block attention module and a multimodal cascade attention module, wherein the multimodal cascade attention module is connected to a classification module; wherein, The hybrid block attention module consists of two submodules: a hybrid channel attention module that is responsible for adaptively assigning weights to channel-level interactions between table and image features; and a spatial attention module that captures and enhances local spatial dependencies. It is characterized by the following steps: S1: MRI, Clinical Table, and PET data are input into their respective encoders to extract initial image features and table features; S2: For MRI and PET branches, their respective image features and table features are input into the mixed channel attention module to calculate the mixed channel attention features; S3: Multiply the mixed channel attention features obtained in step S2 by the original image features of the corresponding branch element by element to achieve feature enhancement in the channel dimension; S4: The features enhanced in step S3 are input into the spatial attention module to calculate the spatial attention features; S5: Multiply the spatial attention feature obtained in step S4 by the output result of step S3 again element by element to obtain the refined fusion feature F mri&cli and F pet&cli ; S6: F after fusion mri&cli and F pet&cli The features are input into the multimodal cascade attention module, and deep cross-modal information interaction is performed through multiple groups of cross attention; S7: The two feature streams output from step S6 are concatenated and then fed into the classification module to obtain the final pMCI and sMCI prediction results.

2. The method for predicting mild cognitive impairment in the progressive stage of Alzheimer's disease according to claim 1, characterized in that: The hybrid block attention module is used to integrate multimodal inputs. The hybrid block attention module processes image and table features and refines the feature tensor extracted from the image encoder by utilizing complementary information from the vectorized representation of clinical table data.

3. The method for predicting mild cognitive impairment in the progressive stage of Alzheimer's disease according to claim 1, characterized in that: A hybrid channel attention module is used to merge information from images and clinical features at the channel level; Initially, the image feature tensor F is processed img and clinical feature vector F cli ; Clinical features are embedded into a format compatible with image features through a linear embedding layer: E cli =Embed(F cli ), Where Embed(·) is a linear layer that transforms F cli from Projection to Subsequently, global average pooling and global max pooling are applied to the image feature tensor to produce two vectors that capture different aspects of the image features: These pooled vectors pass through a shared MLP to generate channel attention weights: M avg =MLP(F avg ),M max =MLP(F max )。 4. The method for predicting mild cognitive impairment in the progressive stage of Alzheimer's disease according to claim 3, characterized in that: M avg 、M max and E cli Combined with element-wise addition and normalization to produce a mixed channel attention map M hc : M hc =σ(M avg +M max +E cli ), Where σ is the sigmoid function; Finally, the mixed channel attention map M hc Applied to the original image feature tensor F through channel-wise multiplication img : in Represents channel-wise multiplication.

5. The method for predicting mild cognitive impairment in the progressive stage of Alzheimer's disease according to claim 1, wherein the spatial Attention feature M s ∈R 1×D×H×W It is created by concatenating channel-wise average pooling and max pooling features, and then processed through a convolutional layer: M s =σ(f conv (Concat(GAP(F hc ),GMP(F hc )))), Where σ is the sigmoid function; the final output feature map is calculated as follows:

6. The method for predicting mild cognitive impairment in the progressive stage of Alzheimer's disease according to claim 1, characterized in that: Feature output F from the hybrid block attention module mri&cli and F pet&cli Processed by a multimodal cascaded attention module to produce two modality-enhanced outputs; these outputs are then concatenated and passed through a classification head for the final prediction; Given two input modalities X and Y, each of shape R B×L×C , where B is the batch size, L is the sequence length, and L is the number of channels. The multimodal cascade attention module first reshapes these inputs into R B×C×L , and divide them into N groups, corresponding to the number of attention groups; for each group i in {1, ..., N}, a cascade computation occurs, where the feature representation of the current group is built on the features of the previous group, thereby hierarchically refining the attention.

7. The method for predicting mild cognitive impairment in the progressive stage of Alzheimer's disease according to claim 6, characterized in that: For each group, three projections are derived from the grouped features of the two modalities; let Represents the query, key and value of modality X, let represents the query, key, and value of modality Y; cross attention is performed bidirectionally, where features from one modality attend to the keys and values ​​of the other modality: The attention operation contains a learnable position-aware bias term b i ∈R L×L To enhance spatial sensitivity; the attention score is calculated as follows: in is the dimension of the key vector; specifically, Softmax transforms the attention log value into a probability distribution across all input positions, defined as: where z i represents the logarithmic value of attention corresponding to position i, and L is the total number of positions; The outputs from all groups are concatenated along the channel dimension and projected to fuse the attended features: where Z X , Z Y ∈R B×L×C are the fusion representations of modalities X and Y respectively; Finally, Z X and Z Y Concatenate along the last dimension to form the final fused representation Z final ∈R B×L×2C : From final =Concat(Z X ,From Y )。 8. The method for predicting mild cognitive impairment in the progressive stage of Alzheimer's disease according to claim 7, characterized in that: The three projections are query, key, and value.

Images