Artificial Intelligence Alzheimer's Disease 3D MRI Multimodal Classification Method and System

Abstract

The application discloses an artificial intelligence Alzheimer's disease 3D MRI multi-modal classification method and system. The method obtains three-dimensional structural magnetic resonance imaging data and structured clinical phenotype data of a subject, and pre-processes clinical variables and MRI body data; then, a three-dimensional residual network is used to extract image spatial anatomical features, a multi-layer perception machine is used to extract clinical phenotype features, and the two types of heterogeneous features are projected to a shared attention space; a multi-head attention mechanism, residual connection and layer normalization are used to model cross-modal dependency, and finally, cognitive normal and cognitive impaired or Alzheimer's disease risk classification results are output. The scheme avoids the problem of insufficient single modal information under small sample conditions, reduces the risk of information leakage and shortcut learning by excluding strong label related cognitive scale variables and introducing missing mode modeling, and can improve the stability, robustness and comparability of the auxiliary screening model.

Description

Technical Field

[0001] This invention relates to the fields of medical artificial intelligence and medical image computing technology, specifically to an artificial intelligence 3D MRI multimodal classification method and system for Alzheimer's disease. Background Technology

[0002] Alzheimer's disease is a chronic neurodegenerative disease characterized by a gradual decline in memory, cognitive function, and daily living abilities. Because the related pathological changes often progress long before obvious clinical symptoms appear, early identification of individuals with potential cognitive impairment and the establishment of stable auxiliary classification procedures are of great significance for clinical intervention, follow-up management, and resource allocation.

[0003] However, in real-world research and early applications, the number of available samples is usually limited, and models are prone to overfitting, large fluctuations in evaluation results, and insufficient generalization ability of a single modality. Therefore, a multimodal assisted classification technology is needed that can fully utilize multi-source information, reduce information leakage, and improve cross-fold stability under small sample conditions. Summary of the Invention

[0004] This disclosure is provided to briefly introduce the concepts, which will be described in detail in the subsequent Detailed Description section. This disclosure is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, embodiments of this application provide an artificial intelligence-based 3D MRI multimodal classification method for Alzheimer's disease, comprising the following steps:

[0006] Acquire medical multimodal data corresponding to the target object, wherein the medical multimodal data includes at least three-dimensional magnetic resonance imaging (MRI) data and structured clinical phenotype data corresponding to the three-dimensional magnetic resonance imaging data;

[0007] The three-dimensional magnetic resonance imaging data and the structured clinical phenotype data are preprocessed respectively to obtain standardized image input data and clinical input data;

[0008] The image input data is input into an image coding network to extract corresponding latent image features; the clinical input data is input into a clinical coding network to extract corresponding latent clinical features.

[0009] The latent features of the images and the latent features of the clinical subjects are mapped to a shared feature space to obtain image modality labels and clinical modality labels of a unified dimension. The image modality labels and the clinical modality labels are input into a multi-head attention fusion module, and the correlation between different modality labels is learned through a multi-head attention mechanism to obtain cross-modality fusion features.

[0010] The cross-modal fusion features are input into a classification network consisting of a random inactivation layer and a linear classification layer for classification processing, and the Alzheimer's disease auxiliary classification result is output.

[0011] Furthermore, the three-dimensional magnetic resonance imaging data includes at least three-dimensional volume data obtained from magnetic resonance imaging of brain structures;

[0012] The structured clinical phenotype data includes continuous clinical variables, categorical clinical variables, and missing indicator variables generated from missing data states;

[0013] The continuous clinical variables include at least age, years of education, brain volume index, and brain tissue volume index;

[0014] The categorical clinical variables include at least gender and other discrete clinical variables.

[0015] Furthermore, the preprocessing of the three-dimensional magnetic resonance imaging data includes: resampling the three-dimensional magnetic resonance imaging data to a preset spatial resolution; performing voxel intensity normalization on the resampled three-dimensional magnetic resonance imaging data; performing range truncation on the normalized voxel intensity; and converting the processed three-dimensional magnetic resonance imaging data into a single-channel floating-point tensor to obtain the normalized image input data.

[0016] Furthermore, the preprocessing of the structured clinical phenotype data includes: converting categorical clinical variables into numerical codes; generating corresponding missing value index variables for clinical variables with missing values; determining missing data imputation parameters based on specific data and using the missing data imputation parameters to impute missing data; and standardizing continuous clinical variables to obtain the standardized clinical input data.

[0017] Furthermore, the image coding network is a three-dimensional residual network, including: an initial three-dimensional convolutional layer, a pooling layer, multiple residual convolutional layers, and a global average pooling layer; the spatial structural features in the three-dimensional magnetic resonance imaging data are extracted through the image coding network to obtain the latent features of the image.

[0018] Furthermore, the clinical coding network includes at least one fully connected network layer; nonlinear feature mapping is performed on the clinical input data through the clinical coding network to obtain potential clinical features.

[0019] Furthermore, the shared feature space is used to transform latent features of different modalities to the same dimension;

[0020] The latent image features and the latent clinical features are respectively linearly mapped to obtain corresponding modality embeddings:

[0021]

[0022] Among them, h img h represents the latent features of an image. tab Indicates potential clinical features, z img Indicates image modality marker, z tab W represents a clinical modality marker. img W represents the image modal weights. tab Represents the clinical modality weights, b img Indicates image modal bias, b tab This indicates clinical modality bias.

[0023] Furthermore, the multi-head attention fusion module includes a multi-head attention computation unit, which is used to learn the correlation between different modal labels and generate cross-modal fusion features.

[0024] Furthermore, the multi-head attention fusion module also includes a residual connection unit and a layer normalization unit. The cross-modal fusion features are input into the classification network after being processed by residual connections and layer normalization.

[0025] Secondly, embodiments of this application provide an artificial intelligence-based 3D MRI multimodal classification system for Alzheimer's disease, applied to the classification method, including:

[0026] The data acquisition module is used to acquire medical multimodal data corresponding to the target object, wherein the medical multimodal data includes at least three-dimensional magnetic resonance imaging (MRI) data and structured clinical phenotype data corresponding to the three-dimensional magnetic resonance imaging (MRI) data;

[0027] The data preprocessing module is used to preprocess the three-dimensional magnetic resonance imaging data and the structured clinical phenotype data respectively to obtain standardized image input data and clinical input data.

[0028] The feature extraction module includes an image coding unit and a clinical coding unit. The image coding unit is used to receive the image input data and extract corresponding latent image features through an image coding network. The clinical coding unit is used to receive the clinical input data and extract corresponding latent clinical features through a clinical coding network.

[0029] The multimodal fusion module is used to map the latent image features and the latent clinical features to a shared feature space to obtain image modality labels and clinical modality labels of the same dimension. The image modality labels and the clinical modality labels are then input into the multi-head attention fusion module, which learns the correlation between different modality labels through the multi-head attention mechanism to generate cross-modal fusion features.

[0030] The classification module is used to receive the cross-modal fusion features and output the Alzheimer's disease auxiliary classification results through the classification network.

[0031] This invention constructs a dual-branch joint coding model of 3D magnetic resonance imaging (MRI) data and structured clinical phenotypic data, achieving joint representation and collaborative learning of spatial anatomical features from imaging and clinical phenotypic features. This fully utilizes complementary information from different data sources, improving the comprehensive utilization of multidimensional features in disease classification. By employing a multi-head attention fusion mechanism for small sample scenarios, image feature representations and clinical feature representations are mapped to a shared attention space, and feature interaction learning is performed based on cross-modal dependencies. Compared to the traditional direct feature concatenation method, this enhances the correlation modeling ability between different modal features, reduces classification bias caused by insufficient information from a single modality, and improves the stability and robustness of the model under limited sample conditions. Simultaneously, a clinical variable missing indicator mechanism is introduced, using the missing variable status in clinical data as an auxiliary feature input to the model. This allows the model to learn both clinical variable value information and missing pattern information simultaneously, avoiding information loss caused by relying solely on missing value imputation results for modeling, and improving the model's adaptability to the distribution characteristics of real clinical data. This invention excludes cognitive evaluation variables highly correlated with classification labels, such as the Mini-Mental State Examination (MMSE), during the model input stage. This reduces information leakage and model shortcut learning risks caused by the introduction of label-related variables, making the model more reliant on brain structural features and basic clinical phenotypic information reflected in 3D magnetic resonance imaging data for discrimination, thereby improving the reliability of model evaluation results and the credibility of clinical applications. Furthermore, it employs a unified hierarchical five-fold cross-validation strategy and consistent model training configuration to evaluate imaging monomodal models, clinical monomodal models, and multimodal fusion models, ensuring consistency and repeatability in performance comparisons between different models and improving the scientific rigor of the model validation process. In addition, by combining optimization strategies such as residual connectivity, layer normalization, dropout, weight decay, and cosine annealing learning rate scheduling, it improves the convergence stability during model training, reduces the risk of overfitting under small sample conditions, and thus enhances the model's generalization ability.

[0032] In summary, this invention can effectively integrate three-dimensional magnetic resonance imaging information with clinical phenotypic information under small sample medical data conditions, improve the accuracy, stability and reliability of disease auxiliary classification, and has good scalability, which can be further applied to intelligent auxiliary analysis scenarios that integrate multi-source medical data such as multicenter magnetic resonance imaging data, positron emission tomography (PET) data, genetic information, blood biomarkers and electronic medical record text. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of an artificial intelligence-based 3D MRI multimodal classification method for Alzheimer's disease according to an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the process of an artificial intelligence-based 3D MRI multimodal classification method for Alzheimer's disease according to another embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the process of an artificial intelligence Alzheimer's disease 3D MRI multimodal classification system according to an embodiment of the present invention;

[0036] Figure 4 The training loss and validation loss curves of the multimodal fusion model, the imaging monomodal model, and the clinical monomodal model in the five-fold cross-validation training process in the embodiments of the present invention are shown.

[0037] Figure 5 The validation accuracy curves of the multimodal fusion model, the imaging monomodal model, and the clinical monomodal model in the embodiments of the present invention are shown during the training process.

[0038] Figure 6 The curves showing the change in the area under the receiver operating characteristic (AUC) of the multimodal fusion model, the imaging monomodal model, and the clinical monomodal model during the optimization process in this embodiment of the invention.

[0039] Figure 7 The F1 score variation curves of the multimodal fusion model, the imaging monomodal model, and the clinical monomodal model in the embodiments of the present invention under small sample conditions are shown.

[0040] Figure 8 The box plot distribution of the optimal checkpoints of the multimodal fusion model, the imaging monomodal model, and the clinical monomodal model in this embodiment of the invention on the area under the receiver operating characteristic curve (AUC), accuracy (ACC), and F1 score. Detailed Implementation

[0041] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0042] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0043] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0044] It should be noted that the concepts of "first" and "second" mentioned in this application are used only to distinguish different devices, modules, or units, and are not used to limit the order of functions performed by these devices, modules, or units or their interdependencies. The modifications of "a" and "a plurality of" mentioned in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0045] The main English abbreviations involved in this invention have the following meanings: MRI stands for Magnetic Resonance Imaging; NIfTI stands for Neuroimaging Information Technology Planning Format; CDR stands for Clinical Dementia Rating Scale; MMSE stands for Mini-Mental State Examination; 3DResNet-18 stands for 18-layer three-dimensional residual network; MLP stands for Multilayer Perceptron; ReLU stands for Linear Rectifier Unit; Dropout stands for Random Inactivation; AdamW stands for Adaptive Moment Estimation Weight Decay Optimizer; AUC stands for Area Under the Receiver Operating Characteristic Curve; ACC stands for Precision; F1 stands for Harmonic Mean Squared Index of Precision and Recall; SE stands for Sensitivity; SP stands for Specificity; and PET stands for Positron Emission Tomography.

[0046] Example 1

[0047] This invention provides an artificial intelligence-based 3D MRI multimodal classification method for Alzheimer's disease based on the fusion of three-dimensional magnetic resonance imaging (MRI) data and structured clinical phenotypic data. Its overall process is similar to... Figures 1-1The technical route shown includes the following steps:

[0048] Step S1: Obtain medical multimodal data corresponding to the target subject, wherein the medical multimodal data includes at least three-dimensional magnetic resonance imaging (MRI) data and structured clinical phenotypic data; the MRI data is used to characterize the spatial structural features of the subject's brain, and the structured clinical phenotypic data includes the subject's basic information, clinical assessment indicators, and related physiological parameters, etc. A one-to-one correspondence between MRI data and clinical phenotypic data is established through subject identification information to ensure the consistency of data sources from different modalities;

[0049] Step S2 involves preprocessing the structured clinical phenotypic data. Specifically, categorical variables are coded according to their data type, and continuous variables are numerically converted. For data variables with missing values, corresponding missing indicators are generated to record the missing status of the variables. Subsequently, missing value imputation methods are used to complete the missing data, and the processed clinical features are standardized to form a unified format of clinical numerical feature vectors and standardized clinical input data.

[0050] The three-dimensional magnetic resonance imaging data is preprocessed. Specifically, the original MRI volume data is spatially resampled to unify the spatial resolution of MRI data from different subjects; the image intensity is standardized, and intensity truncation is used to reduce the impact of abnormal pixel values ​​on model training; further, the processed MRI data is converted into data tensors that conform to the input format of a three-dimensional convolutional neural network, i.e., standardized image input data.

[0051] Step S3 involves feature encoding of the preprocessed MRI data and clinical numerical features, respectively. Specifically, the MRI data is input into a three-dimensional residual network, and brain spatial structural features are extracted through a multi-layer three-dimensional convolutional structure to obtain image feature representations; the clinical numerical features are input into a multi-layer perceptron network, and clinical feature representations are obtained through nonlinear mapping.

[0052] Step S4 involves fusing the image feature representation and the clinical feature representation. Specifically, the image feature representation and the clinical feature representation are mapped to a shared feature space of a preset dimension, and a multi-head attention mechanism is used to calculate the correlation between features of different modalities, thereby realizing information interaction between image features and clinical features and obtaining a cross-modal fused feature representation.

[0053] Step S5: Perform classification prediction based on the fused feature representation. Specifically, input the fused features into a classification network that includes residual connections and layer normalization processing, and output the classification probability corresponding to the target cognitive state by fusing fully connected layers and classification layers.

[0054] The method may further include step S6, which involves training the multimodal assisted classification model. Specifically, the error between the predicted result and the true classification label is calculated using the cross-entropy loss function, and the model parameters are updated through backpropagation to complete end-to-end optimization training of the model.

[0055] Through the above steps, this invention constructs a complete multimodal assisted classification process consisting of data acquisition, data preprocessing, bi-branch feature encoding, cross-modal attention fusion, and classification prediction, realizing the joint analysis of three-dimensional magnetic resonance imaging information and structured clinical phenotype information under a unified model framework.

[0056] Example 2

[0057] Combination Figure 2 As shown, this embodiment of the invention provides an artificial intelligence-based 3D MRI multimodal classification method for Alzheimer's disease based on the fusion of three-dimensional magnetic resonance imaging (MRI) data and structured clinical phenotypic data, comprising the following steps:

[0058] Step S1: Input data includes three-dimensional structural magnetic resonance imaging (MRI) data and its paired structured clinical phenotype data, and establishes subject-level or examination-level pairing relationships to form image-clinical paired samples. This step ensures that different modalities correspond to the same subject or the same examination, reducing the risk of cross-sample mismatch.

[0059] As an example, the open access structural imaging research dataset OASIS-1 cross-sectional data used included 416 right-handed subjects aged 18 to 96 years, of whom 20 non-dementia subjects underwent repeat scans within 90 days; in model development, repeat scans were treated as independent image-clinical paired samples, ultimately resulting in 436 multimodal samples, including 336 cognitively normal samples and 100 cognitively impaired samples.

[0060] Labels can be set as binary labels, with normal cognition recorded as 0 and cognitive impairment or Alzheimer's disease-related risk status recorded as 1. The source of labels can be determined by clinical diagnosis, cognitive assessment, follow-up outcomes, or ethically approved data labeling processes.

[0061] Specifically, the binary labels are generated based on the Clinical Dementia Rating Scale (CDR) fields. For the i-th sample, its label is defined as:

[0062]

[0063] Where I(·) is the indicator function; when When =0, =0 indicates normal cognition; when > 0, =1 indicates a positive result for cognitive impairment or Alzheimer's disease-related risk.

[0064] Step S2: Clinical phenotypic data undergoes a multi-step standardization process: First, categorical variables such as gender are converted into binary numerical representations; second, for variables with missing values, such as years of education (Educ) and socioeconomic status (SES), corresponding binary missing value indicators are constructed to preserve missing value pattern information; third, median imputation parameters are determined based solely on training data, and missing values ​​are imputed to ensure the integrity of the feature matrix; finally, age, demographic variables, brain volume measurements, and other continuous variables are standardized using Z-scores to reduce the impact of different dimensions on model training. This process transforms clinical table fields from different sources into numerical vectors with uniform dimensions and dimensions that contain missing value pattern information, facilitating subsequent co-spatial fusion with potential imaging representations.

[0065] In a preferred embodiment, the model input excludes cognitive scale variables that are strongly correlated with classification labels, such as the Mini-Mental State Examination (MMSE), to reduce information leakage and the risk of shortcut learning. This design does not negate the clinical significance of cognitive scales, but rather avoids the model directly learning label proxy variables when constructing the auxiliary screening model, thereby improving the reliability of the evaluation results.

[0066] As an example, clinical input vector A 9-dimensional feature vector:

[0067]

[0068] Among them, Sex M For binary encoding of gender, Educ missing and SES missing These represent missing indicators for years of education and socioeconomic status, respectively. The Mini-Mental State Examination (MMSE) is not used as an input feature of the model.

[0069] Dynamic preprocessing was performed on 3D structural magnetic resonance imaging (MRI) data. All volumetric data in the Neuroimaging Information Technology Plan (NIfTI) format were first resampled to a uniform spatial resolution using trilinear interpolation to reduce input differences caused by varying voxel scales. Subsequently, voxel intensities were standardized using Z-scores, and outlier extreme values ​​were truncated to the range of [-5, 5] to mitigate the adverse effects of extreme intensity values ​​on convolutional network training. Finally, each MRI volumetric data was converted into a single-channel floating-point tensor with a tensor size of (1, D, H, W) to adapt to the 3D convolutional neural network. This preprocessing ensured consistency in image input across spatial scale, intensity range, and tensor format, thereby improving the stability of small-sample training.

[0070] Let the original MRI volume data be X, and the non-zero voxel region be Ω. Then the standardized and truncated volume data... It can be represented as:

[0071]

[0072] in, and Non-zero voxel regions The mean and standard deviation within the range, clip(·) indicates that the voxel intensity is truncated to [-5,5].

[0073] S3: The image coding branch employs an 18-layer 3D ResNet-18 structure as the volumetric data feature extractor. This network can use standard residual block settings and replace 2D convolutions with 3D convolutions to process volumetric data. The input MRI tensor sequentially passes through an initial 3D convolutional layer, a max pooling layer, and four residual convolutional stages, finally obtaining a 512-dimensional latent image representation through a global average pooling layer. This branch can learn spatial anatomical information of brain structures and is particularly suitable for processing 3D voxel relationships in structural MRI. Adjustments to the number of residual blocks, channels, or pooling methods in practical implementation can achieve an equivalent image coding network configuration.

[0074] To reduce the risk of overfitting under small sample conditions, it is preferable to introduce weight decay regularization and cosine annealing learning rate scheduling strategies during training, so that the model can achieve a smoother optimization process under limited sample conditions.

[0075] The clinical coding branch employs a multilayer perceptron (MLP) architecture. Preferably, the clinical input is a 9-dimensional vector containing demographic variables, brain volume measurements, and missing data indicators. The encoder consists of two fully connected layers with 256 and 128 hidden units, respectively. Each fully connected layer is followed by a rectified linear unit (ReLU) activation function and a dropout operation. The dropout rate can be set to 0.3 to enhance nonlinear representation capabilities and reduce overfitting. This branch ultimately outputs a 128-dimensional clinical latent representation.

[0076] Step S4: The 512-dimensional representations from the imaging branch and the 128-dimensional representations from the clinical branch are first projected onto a shared feature space using linear mappings to form comparable modality tokens or modality embeddings. Preferably, the shared feature space is set to 256 dimensions, which balances expressive power and the risk of overfitting in small samples. Optional dimensions such as 128, 384, or 512 dimensions can also be used depending on the data size and computational resources. This step allows heterogeneous modalities to be compared and interact within the same feature space. Subsequently, a multi-head attention mechanism is used to model the interaction relationships between different modalities, enabling the model to adaptively learn the complementary relationships between imaging and clinical information based on sample features.

[0077] As an example, the number of attention heads in the multi-head attention module is set to 4 to learn the association between images and clinical representations simultaneously from multiple subspaces. In other implementations, the number of attention heads can be set to optional values ​​such as 2, 4, or 8, determined in conjunction with validation set performance, training stability, and computational cost. The attention output undergoes residual connectivity and layer normalization to improve the stability of deep fusion training; it is then processed through a fusion fully connected layer and dropout regularization to obtain the fused representation used for final classification. The entire network is optimized end-to-end using the cross-entropy loss function.

[0078] Specifically, image representation and clinical manifestations Projected onto the shared attention dimension via linear mapping respectively:

[0079]

[0080] Among them, h img h represents the latent features of an image. tab Indicates potential clinical features, z img Indicates image modality marker, z tab W represents a clinical modality marker. img W represents the image modal weights. tab Represents the clinical modality weights, b img Indicates image modal bias, b tab Indicates clinical modality bias;

[0081] Stacking the two modal tokens yields:

[0082]

[0083] Then, T is input into the multi-head attention module simultaneously as the query, key, and value:

[0084]

[0085] Final fusion characterization Defined as:

[0086]

[0087] Here, A+T represents residual connection, LayerNorm represents layer normalization, and mean(·) represents mean aggregation in the modality dimension. This processing allows the attention module to retain both the original single-modal information and introduce cross-modal interaction information, thus avoiding the problem that simple concatenation cannot explicitly model modality dependencies.

[0088] Step S5: Input the obtained fusion features through random inactivation (Dropout) and a linear classification layer to output a binary classification result, obtain the risk probability of normal cognition / cognitive impairment, or output an Alzheimer's disease-related risk score, which can be used for auxiliary classification or auxiliary screening of Alzheimer's disease.

[0089] To ensure fairness in evaluation among different models, this embodiment of the invention trains and evaluates the image-only model, the clinical-only model, and the multi-modal fusion model under the same data partitioning, the same training configuration, and a unified stratified five-fold cross-validation protocol. Evaluation metrics include the area under the receiver operating characteristic curve (AUC), accuracy (ACC), F1 score, sensitivity (SE), and specificity (SP).

[0090] Preferably, the optimal model checkpoint is saved at each fold, and the distribution and stability of each indicator are evaluated based on the optimal checkpoint. By comparing the mean and standard deviation across folds, the robustness of the model under changes in small sample data partitioning can be determined.

[0091] Example 3

[0092] like Figure 3 As shown in the figure, an artificial intelligence Alzheimer's disease 3D MRI multimodal classification system based on the fusion of three-dimensional magnetic resonance imaging data and structured clinical phenotypic data is provided in this embodiment of the invention. It includes a data acquisition module, a data preprocessing module, a feature extraction module, a multimodal fusion module and a classification output module.

[0093] The data acquisition module 101 is used to acquire medical multimodal data corresponding to the target object, wherein the medical multimodal data includes at least three-dimensional magnetic resonance imaging (MRI) data and structured clinical phenotype data corresponding to the three-dimensional magnetic resonance imaging (MRI) data;

[0094] The data preprocessing module 102 is used to preprocess the three-dimensional magnetic resonance imaging data and the structured clinical phenotype data respectively to obtain standardized image input data and clinical input data.

[0095] The feature extraction module 103 includes an image coding unit and a clinical coding unit. The image coding unit is used to receive the image input data and extract corresponding latent image features through an image coding network. The clinical coding unit is used to receive the clinical input data and extract corresponding latent clinical features through a clinical coding network.

[0096] The multimodal fusion module 104 is used to map the image latent features and the clinical latent features to a shared feature space to obtain image modality labels and clinical modality labels of the same dimension, and input the image modality labels and the clinical modality labels into the multi-head attention fusion module to learn the correlation between different modality labels through the multi-head attention mechanism to generate cross-modal fusion features;

[0097] The classification module 105 is used to receive the cross-modal fusion features and output the Alzheimer's disease auxiliary classification result through the classification network.

[0098] The system may also include an evaluation module 106 for calculating the area under the receiver operating characteristic curve (AUC), accuracy (ACC), F1 score, sensitivity (SE), and specificity (SP) during the training phase.

[0099] The multimodal fusion model described in this embodiment of the invention is the core processing unit in the artificial intelligence Alzheimer's disease 3D MRI multimodal classification system that fuses 3D magnetic resonance imaging data with structured clinical phenotypic data. During the model training phase, the image monomodal model, clinical monomodal model, and multimodal fusion model can use the same optimizer, learning rate scheduling strategy, weight decay, batch size, early stopping, or optimal checkpoint selection rules. Preferably, the AdamW optimizer, weight decay, and cosine annealing learning rate scheduling strategy are used, and hierarchical five-fold cross-validation is employed to maintain a relatively consistent proportion of different classes in each fold, thereby reducing the impact of class distribution differences on the evaluation results.

[0100] The loss function can be cross-entropy loss, and the model output is a binary classification probability. For imbalanced class scenarios, class weighting, resampling, or threshold adjustment strategies can be further employed, but these strategies do not affect the core of the multimodal attention fusion of this invention.

[0101] Regarding parameters such as the shared attention space dimension and the number of attention heads, this embodiment uses a 256-dimensional shared attention space and 4 attention heads as the preferred configuration. This configuration is used to balance cross-modal expressive power, computational complexity, and training stability under small sample conditions. In actual implementation, other dimensions or the number of heads can be selected according to the sample size, number of modalities, or hardware resources without changing the multimodal attention fusion concept of this invention.

[0102] The core training procedure uses a single entry point to achieve unified training and comparison of the three types of models. The program predefines three model names: image_only, tabular_only, and multimodal_fusion. Image_only represents an image-only monomodal model using only 3D MRI data; tabular_only represents a clinical monomodal model using only structured clinical phenotypic data; and multimodal_fusion represents a multimodal attention fusion model using both 3D MRI and clinical features. During training, the model type to be trained is specified through model selection parameters. If no specific model is specified, the three types of models are trained sequentially under the same data partition and evaluation protocol to ensure a consistent basis for comparing the technical performance of the models.

[0103] During the data construction phase, the program reads the OASIS cross-sectional clinical table and establishes pairing relationships between examination numbers and Neuroimaging Information Technology Plan Format (NIfTI) image files. Labels are generated from the Clinical Dementia Rating Scale (CDR) field; samples with CDR = 0 are categorized as class 0, and samples with CDR > 0 are categorized as class 1. Clinical input uses a 9-dimensional feature vector, including age, years of education (Educ), socioeconomic status (SES), estimated intracranial volume (eTIV), normalized whole brain volume (nWBV), atlas scaling factor (ASF), male sex code (Sex_M), missing years of education indicator (Educ_missing), and missing socioeconomic status indicator (SES_missing). Educ_missing and SES_missing are used to store missing patterns of years of education and socioeconomic status; the Mini-Mental State Examination (MMSE) is not used as a model input feature.

[0104] In the multimodal fusion model, the imaging branch first obtains latent image representations through a 3D residual network, while the clinical branch obtains latent clinical representations through a multilayer perceptron. Both representations are projected onto the same shared attention dimension via linear mapping, forming two modal tokens. Subsequently, this sequence of modal tokens is simultaneously used as the query, key, and value input to a multi-head attention module, yielding a cross-modal attention output. The attention output is residually concatenated with the original modal tokens and then subjected to layer normalization. Mean aggregation is then performed along the modal dimension to obtain the fused representation. Finally, a binary classification result is output after dropout and a linear classification layer. This fusion process can explicitly express the dependency between anatomical image representations and clinical phenotypic variables.

[0105] During the training phase, all three models employed the same hierarchical K-fold cross-validation partitioning. Within each fold, only the clinical missing filler and normalizer were fitted using the current training fold, and then applied to the corresponding validation fold to avoid information leakage from the validation set. The training process employed a cross-entropy loss function, an adaptive moment estimation weight decay optimizer (AdamW), weight decay, cosine annealing learning rate scheduling, and a strategy for preserving the optimal validation AUC checkpoint. In the evaluation phase, the area under the receiver operating characteristic (AUC), accuracy (ACC), F1 score, sensitivity (SE), and specificity (SP) were calculated uniformly, with the mean and standard deviation of the optimal checkpoint for each fold used as the final result.

[0106] The following embodiments are used to illustrate the technical effects of the present invention and do not constitute a limitation on the scope of protection. The appendix experiments used the OASIS-1 cross-sectional cohort as the data source, constructing 436 multimodal samples, including 336 cognitively normal samples and 100 cognitively impaired samples. Model comparisons included imaging monomodal, clinical monomodal, and multimodal attention fusion models, and a unified hierarchical 5-fold cross-validation was employed.

[0107] Table 1: Comparison of classification performance of imaging monomodal, clinical monomodal, and multimodal fusion models under 5-fold cross-validation.

[0108]

[0109] Note: AUC is the area under the receiver operating characteristic curve, ACC is accuracy, SE is sensitivity, and SP is specificity.

[0110] From Table 1 and in combination Figures 4-8As shown, the multimodal fusion model achieved the highest mean receiver operating characteristic (AUC) of 92.76%, accuracy of 87.84%, and F1 score of 76.23%, all of which are superior to the imaging monomodal model and the clinical monomodal model. Meanwhile, the standard deviation of accuracy for the multimodal model was 2.53, lower than that of the imaging monomodal and clinical monomodal models, indicating better stability under different data partitioning.

[0111] The experimental curves further show that all three types of models can converge, but the imaging monomodal model has more obvious fluctuations in validation loss in the early and mid-stages of training, the clinical monomodal model converges faster but still has some oscillations in the later stage; the multimodal fusion model shows a narrower uncertainty range and a more stable performance trajectory in the later training stage.

[0112] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An artificial intelligence-based 3D MRI multimodal classification method for Alzheimer's disease, characterized in that, Includes the following steps: Acquire medical multimodal data corresponding to the target object, wherein the medical multimodal data includes at least three-dimensional magnetic resonance imaging (MRI) data and structured clinical phenotype data corresponding to the three-dimensional magnetic resonance imaging data; The three-dimensional magnetic resonance imaging data and the structured clinical phenotype data are preprocessed respectively to obtain standardized image input data and clinical input data; The image input data is input into an image coding network to extract corresponding latent image features; the clinical input data is input into a clinical coding network to extract corresponding latent clinical features. The latent features of the images and the latent features of the clinical subjects are mapped to a shared feature space to obtain image modality labels and clinical modality labels of a unified dimension. The image modality labels and the clinical modality labels are input into a multi-head attention fusion module, and the correlation between different modality labels is learned through a multi-head attention mechanism to obtain cross-modality fusion features. The cross-modal fusion features are input into a classification network for classification processing, and Alzheimer's disease auxiliary classification results are output.

2. The method according to claim 1, characterized in that, The three-dimensional magnetic resonance imaging data includes at least three-dimensional volume data obtained from magnetic resonance imaging of brain structures; The structured clinical phenotype data includes at least continuous clinical variables, categorical clinical variables, and missing indicator variables generated from missing data states; The continuous clinical variables include at least age, years of education, brain volume index, and brain tissue volume index; The categorical clinical variables include at least gender and other discrete clinical variables.

3. The method according to claim 1 or 2, characterized in that, The preprocessing of the three-dimensional magnetic resonance imaging data includes: The three-dimensional magnetic resonance imaging data is resampled to a preset spatial resolution; the resampled three-dimensional magnetic resonance imaging data is subjected to voxel intensity normalization; the normalized voxel intensity is subjected to range truncation; the processed three-dimensional magnetic resonance imaging data is converted into a single-channel floating-point tensor to obtain the normalized image input data.

4. The method according to claim 3, characterized in that, The preprocessing of the structured clinical phenotype data includes: Categorical clinical variables are converted into numerical codes; corresponding missing value index variables are generated for clinical variables with missing values; missing data imputation parameters are determined based on specific data, and the missing data is imputed using the missing data imputation parameters; continuous clinical variables are standardized to obtain the standardized clinical input data.

5. The method according to claim 4, characterized in that, The image coding network is a three-dimensional residual network, including: an initial three-dimensional convolutional layer, a pooling layer, multiple residual convolutional layers, and a global average pooling layer; the spatial structural features in the three-dimensional magnetic resonance imaging data are extracted through the image coding network to obtain the latent features of the image.

6. The method according to claim 5, characterized in that, The clinical coding network includes at least one fully connected layer; the clinical input data is nonlinearly mapped through the clinical coding network to obtain potential clinical features.

7. The method according to claim 6, characterized in that, The shared feature space is used to transform latent features of different modalities to the same dimension; The latent image features and the latent clinical features are respectively linearly mapped to obtain corresponding modality embeddings: Among them, h img h represents the latent features of an image. tab Indicates potential clinical features, z img Indicates image modality marker, z tab W represents a clinical modality marker. img W represents the image modal weights. tab Represents the clinical modality weights, b img Indicates image modal bias, b tab This indicates clinical modality bias.

8. The method according to claim 7, characterized in that, The multi-head attention fusion module includes a multi-head attention computation unit, which is used to learn the correlation between different modal labels and generate cross-modal fusion features.

9. The method according to claim 8, characterized in that, The multi-head attention fusion module further includes a residual connection unit and a layer normalization unit. The cross-modal fusion features are input into the classification network after being processed by residual connections and layer normalization.

10. An artificial intelligence-based 3D MRI multimodal classification system for Alzheimer's disease, characterized in that, include: The data acquisition module is used to acquire medical multimodal data corresponding to the target object, wherein the medical multimodal data includes at least three-dimensional magnetic resonance imaging (MRI) data and structured clinical phenotype data corresponding to the three-dimensional magnetic resonance imaging (MRI) data; The data preprocessing module is used to preprocess the three-dimensional magnetic resonance imaging data and the structured clinical phenotype data respectively to obtain standardized image input data and clinical input data. The feature extraction module includes an image coding unit and a clinical coding unit. The image coding unit is used to receive the image input data and extract corresponding latent image features through an image coding network. The clinical coding unit is used to receive the clinical input data and extract corresponding latent clinical features through a clinical coding network. The multimodal fusion module is used to map the latent image features and the latent clinical features to a shared feature space to obtain image modality labels and clinical modality labels of the same dimension. The image modality labels and the clinical modality labels are then input into the multi-head attention fusion module, which learns the correlation between different modality labels through the multi-head attention mechanism to generate cross-modal fusion features. The classification module is used to receive the cross-modal fusion features and output the Alzheimer's disease auxiliary classification results through the classification network.

Images