Long-term individual AD risk prediction model and prediction method based on multi-scale deep learning

Abstract

The invention belongs to the technical field of brain image calculation and medical auxiliary research, and particularly discloses a long-term individual AD risk prediction model and prediction method based on multi-scale deep learning, and the prediction model comprises an MRI data acquisition and processing module, an MRI feature extraction module and a risk prediction module. According to the long-term individual AD risk prediction model and prediction method based on multi-scale deep learning, a novel multi-scale model is provided, the model predicts the risk of converting MCI into AD on the individual level, the conversion risk can be obtained every 6 months within 60 months on the basis of baseline diagnosis data of a subject, and the prediction efficiency is improved. Therefore, the MCI conversion risk condition is effectively tracked, and the conversion time is estimated.

Description

Technical Field

[0001] The present invention belongs to the technical field of brain imaging computing and medical assistance research, and specifically relates to a long-term individual AD risk prediction model and prediction method based on multi-scale deep learning. Background Art

[0002] Alzheimer's disease (AD) is a neurodegenerative disease characterized by the gradual impairment of cognitive ability and memory function. It is estimated that by 2050, approximately 152 million people worldwide will suffer from Alzheimer's disease. Because the disease is irreversible, its treatment remains challenging, and existing treatments mainly focus on symptom control and slowing the progression of the disease. Mild cognitive impairment (MCI) is the early stage of Alzheimer's disease, and approximately 10-15% of MCI patients transform into Alzheimer's disease each year. Therefore, early diagnosis and intervention at the MCI stage are crucial to reducing the incidence of Alzheimer's disease. Predicting whether MCI patients will continue to decline in cognitive ability and progress to Alzheimer's disease is one of the current research focuses.

[0003] Currently, there are two main methods for studying the conversion of MCI to AD. The first method regards it as a classification problem, based on machine learning (ML) or deep learning (DL) models, which aims to predict whether the subject is classified as an AD patient by inputting a feature set at a specific time point. The study used plasma phosphorylated tau in combination with other measurable markers to predict the progression of patients with cognitive decline to AD within 2, 4 or even 6 years based on a logistic regression model. One study constructed a multimodal recurrent neural network to predict the conversion of MCI at multiple time points using longitudinal data. Another study combined interaction effects and multimodal DL models to predict the long-term progression of AD. These studies focused on classification but did not provide a more comprehensive quantitative risk assessment or dynamic disease progression risk analysis.

[0004] The second approach considers the analysis of AD conversion risk as survival analysis, among which the Cox proportional hazards (CPH) model is the most commonly used, which aims to analyze the impact of multiple predetermined variables on patient survival. The two key concepts of survival analysis are event occurrence and survival time. Event occurrence refers to whether MCI converts to AD, while survival time is the time from baseline diagnosis to the last follow-up of the patient's pMCI conversion (MCI population that will develop AD in the future) or sMCI (MCI population that will not develop AD) (for example, the patient was lost to follow-up before the conversion event). The study quantified the impact of sex, MMSE and brain age on the risk of conversion through Cox survival analysis, and created a visual ruler to provide risk probabilities at the individual level. The CPH model allows long-term and quantitative risk prediction, but assumes linear constraints and cannot capture the nonlinear relationship between input covariates and risk. In addition, the input covariates must meet the proportional hazards (PH) assumption, that is, the hazard ratio is assumed to remain constant over time, which limits the flexibility of capturing changes in the hazard ratio over different time periods.

[0005] In recent years, deep learning-based survival analysis methods have gradually emerged, aiming to solve the linear and proportional constraints of the Cox proportional hazard model. The study constructed a DM-GNN model for personalized cancer prognosis prediction using histopathological images. The study proposed a MultiSurv model, which is an end-to-end multimodal discrete-time prediction framework based on deep learning, for predicting long-term all-cancer survival. There are also studies that introduced a novel DeepHit model that can capture the joint distribution of survival time and events and simultaneously deal with competing risks, ignoring the assumptions underlying the data. Subsequently, the authors extended the study to dynamic DeepHit and performed dynamic survival analysis to distinguish the associated competing death risks occurring in patients with cystic fibrosis. In addition, there are studies that have developed deep learning-based survival analysis models for predicting the onset of AD in MCI patients, but the study only showed the survival pattern at the group level and ignored personalized clinical decisions. The literature on predicting the probability of individual AD risk is still relatively scarce.

[0006] Therefore, this field needs to develop a long-term individual AD risk prediction model and prediction method based on multi-scale deep learning that can effectively solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide a long-term individual AD risk prediction model and prediction method based on multi-scale deep learning. The model predicts the risk of MCI converting to AD at the individual level. Based on the baseline diagnostic data of the subjects, the conversion risk can be obtained every 6 months within 60 months, thereby effectively tracking the MCI conversion risk and estimating the conversion time.

[0008] To achieve the above objectives, the present invention provides a long-term individual AD risk prediction model based on multi-scale deep learning, including a magnetic resonance imaging (MRI) data acquisition and processing module, a magnetic resonance imaging (MRI) feature extraction module, and a risk prediction module; the magnetic resonance imaging (MRI) feature extraction module consists of four 3D convolutional layers and two 3D fully connected layers.

[0009] Preferably, the magnetic resonance imaging (MRI) data acquisition and processing module obtains the baseline T1-weighted structural magnetic resonance imaging MRI in the Alzheimer's Disease Neuroimaging Project (ADNI) public data set, and the original T1-weighted magnetic resonance imaging (MRI) scan images collected in the Alzheimer's Disease Neuroimaging Project (ADNI) public data set are all in DICOM format and converted into NIFTI format through Matlab, and the preprocessing is completed using the Matlab-based SPM12 and CAT12 brain image processing software packages.

[0010] Preferably, the magnetic resonance imaging (MRI) data acquisition and processing module obtains the baseline T1-weighted structural magnetic resonance imaging (MRI) in the Alzheimer's Disease Neuroimaging Project (ADNI) public data set, including several healthy person NC samples, Alzheimer's disease AD samples, stable mild cognitive impairment (sMCI) samples, and progressive mild cognitive impairment (pMCI) samples;

[0011] Each MCI sample must contain corresponding structural magnetic resonance imaging (sMRI) data and complete clinical information, including age, gender, Mini-Mental State Examination (MMSE) score, and APOEε4 genotype A1 and genotype A2;

[0012] For the MCI samples, only those with at least two follow-up records were included.

[0013] Preferably, the risk prediction module consists of two sets of fully connected layers, residual connections and a multi-head attention mechanism;

[0014] The two groups of fully connected layers are FCLayersA and FCLayersB respectively.

[0015] FCLayersA contains two fully connected layers with 128 and 256 nodes respectively; FCLayersB contains two fully connected layers with 128 and 64 nodes respectively; the extracted brain structural features, clinical information and gray matter volume constitute multi-scale data and are passed to FCLayersA as input x.

[0016] Preferably, the complex dependencies between elements in the feature sequence are captured and the multi-head attention mechanism formula is introduced.

[0017]

[0018] Among them, Q, K and V represent the features of query, key and value respectively, d k represents the embedding dimension, K T represents the transpose of K;

[0019] The multi-head attention mechanism calculates the attention weights through the formula to preliminarily capture the potential representation of covariates related to the risk of Alzheimer's disease conversion; FCLayersB converts the vector f A (x) is combined with the covariate x as input z, directly learning the residual and maintaining data integrity; the final output vector f B (x) is a probability distribution y = [y1,y2,…,y T ], indicating that for a given mild cognitive impairment subject x i When the covariate x is converted into the estimated risk probability of Alzheimer's disease occurring at time t, that is, Among them, x i is the subject characteristic, and e represents the occurrence of the event.

[0020] A prediction method based on a multi-scale deep learning long-term individual AD risk prediction model, comprising the following steps:

[0021] Step S1, the MRI data acquisition and processing module performs skull removal processing on the initial brain MRI neuroimage and registers it to the MNI152 standard space;

[0022] Step S2, segmenting the image into gray matter, white matter, and cerebrospinal fluid according to the Hammers brain atlas template; obtaining the corresponding gray matter and white matter segmentation images, as well as the volume data of gray matter, white matter, and cerebrospinal fluid of 83 brain regions of the Hammers atlas through segmentation processing;

[0023] Step S3, further correcting the segmented image by deviation correction for intensity inhomogeneity and modulating according to volume changes caused by spatial registration; after preprocessing, images with a quality lower than 80% are excluded;

[0024] Step S4, smoothing the image using an 8 mm full width half maximum smoothing kernel;

[0025] Step S5, the MRI feature extraction module classifies the images of normal cognition and Alzheimer's disease patients through the pre-trained convolutional neural network healthy person NCN model;

[0026] Step S6: fine-tune using images of patients with mild cognitive impairment, and use the output before the last fully connected layer as the extracted brain structure features;

[0027] Step S7, finally, the most important decision-making area of ​​the model in the classification process, i.e., the brain area of ​​interest, is visualized by using the gradient weighted class activation mapping technique;

[0028] Step S8, the risk prediction module uses the gray matter volume of the brain region and the MRI image features obtained by the MRI data acquisition and processing module and the MRI feature extraction module; combined with clinical information, the three are input into the risk prediction model as risk variables;

[0029] Step S9: predict the specific risk probability of each mild cognitive impairment (MCI) patient converting to Alzheimer's disease (AD) at every six-month node in the next five years.

[0030] Preferably, in the magnetic resonance imaging (MRI) data acquisition and processing module, each sample is represented by a triple (x, s, e), where x is a high-dimensional vector representing all features of the sample, s is the survival time, and e is the occurrence of the event;

[0031] When e = 0, it means that no conversion was observed in this sample and the patient was lost to follow-up, and when e = 1, it means that the patient converted to Alzheimer's disease.

[0032] Preferably, the high-dimensional vector x of all features of each sample consists of three types of features, specifically,

[0033] (1) Brain structural features extracted by the MRI feature extraction module;

[0034] (2) clinical information of each sample, including gender, age, Mini-Mental State Examination (MMSE) score, and APOEA1 and APOE A2 gene information;

[0035] (3) gray matter volumes of 83 brain regions of interest obtained during the data preprocessing stage;

[0036] Among them, the sample data set is D = {(x i ,s i ,e i )} i＝1 Represents the mild cognitive impairment (MCI) sample used for risk prediction / survival analysis, with a total number of N.

[0037] Preferably, a five-fold cross validation method is used to evaluate the performance of the pre-trained convolutional neural network healthy person NCN model in the classification of Alzheimer's disease AD and healthy person NC;

[0038] a. The dataset includes several Alzheimer's disease AD samples and healthy human NC samples; the model training is carried out for 30 cycles, the initial learning rate is 1e-6, and it is dynamically adjusted during the training process, decreasing by 10% every 10 cycles; finally, all Alzheimer's disease AD and healthy human NC samples are used to train the final convolutional neural network healthy human NCN model, which is a pre-trained model;

[0039] b. The output of the first fully connected layer is used as the extracted brain structure feature; the gradient weighted class activation mapping technique is used to visualize the image area that contributes most significantly to the classification result by calculating the weighted gradient in the convolutional neural network healthy person NCN and back-propagating it to the input image;

[0040] c. Using a five-fold cross-validation strategy, the pre-trained convolutional neural network (NCN) of healthy subjects was fine-tuned using gray matter images of stable mild cognitive impairment (sMCI) samples and progressive mild cognitive impairment (pMCI) samples;

[0041] (1) In each fold, the pre-trained model is fine-tuned using the training set divided by the fold to capture the specific features of the MCI data by adjusting the weights;

[0042] (2) Then input the test set of the fold into the model;

[0043] (3) Extract the fully connected layer before the classification layer and use the output as the feature representation of each sample in the test set.

[0044] Preferably, the risk prediction module uses a five-fold cross-validation strategy to train and evaluate model performance;

[0045] a. The input of the risk prediction model includes a 256-dimensional magnetic resonance imaging (MRI) brain structure feature vector, a 5-dimensional clinical information vector, and an 83-dimensional brain gray matter volume vector, which together form a 344-dimensional multi-scale feature representation;

[0046] b. Each fully connected layer uses the ReLU activation function;

[0047] c. The module is trained for a total of 1500 cycles, with an initial learning rate of 1e-4, and dynamically adjusted to half of the original learning rate every 500 cycles.

[0048] The present invention adopts the above-mentioned multi-scale deep learning-based long-term individual AD risk prediction model and prediction method, and the beneficial effects are as follows:

[0049] The model in the present invention can quantify the risk probability of each mild cognitive impairment patient converting to Alzheimer's disease at any time in the next five years starting from the baseline time; according to investigations, there are currently few studies on quantitative prediction of the risk of MCI patients converting to AD, and most studies can only identify high-risk populations. Some studies have predicted the conversion probability of AD based on the Cox survival analysis model, but the Cox model has linear and proportional constraints, which limit its application in clinical practice; however, the deep learning-based survival analysis model proposed in the present invention is not subject to constraints and is more flexible in application compared to existing methods.

[0050] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 1 is a model structure diagram of an embodiment of a multi-scale deep learning long-term individual AD risk prediction model and prediction method of the present invention; wherein (a) is a structure diagram of an MRI feature extraction module; (b) is a structure diagram of a risk prediction module;

[0052] Figure 2 The visualization diagram of the region of interest (ROI) in three different sections of the embodiment of the multi-scale deep learning long-term individual AD risk prediction model and prediction method of the present invention; wherein (a) is the axial section; (b) is the sagittal section; (c) is the coronal section;

[0053] Figure 3 Box plots of four models of the embodiment of the multi-scale deep learning long-term individual AD risk prediction model and prediction method of the present invention; wherein, (a) is 6 months; (b) is 12 months; (c) is 18 months; (d) is 24 months; (e) is 30 months; (f) is 36 months; (g) is 42 months; (h) is 48 months; (i) is 54 months; (j) is 60 months;

[0054] Figure 4 It is the Kaplan-Meier survival curve of the long-term individual AD risk prediction model and prediction method embodiment based on multi-scale deep learning of the present invention;

[0055] Figure 5 It is a Bland-Altman diagram of the actual conversion time and the predicted conversion time of an MCI individual to AD in an embodiment of the multi-scale deep learning long-term individual AD risk prediction model and prediction method of the present invention;

[0056] Figure 6 This is a relationship diagram between the actual conversion time and the predicted conversion time of the long-term individual AD risk prediction model and prediction method embodiment based on multi-scale deep learning of the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0058] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0059] Example

[0060] A long-term individual AD risk prediction model and prediction method based on multi-scale deep learning includes a magnetic resonance imaging (MRI) data acquisition and processing module, a magnetic resonance imaging (MRI) feature extraction module, and a risk prediction module.

[0061] The MRI data acquisition and processing module realizes the brain MRI image acquisition and processing functions, preprocessing and enhancing the data. The MRI data acquisition and processing module obtains the baseline T1-weighted structural MRI in the Alzheimer's Disease Neuroimaging Initiative (ADNI) public dataset, including 700 healthy subjects (NC) samples, 766 Alzheimer's disease (AD) samples, 731 stable mild cognitive impairment (sMCI) samples, and 660 progressive mild cognitive impairment (pMCI) samples, as shown in Table 1.

[0062] Table 1 Statistics of the dataset used in the study

[0063] NC AD sMCI pMCI Sample size 700 766 731 660 Gender (male / female) 382 / 318 374 / 392 436 / 295 375 / 285 age 75.34±6.87 75.02±7.70 74.86±7.57 74.59±7.45 MMSE 29.11±1.12 21.8±4.14 27.88±2.03 26.36±2.22

[0064] Brain imaging software such as SPM and CAT12 were used for data preprocessing and gray matter segmentation. The gray matter volume of each brain region was obtained by segmenting 83 brain regions using brain templates. The original T1-weighted MRI scan images collected in the ADNI dataset were all in DICOM format and converted to NIFTI format using Matlab. Subsequent preprocessing was mainly completed using Matlab-based SPM12 and CAT12 brain imaging software packages.

[0065] The MRI feature extraction module extracts brain structural features from gray matter images segmented from MRI based on convolutional neural networks (CNNs), such as Figure 1 As shown in (a), the module consists of four 3D convolutional layers and two 3D fully connected layers.

[0066] The MRI feature extraction module classifies images of patients with normal cognition and Alzheimer's disease through a pre-trained CNN model, and then fine-tunes it with images of patients with mild cognitive impairment. Finally, the gradient-weighted class activation mapping technology is used to visualize the most important decision-making area of ​​the model in the classification process, namely the brain region of interest, to improve the interpretability of the model.

[0067] Several studies have shown that during cognitive decline, abnormal changes in brain gray matter structure are more significant than those in white matter. Therefore, preprocessed gray matter images of NC and AD patients were input into CNN, and the pretrained model became an AD-NC classifier. The pretrained CNN can extract features of brain structural changes associated with cognitive decline in Alzheimer's disease.

[0068] The risk prediction module realizes the AD conversion risk prediction function, such as Figure 1 As shown in (b), the model consists of two groups of fully connected layers, residual connections and multi-head attention mechanisms. The two groups of fully connected layers are fully connected group 1 FCLayersA and fully connected group 2 FCLayersB.

[0069] FCLayersA contains two fully connected layers with 128 and 256 nodes. FCLayersB contains two fully connected layers with 128 and 64 nodes. Brain structural features, clinical information, and gray matter volume extracted from MRI constitute multi-scale data and are passed to FCLayersA as input x.

[0070] In order to capture the complex dependencies between elements in the feature sequence, a multi-head attention mechanism formula is introduced.

[0071]

[0072] Among them, Q, K and V represent the features of query, key and value respectively, d k represents the embedding dimension, K T represents the transpose of K.

[0073] The multi-head attention mechanism calculates the attention weights through the formula to preliminarily capture the potential representation of covariates related to the risk of Alzheimer's disease conversion. FCLayersB converts the vector f A (x) is combined with the covariate x as input z, directly learning the residual and maintaining data integrity. The final output vector f B (x) is a probability distribution y = [y1,y2,…,y T ], indicating that for a given mild cognitive impairment subject x i When the covariate x is converted into the estimated risk probability of Alzheimer's disease occurring at time t, that is, Among them, x i is the subject characteristic, and e represents the occurrence of the event.

[0074] A prediction method based on a multi-scale deep learning long-term individual AD risk prediction model, comprising the following steps:

[0075] Step S1, the MRI data acquisition and processing module removes the skull of the initial brain MRI neural image and registers it to the MNI152 standard space.

[0076] Step S2: Segment the image into gray matter, white matter, and cerebrospinal fluid according to the Hammers brain atlas template. Through segmentation processing, the corresponding gray matter and white matter segmentation images, as well as the volume data of gray matter, white matter, and cerebrospinal fluid of 83 brain regions of the Hammers atlas are obtained.

[0077] Step S3, further correcting the segmented image by deviation correction for intensity inhomogeneity and modulation according to volume changes caused by spatial registration. After preprocessing, images with a quality lower than 80% will be excluded.

[0078] Step S4: Use an 8 mm full-width half-maximum smoothing kernel to smooth the image to improve the signal-to-noise ratio of the image.

[0079] Step S5, the MRI feature extraction module classifies images of normal cognition and Alzheimer's disease patients through the pre-trained CNN model.

[0080] Step S6: Fine-tune using images of patients with mild cognitive impairment, and use the output before the last fully connected layer as the extracted brain structure features.

[0081] Step S7: Finally, the most important decision-making area of ​​the model in the classification process, namely the brain area of ​​interest, is visualized through the gradient weighted class activation mapping technology.

[0082] Step S8: The risk prediction module uses the gray matter volume of the brain region and the MRI image features obtained by the MRI data acquisition and processing module and the MRI feature extraction module, and combines the clinical information to input the three together as risk variables into the risk prediction model.

[0083] Step S9: predict the specific risk probability of each mild cognitive impairment (MCI) patient converting to AD at every six-month node in the next five years, so as to understand the pathological process of MCI patients.

[0084] In the MRI data acquisition and processing module, each sample is represented by a triple (x, s, e), where x is a high-dimensional vector representing all features of the sample, s is the survival time, and e is the occurrence of the event.

[0085] When e = 0, it means that no conversion was observed in this sample and the patient was lost to follow-up, and when e = 1, it means that the patient converted to Alzheimer's disease.

[0086] The high-dimensional vector x of all features of each sample consists of three types of features, specifically,

[0087] (1) Brain structural features extracted by the MRI feature extraction module.

[0088] (2) Clinical information of each sample, including gender, age, Mini-Mental State Examination (MMSE) score, and APOE A1 and APOE A2 gene information.

[0089] (3) Gray matter volumes of 83 brain regions of interest obtained during the data preprocessing stage.

[0090] Among them, the sample data set is D = {(x i ,s i ,e i )} i＝1 Represents the MCI samples used for risk prediction / survival analysis, with a total number of N.

[0091] The five-fold cross-validation method was used to evaluate the performance of the pre-trained CNN model in AD and NC classification.

[0092] a. The dataset includes 766 AD samples and 700 NC samples. The model training was performed for 30 cycles, with an initial learning rate of 1e-6, which was dynamically adjusted during the training process, decreasing by 10% every 10 cycles. Finally, the final CNN model was trained using all AD and NC samples to obtain stronger classification capabilities. This model is a pre-trained model.

[0093] b. The output of the first fully connected layer is used as the extracted brain structure feature. The gradient-weighted class activation mapping technique is used to visualize the image regions that contribute most significantly to the classification results by calculating the weighted gradients in CNN and back-propagating them to the input image.

[0094] c. A five-fold cross-validation strategy was used to fine-tune the pre-trained CNN using gray matter images of sMCI and pMCI.

[0095] (1) In order to effectively extract the features of all MCI samples and avoid data leakage, in each fold, the pre-trained model is fine-tuned using the training set divided by the fold to capture the specific features of the MCI data by adjusting the weights.

[0096] (2) Then input the test set of the fold into the model.

[0097] (3) Extract the fully connected layer before the classification layer and use the output as the feature representation of each sample in the test set.

[0098] Since the test set for each fold is different, the feature representation of all MCI samples can be obtained after the five-fold cross-validation is completed.

[0099] The risk prediction module uses a five-fold cross-validation strategy to train and evaluate model performance.

[0100] a. The input of the risk prediction model includes a 256-dimensional MRI brain structure feature vector, a 5-dimensional clinical information vector, and an 83-dimensional brain gray matter volume vector, which together form a 344-dimensional multi-scale feature representation.

[0101] b. Each fully connected layer uses the ReLU activation function.

[0102] c. The module is trained for a total of 1500 cycles, with an initial learning rate of 1e-4, and dynamically adjusted to half of the original learning rate every 500 cycles.

[0103] This embodiment uses the Concordance Index (C index), a commonly used indicator in survival analysis, to evaluate the performance of the survival analysis model. This index can measure the consistency between the survival risk predicted by the model and the actual survival results. Specifically, all samples are grouped, and every two samples are considered a pair. For each pair of samples, if the predicted survival time of an individual with a longer actual survival time is also longer, or the predicted survival time of an individual with a shorter actual survival time is also shorter, then the pair is considered to be consistent. The value of the C index is between 0 and 1, and the higher the value, the better the prediction accuracy.

[0104] In addition, this embodiment also uses the Brier Score (BS) evaluation index, which measures the prediction performance of the survival model at a specific time point t by evaluating the average square distance between the predicted survival probability and the actual survival status. The range of BS is between 0 and 1, and the lower the value, the better the prediction accuracy. By integrating BS at different time points, an integrated BrierScore (IBS) can be obtained to evaluate the prediction performance of the survival probability over a certain period of time (5 years in this embodiment).

[0105] The prediction model of this embodiment is subjected to prediction evaluation, specifically:

[0106] 1. Feature extraction.

[0107] The sMRI collected from the ADNI dataset was preprocessed to generate GM images and GM volumes of 83 brain regions. In this embodiment, a five-fold cross-validation scheme was used to train and evaluate the CNN classification model based on GM images of AD and CN. The average accuracy of each fold test set was 92.91%, indicating that the model performed well in distinguishing AD from NC. After fine-tuning using the MCI training set, the output of the first FC layer was used as the result of the GM feature extraction module. Finally, a 256-dimensional vector was obtained for each MCI individual, representing its brain structural characteristics.

[0108] In this embodiment, the Grad-CAM method is used to visualize the regions of interest (ROI) in three different sections (axial, sagittal and coronal sections) of the MRI scan to identify the brain areas that have a significant impact on the AD and NC classification tasks, such as Figure 2 As shown. The visualization results include weighted CAM maps, which highlight areas that play an important role in the classification task, with darker red indicating a greater impact on the classification. This embodiment superimposes the weighted CAM map on the original GM image, combining the anatomical information in the MRI scan with the ROI identified by the model, thereby more comprehensively displaying the relationship between brain structure and AD cognitive decline. Consistent with previous research results, this embodiment shows that AD cognitive decline is significantly associated with specific brain areas (mainly concentrated in the frontal and temporal lobes). These findings not only verify existing research results, but also lay a solid foundation for subsequent risk prediction research.

[0109] 2. Long-term and quantitative risk prediction.

[0110] To verify the effectiveness of the input feature combination, this embodiment uses a five-fold cross-validation method to train and evaluate the multi-scale deep model using different combinations of three types of features, as shown in Table 2. A, B, and C represent three types of data: clinical data, gray matter (GM) volume of brain regions, and brain structural features. The results show that when the three types of data (A, B, and C) are used in combination, the multi-scale risk prediction model performs best, with a C index of 0.8881 (95% confidence interval lower limit of 0.8709, upper limit of 0.9053), and an IBS of 0.0806 (95% confidence interval lower limit of 0.0686, upper limit of 0.0927). It can be seen from the results that GM volume contributes the most to the prediction model. When only this type of data is used, the C index is 0.8778 (0.8553, 0.9003) and the IBS is 0.0837 (0.0766, 0.0908), indicating that the ability to reflect the trend of cognitive decline based on brain GM volume is the strongest. The second is the extracted brain structure features, with a C index of 0.8604 (0.8222, 0.8986) and IBS of 0.0933 (0.0704, 0.1163). The contribution of clinical data to the risk prediction model is relatively weak, with a C index of 0.7199 (0.6849, 0.7549) and IBS of 0.1802 (0.1689, 0.1916).

[0111] In order to verify the effectiveness of the multi-head attention mechanism, this embodiment conducted an experiment to compare the multi-scale model that does not use the attention mechanism (called the noAtt model), and the results are shown in Table 2. When using the combined ABC data, the C index of the noAtt model is 0.8789 (95% confidence interval is 0.8616, 0.8962), which is slightly lower than the multi-scale model based on the attention mechanism, and the IBS is 0.0854 (0.0748, 0.0960), which is slightly higher. When using other single data or data combinations, the C index is relatively low and the IBS is relatively high, which proves the effectiveness of introducing the multi-head attention mechanism.

[0112] In order to demonstrate the superiority of the multi-scale model, this embodiment conducted a comparative experiment with the Cox proportional hazard model (CPH) and the random survival forest (RSF) model. In the two prediction methods, when the three types of data were used in combination, the best prediction performance was obtained, with C indexes of 0.8162 (0.7892, 0.8433) and 0.8352 (0.8149, 0.8554), respectively, and IBS values ​​of 0.1352 (0.1145, 0.1560) and 0.1178 (0.1126, 0.1231), respectively. Regardless of which data combination is used for risk prediction, the C index of the multi-scale model is always higher and the IBS value is lower, which outperforms the CPH and RSF methods.

[0113] This example uses a combination of three data features to evaluate the risk prediction capabilities of the multiscale model and other models at different time points over 60 months. The risk prediction of each model is evaluated by five-fold cross validation. In each fold validation, the model is trained on the training set and then used to predict the test set to generate the corresponding C index. Figure 3 As shown, the C-index distribution of the four models at different time points is displayed, so that the performance of the models at each time point can be compared. The size of the box plot reflects the volatility of the model performance, and a larger box indicates that the model performance fluctuates more. The CPH model has generally higher boxes at all time points, indicating that its C-index fluctuates more and is less stable. In contrast, the multiscale model has smaller boxes at all time points, indicating that its performance is more consistent. The C-index of the multiscale model at each time point ranged from 0.8336 to 0.8992, indicating that it has strong predictive consistency and reliable calibration when using all types of data on the basis of baseline data to predict the risk of AD conversion within the next 60 months.

[0114] In order to evaluate the difference in C index of the four models at different time points, this embodiment performed an analysis of variance (ANOVA). At 6 months, the difference between the models was not significant (p>0.05), the performance of each model was very close, and there was also a large overlap in the box plot. From 12 months on, the performance difference between the models became significant (p<0.05), indicating that in short-term predictions, there was little difference between the models, and the deep learning model was more suitable for long-term predictions. At the same time, from 12 months on, the F statistic was high, indicating that the difference between the models was significant, while the difference within the group was small.

[0115] This example verifies whether the multiscale model can effectively predict the occurrence of MCI conversion. When combining clinical data, GM volume and brain structural characteristics, the model showed the best risk prediction ability. Therefore, the combined data was used to calculate the predicted risk value for all MCI individuals, and the average predicted risk within 60 months was used as the risk score for each individual. This method comprehensively considers the long-term risk level of MCI individuals within 5 years, and is not limited to a single risk value. According to the risk score, the instances were divided into high-risk and low-risk groups, and the threshold for grouping was based on the median of the risk score. The Kaplan-Meier survival curve was plotted to reflect the survival probability of the two groups, as shown in Figure 4 The log-rank test was used to compare the high-risk group and the low-risk group, and the p-value of the multiscale model was 2.0758×10 -243 , indicating that there were significant differences between the two groups and that the model had a strong ability to identify and predict conversion in MCI patients.

[0116] This embodiment also predicts the specific time of AD conversion based on the risk value. Figure 5 As shown, the actual conversion time of MCI individuals to AD is compared with the predicted conversion time. If the actual conversion does not occur and the conversion is not predicted to occur, the predicted conversion time is recorded as 0. There are many overlapping data points in the figure, and the points with an overlap of greater than or equal to 10 have been marked. 89.6477% of the data points are between the upper and lower limits represented by the dotted lines, which are 30.77 and -32.37 respectively, and the average difference of the center line is -0.80. This shows that there is generally good consistency between the predicted conversion time and the actual conversion time, showing that the model has satisfactory accuracy and consistency in predicting the time when MCI converts to AD.

[0117] In addition, the density distribution map also provides additional evidence to support the consistency between the actual conversion time and the predicted conversion time. Through the two-dimensional kernel density estimation (KDE), the distribution of data points on the two-dimensional plane can be estimated. Lighter colors indicate higher density of data points. Figure 6As shown, the data points show an obvious linear distribution trend, indicating that there is a strong positive linear relationship between the actual conversion time and the predicted conversion time, which shows that the model has good prediction performance.

[0118] This example also evaluates the mean absolute error (MAE) between the actual conversion time and the predicted conversion time when different models are combined with various data types, as shown in Table 2. The results show that when the three types of data are combined, the multi-scale model has the strongest prediction performance, with a MAE value of 7.3738 (6.7418, 8.0081).

[0119] Table 2 Model performance evaluation and comparison

[0120]

[0121]

[0122] Therefore, the present invention adopts the above-mentioned long-term individual AD risk prediction model and prediction method based on multi-scale deep learning. The model predicts the risk of MCI converting to AD at the individual level. Based on the baseline diagnostic data of the subjects, the conversion risk can be obtained every 6 months within 60 months, thereby effectively tracking the MCI conversion risk and estimating the conversion time.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A long-term individual AD risk prediction model based on multi-scale deep learning, characterized by: It includes a magnetic resonance imaging (MRI) data acquisition and processing module, a magnetic resonance imaging (MRI) feature extraction module, and a risk prediction module; the magnetic resonance imaging (MRI) feature extraction module consists of four 3D convolutional layers and two 3D fully connected layers.

2. The multi-scale deep learning long-term individual AD risk prediction model according to claim 1 is characterized by: The magnetic resonance imaging (MRI) data acquisition and processing module obtains the baseline T1-weighted structural magnetic resonance imaging (MRI) in the Alzheimer's Disease Neuroimaging Project (ADNI) public dataset. The original T1-weighted magnetic resonance imaging (MRI) scan images collected in the Alzheimer's Disease Neuroimaging Project (ADNI) public dataset are all in DICOM format and converted to NIFTI format through Matlab. The preprocessing is completed using the Matlab-based SPM12 and CAT12 brain image processing software packages.

3. The multi-scale deep learning long-term individual AD risk prediction model according to claim 2 is characterized by: The MRI data acquisition and processing module obtains the baseline T1-weighted structural MRI in the Alzheimer's Disease Neuroimaging Project (ADNI) public dataset, including several healthy subjects (NC) samples, Alzheimer's disease (AD) samples, stable mild cognitive impairment (sMCI) samples, and progressive mild cognitive impairment (pMCI) samples. Each MCI sample must contain corresponding structural magnetic resonance imaging (sMRI) data and complete clinical information, including age, gender, Mini-Mental State Examination (MMSE) score, and APOEε4 genotype A1 and genotype A2; For the MCI samples, only those with at least two follow-up records were included.

4. The multi-scale deep learning long-term individual AD risk prediction model according to claim 1, characterized in that: The risk prediction module consists of two sets of fully connected layers, residual connections, and a multi-head attention mechanism; The two groups of fully connected layers are FCLayersA and FCLayersB respectively. FCLayersA contains two fully connected layers with 128 and 256 nodes respectively; FCLayersB contains two fully connected layers with 128 and 64 nodes respectively; the extracted brain structural features, clinical information and gray matter volume constitute multi-scale data and are passed to FCLayersA as input x.

5. The multi-scale deep learning-based long-term individual AD risk prediction model according to claim 4 is characterized by: Capture the complex dependencies between elements in the feature sequence and introduce the multi-head attention mechanism formula. Among them, Q, K and V represent the features of query, key and value respectively, d k represents the embedding dimension, K T represents the transpose of K; The multi-head attention mechanism calculates the attention weights through the formula to preliminarily capture the potential representation of covariates related to the risk of Alzheimer's disease conversion; FCLayersB converts the vector f A (x) is combined with the covariate x as input z, directly learning the residual and maintaining data integrity; the final output vector f B (x) is a probability distribution y = [y1,y2,…,y T ], indicating that for a given mild cognitive impairment subject x i When the covariate x is converted into the estimated risk probability of Alzheimer's disease occurring at time t, that is, Among them, x i is the subject characteristic, and e represents the occurrence of the event.

6. A prediction method based on a multi-scale deep learning long-term individual AD risk prediction model as claimed in any one of claims 1 to 5, characterized in that: The following steps are involved: Step S1, the MRI data acquisition and processing module performs skull removal processing on the initial brain MRI neuroimage and registers it to the MNI152 standard space; Step S2, segmenting the image into gray matter, white matter, and cerebrospinal fluid according to the Hammers brain atlas template; obtaining the corresponding gray matter and white matter segmentation images, as well as the volume data of gray matter, white matter, and cerebrospinal fluid of 83 brain regions of the Hammers atlas through segmentation processing; Step S3, further correcting the segmented image by deviation correction for intensity inhomogeneity and modulating according to volume changes caused by spatial registration; after preprocessing, images with a quality lower than 80% are excluded; Step S4, smoothing the image using an 8 mm full width half maximum smoothing kernel; Step S5, the MRI feature extraction module classifies the images of normal cognition and Alzheimer's disease patients through the pre-trained convolutional neural network healthy person NCN model; Step S6: fine-tune using images of patients with mild cognitive impairment, and use the output before the last fully connected layer as the extracted brain structure features; Step S7, finally, the most important decision-making area of ​​the model in the classification process, i.e., the brain area of ​​interest, is visualized by using the gradient weighted class activation mapping technique; Step S8, the risk prediction module uses the gray matter volume of the brain region and the MRI image features obtained by the MRI data acquisition and processing module and the MRI feature extraction module; combined with clinical information, the three are input into the risk prediction model as risk variables; Step S9: predict the specific risk probability of each mild cognitive impairment (MCI) patient converting to Alzheimer's disease (AD) at every six-month node in the next five years.

7. The prediction method based on multi-scale deep learning long-term individual AD risk prediction model according to claim 6, characterized in that: In the MRI data acquisition and processing module, each sample is represented by a triple (x, s, e), where x is a high-dimensional vector representing all features of the sample, s is the survival time, and e is the event occurrence; When e = 0, it means that no conversion was observed in this sample and the patient was lost to follow-up, and when e = 1, it means that the patient converted to Alzheimer's disease.

8. The multi-scale deep learning-based long-term individual AD risk prediction model and prediction method according to claim 7, characterized in that: The high-dimensional vector x of all features of each sample consists of three types of features: for, (1) Brain structural features extracted by the MRI feature extraction module; (2) Clinical information of each sample, including gender, age, Mini-Mental State Examination (MMSE) score, and APOE A1 and APOE A2 gene information; (3) gray matter volumes of 83 brain regions of interest obtained during the data preprocessing stage; The sample data set is D = {(x i ,s i ,e i )} i＝1 Represents the mild cognitive impairment (MCI) sample used for risk prediction / survival analysis, with a total number of N.

9. The prediction method based on multi-scale deep learning long-term individual AD risk prediction model according to claim 6, characterized in that: The five-fold cross-validation method was used to evaluate the performance of the pre-trained convolutional neural network healthy person NCN model in the classification of Alzheimer's disease AD and healthy person NC; a. The dataset includes several Alzheimer's disease AD samples and healthy human NC samples; the model training is carried out for 30 cycles, the initial learning rate is 1e-6, and it is dynamically adjusted during the training process, decreasing by 10% every 10 cycles; finally, all Alzheimer's disease AD and healthy human NC samples are used to train the final convolutional neural network healthy human NCN model, which is a pre-trained model; b. The output of the first fully connected layer is used as the extracted brain structure feature; the gradient weighted class activation mapping technique is used to visualize the image area that contributes most significantly to the classification result by calculating the weighted gradient in the convolutional neural network healthy person NCN and back-propagating it to the input image; c. Using a five-fold cross-validation strategy, the pre-trained convolutional neural network (NCN) of healthy subjects was fine-tuned using gray matter images of stable mild cognitive impairment (sMCI) samples and progressive mild cognitive impairment (pMCI) samples; (1) In each fold, the pre-trained model is fine-tuned using the training set divided by the fold to capture the specific features of the MCI data by adjusting the weights; (2) Then input the test set of the fold into the model; (3) Extract the fully connected layer before the classification layer and use the output as the feature representation of each sample in the test set.

10. The prediction method based on multi-scale deep learning long-term individual AD risk prediction model according to claim 6, characterized in that: The risk prediction module uses a five-fold cross-validation strategy to train and evaluate model performance; a. The input of the risk prediction model includes a 256-dimensional magnetic resonance imaging (MRI) brain structure feature vector, a 5-dimensional clinical information vector, and an 83-dimensional brain gray matter volume vector, which together form a 344-dimensional multi-scale feature representation; b. Each fully connected layer uses the ReLU activation function; c. The module is trained for a total of 1500 cycles, with an initial learning rate of 1e-4, and dynamically adjusted to half of the original learning rate every 500 cycles.

Images