A method for predicting future progression of MCI based on sequential relationship prototype matching

Abstract

The present application belongs to the technical field of medical image analysis, and particularly relates to a MCI future progression prediction method based on sequential relationship prototype matching. The present application uses T1 MRI images of subjects at different stages of Alzheimer's disease (from NC to MCI to AD), encodes the images into low-dimensional features with the help of a deep learning method, and is consistent with the order of the subject labels, thereby learning the global ordered relationship of Alzheimer's disease, and can distinguish whether the current MCI subject is pMCI or sMCI without the need for accurate MCI labels. In the training stage, the subjects at different AD stages are sorted by using a sequence loss, the features are learned to be compact within the class and separable between the classes by using a Fisher discriminant loss, and the subjects at different AD stages are distinguished by using a cross-entropy; in the inference stage, whether the current MCI patient will be converted into AD or will stop deteriorating in the future is inferred by using the learned NC and AD prototypes. The present application can greatly improve the prediction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of medical image analysis technology, specifically relating to a method for predicting the future progress of MCI. Background Technology

[0002] Alzheimer's disease (AD) is characterized by its irreversibility and progressive nature, which can lead to severe cognitive impairment and behavioral problems

[23] . Mild cognitive impairment (MCI) is a precursor stage of AD, usually characterized by mild cognitive deficits and a high risk of progression to AD

[16] . According to the DSM-V criteria [1] regarding whether MCI patients will progress to AD after a period of follow-up diagnosis (usually 36 months), current MCI patients can be divided into two subtypes: progressive MCI (pMCI) and stable MCI (sMCI). Unlike the classification between AD and normal cognition (NC), which can be easily achieved through differences in patient behavior, distinguishing between pMCI and sMCI is very challenging due to their high similarity. Therefore, early prediction of whether MCI will progress to AD is particularly important for subsequent intervention and treatment.

[0003] According to the classification system of biomarkers for Aβ, tau, and neurodegenerative changes (A / T / N)[9], various longitudinal biomarkers and cognitive measurements show a unidirectional and orderly development trend in the development of AD[6]. Figure 1 As shown. Existing work on predicting the progression of MCI usually requires accurate MCI subtype labels, but collecting subtype labeled data requires a lot of time and resources, and the resulting labeled datasets are usually relatively small. According to the disease progression sequence of AD, the pathological differences between sMCI and pMCI are similar to those between NC and AD, but much milder [8]. Therefore, we can use additional subjects in NC and AD to learn the global ordered relationship of Alzheimer's disease progression to help predict the development of MCI.

[0004] Previous studies have shown that models trained on NC / AD perform better than models trained on sMCI / pMCI [4,8,10,11] for predicting the progression of MCI. However, these studies only consider the differences between sMCI and pMCI, similar to those between NC and AD, while ignoring the globally ordered relationships in Alzheimer's disease development. Summary of the Invention

[0005] The purpose of this invention is to propose a method for predicting the future progress of MCI based on order relation prototype matching that does not require subtype labels and has high accuracy.

[0006] In this invention, it is assumed that the i-th three-dimensional T1 MRI scan is represented in image space. And there is an ordered label space Where K = 3, and 1, 2, and 3 represent NC, MCI, and AD, respectively. Using Let x be the value generated by encoder f(·). i The feature representation, given a mini-batch of size M, This invention aims to obtain from mapping (From image to feature to ordered label) learns the global ordered relationship of AD progression, further distinguishing pMCI (progressive MCI) and sMCI (stable MCI) without requiring precise MCI labels.

[0007] In this invention, during the training phase, order loss is used to rank all subjects at different AD stages—from NC to MCI to AD—to learn a globally ordered relationship. Fisher discriminant loss learns intra-class compactness and inter-class separability features, and cross-entropy (CE) is used to distinguish subjects at different AD stages. During the inference phase, the learned NC and AD prototypes are used to infer whether a current MCI patient will progress to AD or stop deteriorating in the future. Figure 2 The diagram illustrates the training and inference process corresponding to our proposed method.

[0008] The method for predicting the future progress of MCI based on order relation prototype matching provided by this invention includes the following specific steps:

[0009] (1) First, for the ordered relationship of Alzheimer's disease development, an order loss that can effectively learn the order of disease development is adopted, aiming to learn the global ordered relationship of AD development and learn to compare with each other in the feature space. Here, the order loss[5] is used to align the label space. and feature space The order relation, specifically as follows: Figure 3 As shown;

[0010] Let S y ∈R M×M Represents all in the ordered label space The pairwise similarity matrix of the subjects, then S y The (i,j)th element in the metric y i and y j The similarity between them is defined as:

[0011]

[0012] Where, d y This indicates a negative absolute distance.

[0013] Similarly, the pairwise similarity matrix in the feature space is Sz∈R M×M Then, the (i,j)th element in Sz measures the similarity between zi and zj, and the specific calculation is as follows:

[0014]

[0015] Where, d z This represents the cosine similarity.

[0016] The order loss encourages that the features z in each mini-batch of data have the same order as the ordered labels y. According to [5], we minimize the mean minimum error between features z and labels y, and the corresponding order loss is defined as follows:

[0017]

[0018] in, It is a sorting function.

[0019] here, This indicates that it is just a function, and the subscript [i,:] represents the i-th row of a matrix.

[0020] In fact, formula (3) is difficult to optimize because of the sorting function. The properties of non-differentiable piecewise functions. Because Since the gradient of is almost zero everywhere with respect to a, we recurse the sorting operation as minimizing the objective of the linear combination problem

[20] :

[0021]

[0022] here, It is a set of permutations and combinations of vectors π. To achieve efficient gradient propagation through a black-box combinatorial optimizer

[17] , we construct a continuous interpolation for the piecewise constant loss function to obtain gradients with rich information. In specific training, we ensure that the label of each mini-batch appears only once. For subjects with the same label in the same batch, the ranking function is used. Subjects with smaller indices will be prioritized, which introduces some perturbation and makes the learned features more robust.

[0023] (2) In order to enable the learned features to achieve high cohesion and low coupling, this invention utilizes Fisher's discrimination criterion to act on the feature level [2,3];

[0024] use Let μ represent the feature set of the k-th class. k μ and μ represent sets respectively and The average characteristic of is given by the following formula:

[0025]

[0026] Here, |·| represents the cardinality of the corresponding set. Next, the scatter matrix between classes and the scatter matrix within classes are defined as follows:

[0027]

[0028]

[0029] Finally, Fisher determined the loss. It can be represented as:

[0030]

[0031] Here, Trace(·) means finding the trace of a matrix.

[0032] (3) In order to predict the future development of MCI without subtype labels based on order relations, a prototype matching method is used to make corresponding predictions for current MCI subjects during inference.

[0033] During the training phase, the network utilizes 3D T1 MRI images and labeled data from different phases to pair (x i ,y i From NC to MCI to AD, cross-entropy loss is used as input to classify them:

[0034]

[0035] here, The labels represent reality and prediction, respectively.

[0036] Based on the orderly disease progression of Alzheimer's disease, using order loss The input data is sorted to learn a globally ordered representation and how to compare data in the feature space. Simultaneously, Fisher's discriminative loss is utilized. To achieve a state of high cohesion and low coupling in the learned features, the overall objective function can ultimately be defined as:

[0037]

[0038] Here, λ Ordinal and λ FisherThe weighting coefficients are all set to 1.0 in the examples; note that the overall objective function is calculated in each mini-batch.

[0039] During the inference phase, the encoder is first fed with all 3D MRI images from the training set labeled NC (k=1) and AD (k=3). Therefore, the corresponding prototypes of NC and AD are calculated as follows:

[0040]

[0041] Then, compare the characteristics x of the MCI subjects to be predicted. q And the NCP learned NC and ADP AD Matching the prototypes of the subjects with each other to infer whether the subject will gradually become AD in the future

[22] . The probability of a subject gradually becoming AD in the future, i.e., pMCI, is defined as:

[0042]

[0043] Figure 2 Detailed diagrams of training and inference are shown. Based on the fact that AD development is orderly, if the input MCI feature is closer to the AD prototype, it means that the MCI subject has a high probability of asymptotically progressing to AD, so it is classified as pMCI. Similarly, if it is closer to the NC prototype, it means that the MCI subject has a high probability of not asymptotically progressing to AD, so it is classified as sMCI.

[0044] Compared with other methods, the main advantages of the method of the present invention are:

[0045] (1) A novel method based on order relation prototype matching is proposed to learn the ordered developmental relationship of Alzheimer's disease, which can distinguish between pMCI and sMCI without the need for precise MCI subtype labels;

[0046] (2) The order loss and Fisher discriminant loss are used to learn the global ordered relationship, while the learned features have stronger intra-class compactness and inter-class separability.

[0047] (3) Experimental results on the ADNI dataset show that the method of the present invention is superior to the recently published method, and the accuracy is improved by 8.8% compared with the traditional method using precise MCI subtype labels. Attached Figure Description

[0048] Figure 1Examples of biomarkers and cognitive measurements illustrating the orderly progression of Alzheimer's disease. (a) shows the abnormal progression sequence of different biomarkers, and (b) is a violin plot of Mini-Mental State Examination (MMSE) scores.

[0049] Figure 2 This diagram illustrates the training and reasoning stages of the method of the present invention.

[0050] Figure 3 Explanation of order loss.

[0051] Figure 4 The flowchart shows the corresponding operation for preprocessing.

[0052] Figure 5 To visualize data using tsne.

[0053] Figure 6 This is an explanation of the interpretability diagram. Detailed Implementation

[0054] The present invention will be further described below with reference to experimental examples and accompanying drawings, and the specific results of the invention will be compared with those of other methods.

[0055] The data used in this study came from the Alzheimer's Disease Neuroimaging Initiative (ADNI) program and included a total of 1,050 three-dimensional T1 MRI images, including 317 NC (normal control group), 317 sMCI (stable mild cognitive impairment group), 169 pMCI (progressive mild cognitive impairment group) and 247 AD (Alzheimer's disease group). The demographic information of the subjects is shown in Table 1. If an MCI patient developed AD within 36 months, he / she was classified as pMCI; otherwise, he / she was classified as sMCI. The MRI images were processed using the standard preprocessing workflow

[19] using the Freesurfer and FSL software, for details. Figure 4 Finally, all MRI images were resized to 128×128×128. The data was divided in a 7:1:2 ratio, with 70% used for training, 10% for validation, and the remainder for testing. It is important to note that in the training process of the model in this invention, precise MCI subtype labels were not used; instead, sMCI and pMCI were combined into a single MCI for model training.

[0056] Table 1: Statistical Information of Corresponding Subjects in Corresponding Roles

[0057]

[0058] All methods in this invention are implemented using the PyTorch library

[15] and the model is trained on an NVIDIA V100 GPU. All networks are built with 3D ResNet18 as the backbone, initialized using the Kaiming method [7], and trained for 60 epochs using the Adam optimizer. The initial learning rate is set to 2×10⁻⁴ for the first 15 epochs, and then the learning rate is gradually reduced using exponential decay for the remaining duration, with a decay rate of 0.95. The results presented in this invention are the average of five independent experiments. Performance evaluation used six metrics, including accuracy (ACC), area under the ROC curve (AUC), F1 score (F1S), precision (PRE), sensitivity (SEN), and specificity (SPE).

[0059] Example 1: Order Relation Verification Experiment

[0060] To verify that our method can learn the ordered development of AD, we present a series of two-dimensional visualization results using t-distributed stochastic neighbor embedding (t-SNE)

[12] . Figure 5 (a) is the input three-dimensional MRI scan, which shows no significant difference between NC / MCI / AD. Figure 5 (b) shows only the use of The output of the trained encoder f can roughly divide the data into three different stages. Furthermore, we... Figure 5 (cd) shows the use of The output of the trained encoder. We observe that, using The trained encoder can learn the sequence of AD disease progression. When we... Figure 5 When we marked the precise MCI subtype labels in (d), we found that the features of sMCI and pMCI were closer to the NC and AD prototypes, respectively.

[0061] Example 2: Interpretability Verification Experiment

[0062] We calculated the cosine similarity between the convolutional feature maps of the prototype and individual MRI images

[13] , and in Figure 6 The diagram shows the discriminant regions associated with prediction. We found that the discriminant regions in three directions roughly correspond to the location of the hippocampus, and changes in the hippocampus have been verified to be associated with the progression of Alzheimer's disease.

[0063] Example 3: Loss Ablation Experiment

[0064] Ablation experiments were conducted to demonstrate the loss of components. and The efficacy is shown in Table 2. The following results were obtained.

[0065] 1) Both types of losses can independently improve prediction performance. 2) and The loss function is more effective in improving predictive performance, indicating its ability to learn the globally ordered relationships of AD progression. 3) When used simultaneously and Even with losses, performance can be further improved.

[0066] Table 2: Results of Loss Ablation Experiment

[0067]

[0068] Example 4: Comparative Experiment of Classical Prediction (sMCI / pMCI) and Diagnosis (NC / AD) Methods

[0069] Table 3 presents the comparison results of the predictions, demonstrating that the method of this invention performs well in terms of prediction performance. CE2 and CE3 are classification methods utilizing CE loss, used for two and three classes, respectively. The CE2 method mainly utilizes different training dataset configurations (sMCI / pMCI and NC / AD) to predict the progression of MCI. The results of this study are consistent with previous studies [4,8,10,11], showing that using NC / AD as the training dataset for predicting MCI progression results in a higher performance improvement compared to using precise subtype labels sMCI / pMCI. This indicates that the information learned by the network from NC / AD data is more useful than that from sMCI / pMCI.

[0070] We modified the training / inference processes of CE3, Ordinal Regression CNN (OR-CNN) [14,21], and ADRank

[18] as our approach. While the CE3 approach only introduces MCI objects, it makes learning easier to distinguish between NC and AD, leading to improved performance. OR-CNN transforms the classification problem with ordinal relations into a series of binary classification subproblems, improving prediction performance by ordering the label space. However, it does not guarantee that the learned features are ordered. ADRank adds pairwise ordering constraints to the feature space in OR-CNN, but the ordering operation is still implemented through fully connected layers. In fact, the global ordinal relations in each low-dimensional feature subspace still need to be guaranteed. However, our approach guarantees the consistency of the order between the feature space and the low-dimensional space, making the learned features more compact within classes and more separable between classes.

[0071] Similar conclusions can also be seen in the NC and AD classification diagnostic tasks in Table 4.

[0072] Table 3: Comparison of Experimental Results between Classical Prediction (sMCI / pMCI) Methods

[0073]

[0074] Table 4: Comparison Experiment Results Table of Classical Diagnostic (NC / AD) Methods

[0075]

[0076] References:

[0077] [1] American Psychiatric Association, D., Association, A.P., et al.: Diagnostic and statistical manual of mental disorders: DSM-5, vol. 5. American psychiatric association Washington, DC (2013)

[0078] [2] Cheng, G., Zhou, P., Han, J.: RIFD-CNN: Rotation-invariant and fisher discriminative convolutional neural networks for object detection. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. pp. 2884–2893 (2016)

[0079] [3] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. In: International Conference on Learning Representations (2015)

[0080] [4] Falahati, F., Westman, E., Simmons, A.: Multivariate data analysis and machine learning in Alzheimer’s disease with a focus on structural magnetic resonance imag-ing. Journal of Alzheimer’s disease 41(3), 685–708 (2014)

[0081] [5]Gong,Y.,Mori,G.,Tung,F.:RankSim:Ranking similarity regularizationfor deep imbalanced regression.In:International Conference on MachineLearning.pp.7634–7649.PMLR(2022)

[0082] [6]Guo,T.,Korman,D.,Baker,S.L.,Landau,S.M.,Jagust,W.J.,Initiative,A.D.N.,et al.:Longitudinal cognitive and biomarker measurements support aunidirectional pathway in Alzheimer’s disease pathophysiology.Biologicalpsychiatry 89(8),786–794(2021)

[0083] [7]He,K.,Zhang,X.,Ren,S.,Sun,J.:Delving deep into rectifiers:Surpassing human-level performance on ImageNet classification.In:Proceedingsof the IEEE Inter-national Conference on Computer Vision.pp.1026–1034(2015)

[0084] [8]Huang,Y.,Xu,J.,Zhou,Y.,Tong,T.,Zhuang,X.,Alzheimer’s Disease Neu-roimaging Initiative(ADNI):Diagnosis of Alzheimer’s disease via multi-modality3D convolutional neural network.Frontiers in Neuroscience 13,509(2019)

[0085] [9]Jack Jr,C.R.,Bennett,D.A.,Blennow,K.,Carrillo,M.C.,Dunn,B.,Haeberlein,S.B.,Holtzman,D.M.,Jagust,W.,Jessen,F.,Karlawish,J.,et al.:NIA-AAre-search framework:toward a biological definition of Alzheimer’sdisease.Alzheimer’s&Dementia 14(4),535–562(2018)

[0086]

[10] Kwak,K.,Giovanello,K.S.,Bozoki,A.,Styner,M.,Dayan,E.:Subtyping ofmild cognitive impairment using a deep learning model based on brain atrophypatterns.Cell Reports Medicine 2(12)(2021)

[0087]

[11] Kwak,K.,Stanford,W.,Dayan,E.,Initiative,A.D.N.:Identifying theregional substrates predictive of Alzheimer’s disease progression through aconvolutional neural network model and occlusion.Human Brain Mapping 43(18),5509–5519(2022)

[0088]

[12] Van der Maaten,L.,Hinton,G.:Visualizing data using t-SNE.Journalof Machine Learning Research 9(11)(2008)

[0089]

[13] Niu,C.,Shan,H.,Wang,G.:SPICE:Semantic pseudo-labeling for imagecluster-ing.IEEE Transactions on Image Processing 31,7264–7278(2022)

[0090]

[14] Niu,Z.,Zhou,M.,Wang,L.,Gao,X.,Hua,G.:Ordinal regression withmulti-ple output CNN for age estimation.In:Proceedings of the IEEE conferenceon Computer Vision and Pattern Recognition.pp.4920–4928(2016)

[0091]

[15] Paszke,A.,Gross,S.,Massa,F.,Lerer,A.,Bradbury,J.,Chanan,G.,Killeen,T.,Lin,Z.,Gimelshein,N.,Antiga,L.,et al.:PyTorch:An imperative style,high-performance deep learning library.Advances in Neural InformationProcessing Sys-tems 32(2019)

[0092]

[16] Petersen,R.C.:Mild cognitive impairment as a diagnosticentity.Journal of Inter-nal Medicine 256(3),183–194(2004)

[0093]

[17] M.V.,Paulus,A.,Musil,V.,Martius,G.,Rolinek,M.:Differentiation of blackbox combinatorial solvers.In:International Conferenceon Learning Repre-sentations(2020)

[0094]

[18] Qiao,H.,Chen,L.,Zhu,F.:Ranking convolutional neural network forAlzheimer’s disease mini-mental state examination prediction at multipletime-points.Com-puter Methods and Programs in Biomedicine 213,106503(2022)

[0095]

[19] Qiu,S.,Joshi,P.S.,Miller,M.I.,Xue,C.,Zhou,X.,Karjadi,C.,Chang,G.H.,Joshi,A.S.,Dwyer,B.,Zhu,S.,et al.:Development and validation of aninterpretable deep learning framework for Alzheimer’s diseaseclassification.Brain 143(6),1920–1933(2020)

[0096]

[20] M.,Musil,V.,Paulus,A.,Vlastelica,M.,Michaelis,C.,Martius,G.:Optimizing rank-based metrics with blackbox differentiation.In:Proceedingsof the IEEE / CVF Conference on Computer Vision and PatternRecognition.pp.7620–7630(2020)

[0097]

[21] Rosati,R.,Romeo,L.,Vargas,V.M., P.A., C.,Fron-toni,E.:A novel deep ordinal classification approach for aestheticquality control classification.Neural Computing and Applications 34(14),11625–11639(2022)

[0098]

[22] Snell,J.,Swersky,K.,Zemel,R.:Prototypical networks for few-shotlearning.Ad-vances in neural information processing systems 30(2017)

[0099]

[23] Winblad,B.,Amouyel,P.,Andrieu,S.,Ballard,C.,Brayne,C.,Brodaty,H.,Cedazo-Minguez,A.,Dubois,B.,Edvardsson,D.,Feldman,H.,et al.:DefeatingAlzheimer’s disease and other dementias:a priority for european science andsoci-ety.The Lancet Neurology 15(5),455–532(2016)。

Claims

1. A method for predicting the future progress of MCI based on order relation prototype matching, characterized in that: Suppose that the i-th three-dimensional T1 MRI scan is represented in image space as And it has an ordered label space. ,in = 3, where 1, 2, and 3 represent NC, MCI, and AD respectively; using To indicate that it is generated by the encoder generated Feature representation, given size Small batches, = Establish mapping That is, learning the global ordered relationship of AD progress from image to feature to ordered label, and further distinguishing pMCI and sMCI without needing precise MCI labels; During the training phase, order loss is used to rank all subjects at different AD stages—from NC to MCI to AD—to learn global ordered relationships. Fisher discriminant loss is used to learn intra-class compactness and inter-class separability features, and cross-entropy (CE) is used to distinguish subjects at different AD stages. During the reasoning phase, the learned NC and AD prototypes are used to infer whether current MCI patients will progress to AD or stop deteriorating in the future; The specific steps are as follows: (1) First, for the ordered relationship of Alzheimer's disease development, an order loss that can effectively learn the order of disease development is adopted to learn the global ordered relationship of AD development and learn to compare with each other in the feature space; specifically, the order loss is used to align the label space. and feature space The order relation; set up Represents all in the ordered label space Pairwise similarity matrix of subjects The first in Element measurement and The similarity between them is defined as: ， （1） in, Indicates negative absolute distance; Similarly, the pairwise similarity matrix of the feature space is , The first in Element measurement and The similarity between them is calculated as follows: ， （2） in, Indicates cosine similarity; Ordered loss encourages the use of features in each mini-batch of data. With ordered labels Consistent sorting method; minimizing features and tags The average minimum error between them is defined by the following order loss: ， （3） in, It is a sorting function, here, Indicates that it is just a function, subscript Describe the first element of a matrix OK; Refactor the sorting operation into minimizing the linear combination problem: , (4) here, It is a set of permutation and combination vectors A set; To achieve efficient gradient propagation, a black-box combinatorial optimizer is used to construct a continuous interpolation for the piecewise constant loss function, thereby obtaining gradients with rich information. (2) In order to achieve high cohesion and low coupling in the learned features, Fisher's discrimination criterion is applied to the feature level; use Let represent the feature set of the k-th class, and the corresponding ... and Representing sets and The average characteristic of is given by the following formula: and (5) here, The cardinality of the corresponding set is represented; the scatter matrix between classes and the scatter matrix within classes are defined as follows: , (6) ； (7) Finally, Fisher determined the loss. It is represented as: ， (8) here, This is shown as solving for the trace of a matrix; (3) Prototype matching is used to make corresponding predictions for the current MCI subjects during inference; specifically: During the training phase, the network utilizes 3D T1 MRI images and labeled data from different phases to... From NC to MCI to AD, as input, cross-entropy loss is used for classification: ， (9) here, Labels representing reality and prediction, respectively; Based on the orderly disease progression of Alzheimer's disease, using order loss The input data is sorted to learn a globally ordered representation and how to compare data in the feature space; simultaneously, Fisher's discriminant loss is used. To achieve a state of high cohesion and low coupling in the learned features, the final overall objective function is defined as: ， （10） here, and The weighting coefficients are used to calculate the overall objective function in each mini-batch. During the inference phase, the encoder is first fed all the labels from the training set: NC and AD 3D MRI images }, thus the corresponding prototypes of NC and AD are calculated as follows: and (11) Then, the characteristics of the MCI subjects to be predicted are compared. and the NC learned and AD The probabilities are matched between the prototypes to infer whether the subject will gradually develop AD in the future, i.e., pMCI, with the probability defined as: (12) Based on the fact that AD development is orderly, if the input MCI feature is closer to the AD prototype, it means that the MCI subject has a high probability of asymptotically progressing to AD, so it is classified as pMCI; if it is closer to the NC prototype, it means that the MCI subject has a high probability of not asymptotically progressing to AD, so it is classified as sMCI.

Images