Alzheimer's disease classification method and system based on multi-modal evidence deep learning

Abstract

The invention discloses an Alzheimer's disease classification method and system based on multi-modal evidence deep learning, and relates to the technical field of machine learning, and the method comprises the steps: obtaining an Alzheimer's disease multi-modal data set; inputting samples in the Alzheimer's disease multi-modal data set into a pre-established Alzheimer's disease classification model, and adjusting parameters of the Alzheimer's disease classification model to be optimal through an error back propagation algorithm to obtain an optimized Alzheimer's disease classification model; and obtaining subject sample data, inputting the subject sample data into the optimized Alzheimer's disease classification model, and outputting to obtain a classification result.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, specifically to an Alzheimer's disease classification method and system based on multi-modal evidence deep learning. Background Art

[0002] Alzheimer's Disease (AD) is an irreversible neurodegenerative disease mainly characterized by memory loss and cognitive impairment. Its prevalence has been increasing year by year with the aggravation of population aging. It is a major disease endangering the health of the elderly, seriously affecting the quality of life of patients and their families. At present, the pathogenesis of Alzheimer's disease is not yet clear. Due to the latent onset and long course of the disease, patients often reach the advanced stage when obvious symptoms appear, and effective treatment cannot be carried out. Clinically, the development of AD is usually divided into three stages: Cognitively Normal (CN), Mild Cognitive Impairment (MCI), and AD. Among them, MCI has a high risk of developing into dementia caused by AD and is considered the early stage of AD. The early diagnosis of AD is crucial because MCI is a more effective treatment time window than AD. Adequate treatment in the early stage of the disease can help patients maintain an autonomous state for as long as possible, making it possible to reverse the disease progression and improve the subsequent treatment effect. Therefore, there is an urgent need for an accurate and efficient AD early diagnosis technology to help clinicians develop individualized diagnosis and treatment plans.

[0003] In recent years, with the rapid development of artificial intelligence technology and the improvement of computer hardware computing power, people have begun to study the use of computer technology to assist doctors in the diagnosis of Alzheimer's disease. Using deep learning technology to process neuroimages for Alzheimer's disease classification has become an important research topic. However, in traditional deep neural network classification models, the Softmax activation function is usually used in the last layer to output the prediction probability of each category, which may lead to overconfidence of the model and cannot effectively reflect the uncertainty of the classification result. To solve this problem, a new paradigm called "Evidential Deep Learning" (EDL) has emerged. It provides reliable uncertainty estimation in a single forward pass without additional extensive calculations. The basic assumption of evidential deep learning is that the classification prediction probability output by the model follows a Dirichlet prior. By maximizing the probability likelihood of the observed data, the posterior distribution of the output classification probability can be obtained using a deep learning model. In this way, the evidential deep learning model directly learns the uncertainty of the classification probability, a second-order uncertainty similar to entropy. Evidential deep learning applies the theory of Subjective Logic to formalize the basic probability assignment function in Dempster-Shafer evidence theory as a Dirichlet distribution in the frame of discernment and realizes the quantification of credibility and uncertainty.

[0004] On the other hand, currently, single-modal images are mainly used in the field of Alzheimer's disease auxiliary diagnosis. However, due to the limitations of various imaging technologies in terms of specificity, reliability, and sensitivity, only single-modal image data cannot provide more comprehensive pathological information, resulting in difficulties in improving the classification accuracy of Alzheimer's disease. Summary of the Invention

[0005] To solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide an Alzheimer's disease classification method and system based on multi-modal evidential deep learning, which can improve the accuracy and robustness of Alzheimer's disease classification, and can also give an overall classification uncertainty assessment of the final decision, providing better model interpretability and decision credibility.

[0006] In the first aspect, the purpose of the present invention can be achieved through the following technical solutions: An Alzheimer's disease classification method based on multi-modal evidential deep learning, the method comprising the following steps:

[0007] Obtain a multi-modal dataset for Alzheimer's disease; input the samples in the multi-modal dataset for Alzheimer's disease into a pre-established Alzheimer's disease classification model, and adjust the parameters of the Alzheimer's disease classification model to the optimal through the error backpropagation algorithm to obtain an optimized Alzheimer's disease classification model; wherein, the process of processing by the pre-established Alzheimer's disease classification model includes data processing, evidence extraction, class confidence and classification result uncertainty estimation, and decision fusion;

[0008] Obtain subject sample data, input the subject sample data into the optimized Alzheimer's disease classification model, and output a classification result.

[0009] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: the samples in the multi-modal dataset for Alzheimer's disease include image data of magnetic resonance imaging (MRI) and positron emission tomography (PET), non-image data of clinical cognitive test scales, biomarkers, and genes, as well as corresponding Alzheimer's disease category labels.

[0010] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: the process of processing the multi-modal data for Alzheimer's disease includes:

[0011] Perform head motion correction, neck removal, image registration, skull stripping, brain tissue extraction, bias field correction, and slicing on the input MRI images to obtain MRI samples; perform image registration, skull stripping, bias field correction, and slicing on the input PET images to obtain PET samples; directly perform vector splicing on the input non-image data of clinical cognitive test scales, biomarkers, and genes to obtain non-image samples.

[0012] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: the process of evidence extraction includes:

[0013] MRI evidence extraction, PET evidence extraction, and non-image evidence extraction:

[0014] MRI evidence extraction uses a deep neural network and the Softplus activation function to extract MRI evidence supporting the k-th category from the MRI samples The superscript M in represents the MRI modality, and the subscript k represents the index number of the Alzheimer's disease category, k = 1, 2,..., K, where K is the number of Alzheimer's disease categories, The larger the value of, the higher the confidence that the MRI modality supports the k-th category; PET evidence extraction uses a deep neural network and the Softplus activation function to extract PET evidence supporting the k-th category from the PET samples The superscript P therein represents the PET modality, and the subscript k represents the index number of the Alzheimer's disease category, where k = 1, 2, …, K. The larger the value of, the higher the confidence that the PET modality supports the k-th category; for non-imaging evidence extraction, a deep neural network and the Softplus activation function are used to extract non-imaging evidence that supports the k-th category from non-imaging samples. The superscript D therein represents the non-imaging modality, and the subscript k represents the index number of the Alzheimer's disease category, where k = 1, 2, …, K. The larger the value of, the higher the confidence that the non-imaging modality supports the k-th category.

[0015] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: the category confidence and classification result uncertainty estimation utilize the evidence that each modality supports the k-th category to generate the confidence of the Alzheimer's disease category provided by the corresponding modality and the classification result uncertainty u m ; the more evidence that each modality supports the k-th category is, the higher the confidence of the Alzheimer's disease category provided by the corresponding modality will be; the more the sum of the evidence that each modality supports all categories is, the lower the classification result uncertainty u m provided by the corresponding modality will be, and a classification result with higher confidence is obtained; wherein, u m ≥0, and the superscript m in u m represents the modality category, where m ∈ {M, P, D}, m = M represents the MRI modality, m = P represents the PET modality, m = D represents the non-imaging modality, and k = 1, 2, …, K;

[0016] Utilize the evidence that each modality supports the k-th category to generate the confidence of the Alzheimer's disease category provided by the corresponding modality and the classification result uncertainty u m The process is as follows:

[0017] Convert the evidence that each modality supports the k-th category extracted into the concentration parameter of the Dirichlet distribution The calculation formula is:

[0018]

[0019] wherein, and The superscript m in it represents the modality category, m ∈ {M, P, D}, k = 1, 2, …, K;

[0020] For each sample in the K-classification, assign the K Alzheimer's disease class credibilities of each modality and the classification result uncertainty u m , and satisfy:

[0021]

[0022] Among them, u m ≥ 0;

[0023] According to the evidence of each modality supporting the k-th class Calculate the Alzheimer's disease class credibility provided by the corresponding modality and the classification result uncertainty u m , and the calculation formulas are respectively:

[0024]

[0025] Among them, S m is the Dirichlet strength, S m The superscript m in it represents the modality category, m ∈ {M, P, D}, k = 1, 2, …, K.

[0026] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: The decision fusion utilizes the Alzheimer's disease class credibilities provided by each modality and the classification result uncertainty u m , and according to the Dempster combination rule, evaluate the similarity and conflict between the evidences of each modality, and obtain the Alzheimer's disease class credibility b k after multi-modal fusion and the overall classification result uncertainty u, and output the final classification result;

[0027] The steps of outputting the final classification result according to the Dempster combination rule are as follows:

[0028] Utilize the Alzheimer's disease class credibilities provided by each modality and the classification result uncertainty u m , and respectively obtain the subjective opinions of the MRI modality, PET modality, and non-imaging modality And according to the Dempster combination rule, calculate the subjective opinion after fusing the three modalities. The calculation formulas are respectively:

[0029]

[0030] Among them, C is the normalization factor, and the calculation formula is

[0031]

[0032] Calculate the evidence e that supports the k-th category after fusing the three modalities k , the concentration parameter α of the Dirichlet distribution k and the Dirichlet strength S, and the calculation formulas are respectively:

[0033]

[0034] The predicted probability p that the input sample belongs to the k-th category k is equal to the mean of the Dirichlet distribution, and the calculation formula is:

[0035]

[0036] Among them, k = 1, 2,..., K, and the category corresponding to the maximum predicted probability is used as the final classification result.

[0037] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: The loss function used for inputting the samples in the Alzheimer's disease multi-modal dataset into the pre-established Alzheimer's disease classification model for training is defined as:

[0038]

[0039] Among them, L(α M ), L(α P ) and L(α D ) respectively represent the independent loss functions of the MRI modality, the PET modality and the non-imaging modality; λ is a hyperparameter for adjusting the single-modal loss weight, 0 < λ < 1; L(α) represents the comprehensive loss function after evidence fusion of the three modalities; The calculation formulas of L(α M ), L(α P ), L(α D ) and L(α) are respectively:

[0040]

[0041]

[0042] Among them, y k represents the true category label of the training sample assigned to the k-th category. When using one-hot encoding, if the training sample is assigned to the k-th category, then y k = 1, otherwise y k= 0; Ψ(·) is the Digamma function, which is monotonically increasing on (0, +∞).

[0043] In a second aspect, to achieve the above object, the present invention discloses an Alzheimer's disease classification system based on multi-modal evidence deep learning, including:

[0044] A model construction module, configured to construct an Alzheimer's disease classification model based on multi-modal evidence deep learning, the model including a data processing module, an evidence extraction module, a class credibility and classification uncertainty estimation module, and a decision fusion module;

[0045] A model optimization module, configured to input samples in the Alzheimer's disease multi-modal dataset into a pre-established Alzheimer's disease classification model, and adjust the parameters of the Alzheimer's disease classification model to the optimal through the error backpropagation algorithm to obtain an optimized Alzheimer's disease classification model;

[0046] A classification module, configured to obtain subject sample data, input the subject sample data into the optimized Alzheimer's disease classification model, and output a classification result.

[0047] In another aspect of the present invention, to achieve the above object, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The computer program stored in the memory can run on the processor. When the processor loads and executes the computer program, the above-mentioned Alzheimer's disease classification method based on multi-modal evidence deep learning is adopted.

[0048] Advantages of the present invention:

[0049] The present invention innovatively introduces the deep evidence learning theory into the field of early diagnosis of Alzheimer's disease, uses a dedicated deep neural network to learn the features of different modalities such as MRI, PET, and non-imaging data, extracts key evidence required for decision-making through the Softplus activation function, and introduces evidence-based uncertainty estimation technology to effectively quantify the class credibility and classification result uncertainty of various modalities in the classification decision, reflect the data quality differences and noise levels between different modalities, overcome the problem of overconfidence caused by the use of the Softmax activation function in existing deep neural networks, provide a reliable basis for risk assessment in the multi-modal fusion process, and enhance the robustness of the Alzheimer's disease classification method.

[0050] The present invention innovatively proposes a decision-level multi-modal evidence fusion method. Different from existing decision-level multi-modal fusion methods, this method particularly focuses on the decision risks of each modality, comprehensively considers the similarity and conflict between different modality evidences, and effectively fuses multiple modality evidences based on the Dempster combination rule, so as to form a credible and reliable classification decision. This fusion method not only improves the accuracy and robustness of Alzheimer's disease classification, but also can give an overall classification uncertainty assessment of the final decision, providing better model interpretability and decision credibility.

[0051] The decision-level fusion method proposed by the present invention overcomes the instability that the fusion results are very different due to the slight change of the basic probability assignment function when using the decision rule based on the traditional Dempster-Shafer evidence theory for fusion, and the problem of producing results contrary to common sense when dealing with highly conflicting evidences. Brief Description of the Drawings

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0053] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0054] Figure 2 It is a schematic diagram of the Alzheimer's disease classification model structure of the embodiment of the present invention;

[0055] Figure 3 It is a schematic diagram of the system structure of the present invention. Detailed Embodiments

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

[0057] Embodiment 1:

[0058] As Figure 1 shown, an Alzheimer's disease classification method based on multi-modal evidence deep learning, the method includes the following steps:

[0059] S101: Obtain the Alzheimer's disease multi-modal dataset; input the samples in the Alzheimer's disease multi-modal dataset into the pre-established Alzheimer's disease classification model, and adjust the parameters of the Alzheimer's disease classification model to the optimal through the error backpropagation algorithm to obtain the optimized Alzheimer's disease classification model; wherein, the process of processing by the pre-established Alzheimer's disease classification model includes data processing, evidence extraction, class confidence and classification result uncertainty estimation, and decision fusion.

[0060] Create an Alzheimer's disease multi-modal dataset. The Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset was constructed for the early detection and tracking of Alzheimer's disease (AD), mainly including data types such as imaging data, clinical cognitive test scales, biomarkers, and genetic data. ADNI has developed a set of standardized protocols. Before each subject undergoes various data acquisitions, they need to take a series of neuropsychiatric clinical cognitive test scales, such as the Mini-Mental State Examination (MMSE) and the Clinical Dementia Rating (CDR), to preliminarily evaluate the subject's status and classify the subjects into AD patients, mild cognitive impairment (MCI) patients, and cognitively normal (CN) subjects. The ADNI dataset contains samples of 483 CN subjects, 551 MCI subjects, and 437 AD subjects.

[0061] The process of processing the Alzheimer's disease multi-modal data includes:

[0062] Perform head motion correction, remove the neck, image registration, skull stripping, brain tissue extraction, bias field correction, and slicing on the input MRI image to obtain MRI samples; perform image registration, skull stripping, bias field correction, and slicing on the input PET image to obtain PET samples; directly perform vector splicing on the input non-imaging data of clinical cognitive test scales, biomarkers, and genes to obtain non-imaging samples.

[0063] Specifically, the preprocessing of the MRI image includes the following steps:

[0064] Perform head motion correction on the MRI image to eliminate image blurring and geometric deformation caused by the subject's head movement during scanning. The line connecting the midpoint of the posterior edge of the anterior commissure (AC) to the midpoint of the anterior edge of the posterior commissure (PC) is called the AC-PC reference line. In this embodiment, the Statistical Parametric Mapping (SPM) tool in the MATLAB built-in toolbox is used to correct the MRI image with the AC-PC reference line to correct the positioning deviation and geometric deformation in the image, and thus achieve head motion correction.

[0065] Use the robustfov tool in the neuroimaging data analysis software package FSL (FMRIB’s Software Library) to remove the neck part from the motion-corrected MRI images, leaving only the brain region for subsequent registration of the images to the MNI standard space.

[0066] Use the non-linear image registration tool (FMRIB’s Non-linear Image Registration Tool, FNIRT) in the neuroimaging data analysis software package FSL. Taking the MNI152_T1_2mm.nii.gz image as the registration reference image, register the MRI images to the MNI152 standard space according to the affine transformation.

[0067] Use the Brain Extraction Tool (BET) in the neuroimaging data analysis software package FSL to complete skull stripping and remove non-brain regions such as the skull and soft tissues in the MRI images that are irrelevant to the disease.

[0068] Use the Automated Segmentation Tool (FAST) in the neuroimaging data analysis software package FSL to extract gray matter, white matter, and cerebrospinal fluid tissues from the MRI images and combine them into one image for subsequent processing.

[0069] Use the N4BiasFieldCorrection.sh module algorithm integrated in the Advanced Normalization Tools (ANTs) to perform bias field correction on the MRI images to eliminate the problem of uneven brightness caused by changes in the device magnetic field during scanning, making the gray level distribution of each region of the image more uniform.

[0070] Perform slicing operations on the MRI images after the above processing from three different sections: the coronal plane, the sagittal plane, and the axial plane, and scale the size of each slice to 112×112 pixels through the bilinear interpolation algorithm.

[0071] Preprocessing of PET images

[0072] Since in this embodiment, the MRI images of the same subject are used as an intermediary, through multi-stage registrations, first register the PET images to the MRI image structure space and then to the MNI152 standard space, so there is no need to perform motion correction operations on the PET images. Additionally, since the PET images themselves do not contain the neck, there is also no need to perform neck removal operations on the PET images. The preprocessing of the PET images includes the following steps:

[0073] First, use the linear image registration tool (FMRIB’s Linear Image Registration Tool, FLIRT) in the FSL software package to register the PET image to the MRI image structure space of its corresponding subject according to the affine transformation. Secondly, use the non-linear image registration tool (FMRIB’s Non-linear Image Registration Tool, FNIRT) in the FSL software package to register the PET image in the MRI image structure space to the MNI152 standard space.

[0074] Use the brain extraction tool (Brain Extraction Tool, BET) in the FSL software package to complete skull stripping and remove non-brain regions unrelated to the disease, such as the skull and soft tissues, from the PET image.

[0075] Adopt the N4BiasFieldCorrection.sh module algorithm integrated in the Advanced Normalization Tools (ANTs) to perform bias field correction on the PET image to eliminate the problem of uneven brightness caused by the change of the device magnetic field during scanning, making the gray distribution of each region of the image more uniform.

[0076] Perform slicing operations on the PET image after the above processing from three different sections: the coronal plane, the sagittal plane, and the axial plane, and scale the size of each slice to 112×112 pixels through the bilinear interpolation algorithm.

[0077] Preprocessing of non-imaging data

[0078] Directly perform vector splicing on the input non-imaging data such as clinical cognitive test scales, biomarkers, genes, etc. to obtain non-imaging samples of 323 dimensions.

[0079] The process of processing by the pre-established Alzheimer's disease classification model includes evidence extraction, estimation of class confidence and classification result uncertainty, and decision fusion;

[0080] The process of the evidence extraction includes:

[0081] MRI evidence extraction, PET evidence extraction, and non-imaging evidence extraction:

[0082] MRI evidence extraction is used to extract MRI evidence supporting the k-th category from MRI samples The superscript M in it represents the MRI modality, and the subscript k represents the index number of the Alzheimer's disease category, where k = 1, 2, …, K, and K is the number of Alzheimer's disease categories. The larger the value of indicates that the MRI modality has a higher degree of confidence in supporting the k-th category; PET evidence extraction is used to extract PET evidence supporting the k-th category from PET samples. The superscript P in it represents the PET modality, and the subscript k represents the index number of the Alzheimer's disease category, where k = 1, 2, …, K. The larger the value of indicates that the PET modality has a higher degree of confidence in supporting the k-th category; non-imaging evidence extraction is used to extract non-imaging evidence supporting the k-th category from non-imaging samples. The superscript D in it represents the non-imaging modality, and the subscript k represents the index number of the Alzheimer's disease category, where k = 1, 2, …, K. The larger the value of indicates that the non-imaging modality has a higher degree of confidence in supporting the k-th category.

[0083] In this embodiment, the subjects are divided into three categories: AD patients, MCI patients, and cognitively normal (CN) subjects, with K = 3; the MRI evidence extraction module uses a pre-trained MobileViT backbone to perform feature learning on the preprocessed MRI samples, and generates MRI evidence supporting the k-th category through a fully connected layer with 3 neurons and a Softplus activation function. The PET evidence extraction module uses a pre-trained MobileViT backbone to perform feature learning on the preprocessed PET samples, and generates PET evidence supporting the k-th category through a fully connected layer with 3 neurons and a Softplus activation function. The non-imaging evidence extraction module first performs feature learning on the preprocessed non-imaging samples through two fully connected layers, and then generates non-imaging evidence supporting the k-th category through a fully connected layer with 3 neurons and a Softplus activation function. The number of neurons in the first two fully connected layers is set to 1024 and 64 respectively, and Dropout is used to randomly inactivate neurons to prevent network overfitting, with the initial value of Dropout set to 0.5.

[0084] The category confidence and classification result uncertainty estimation utilize the evidence of each modality supporting the k-th category. to generate the confidence of the Alzheimer's disease category provided by the corresponding modality. and the classification result uncertainty u. m The evidence of each modality supporting the k-th category. The more, the higher the credibility of the Alzheimer's category provided by the corresponding modality is; the more the total evidence of all categories supported by each modality, the lower the uncertainty u m of the classification result provided by the corresponding modality, and thus a more credible classification result is obtained; where u m ≥0, and the superscript m in u m represents the modality category, m ∈ {M, P, D}, m = M represents the MRI modality, m = P represents the PET modality, m = D represents the non-imaging modality, and k = 1, 2, 3. The specific steps are as follows:

[0085] Based on the subjective logic theory, the evidence of each modality supporting the k-th category extracted is converted into the concentration parameter of the Dirichlet distribution The calculation formula is:

[0086]

[0087] where and the superscript m in represents the modality category, m ∈ {M, P, D}, k = 1, 2, 3;

[0088] For each sample in the three-class classification, the subjective logic theory assigns 3 Alzheimer's category credibilities of each modality and the classification result uncertainty u m , and satisfies:

[0089]

[0090] where u m ≥0;

[0091] According to the evidence of each modality supporting the k-th category calculate the Alzheimer's category credibility provided by the corresponding modality and the classification result uncertainty u m , and the calculation formulas are respectively:

[0092]

[0093] where S m is the Dirichlet strength, S m the superscript m in represents the modality category, m ∈ {M, P, D}, k = 1, 2, 3.

[0094] The decision fusion uses the Alzheimer's category credibilities provided by each modality and the uncertainty u of the classification result m , according to the Dempster combination rule, evaluate the similarity and conflict between the evidence of each modality, and obtain the credibility b of the Alzheimer's disease category after multi-modal fusion k and the overall uncertainty u of the classification result, and output the final classification result;

[0095] Specifically, use the credibility of the Alzheimer's disease category provided by each modality and the uncertainty u of the classification result m , and obtain the subjective opinions of the MRI modality, PET modality, and non-imaging modality respectively And according to the Dempster combination rule, calculate the subjective opinion after fusing the three modalities The calculation formulas are respectively:

[0096]

[0097] Among them, C is the normalization factor, and the calculation formula is:

[0098]

[0099] Calculate the evidence e that supports the k-th category after fusing the three modalities k , the concentration parameter α of the Dirichlet distribution k and the Dirichlet strength S, and the calculation formulas are respectively:

[0100]

[0101] The predicted probability p that the input sample belongs to the k-th category k is equal to the mean value of the Dirichlet distribution, and the calculation formula is

[0102]

[0103] Among them, k = 1, 2, 3, and the category corresponding to the maximum predicted probability is the final classification result.

[0104] S102: Obtain the subject sample data, input the subject sample data into the optimized Alzheimer's disease classification model, and output the classification result.

[0105] The loss function used for training by inputting the Alzheimer's disease multi-modal dataset into the pre-established Alzheimer's disease classification model is defined as:

[0106]

[0107] Among them, L(α M ), L(αP ) and \(L(\alpha\) D ) represent the independent loss functions of the MRI modality, PET modality, and non-imaging modality respectively, which are used to constrain the classification accuracy of a single modality during independent learning; \(\lambda\) is a hyperparameter that adjusts the weight of the single-modal loss, where \(0 < \lambda < 1\); \(L(\alpha)\) represents the comprehensive loss function after evidence fusion of the three modalities, which takes into account the interaction and complementary information between the three modalities and is used to constrain the overall decision-making accuracy of the model; \(L(\alpha\) M ), \(L(\alpha\) P ), \(L(\alpha\) D ), and \(L(\alpha)\) are calculated as follows:

[0108]

[0109]

[0110] where \(y\) k represents the true class label of the training sample assigned to the \(k\)-th class. When using one-hot encoding, if the training sample is assigned to the \(k\)-th class, then \(y\) k = 1; otherwise, \(y\) k = 0; \(\Psi(\cdot)\) is the Digamma function, which is monotonically increasing on \((0, +\infty)\).

[0111] Specifically, the solution of the present invention will be further elaborated by the following embodiments:

[0112] During the model training process of this embodiment, the batch size is set to 64, the Adam optimizer is used to update the model parameters, the initial learning rate is set to 0.001, and the learning rate is adjusted by the sine oscillation method.

[0113] Embodiment 2: Second, as Figure 3 shown, to achieve the above object, the present invention discloses an Alzheimer's disease classification system based on multi-modal evidence deep learning, including:

[0114] A model construction module 11, which is used to construct an Alzheimer's disease classification model based on multi-modal evidence deep learning. This model includes a data processing module, an evidence extraction module, a class confidence and classification uncertainty estimation module, and a decision fusion module;

[0115] A model optimization module 12, which is used to input the samples in the Alzheimer's disease multi-modal dataset into the pre-established Alzheimer's disease classification model, and adjust the parameters of the Alzheimer's disease classification model to the optimal through the error backpropagation algorithm to obtain an optimized Alzheimer's disease classification model;

[0116] A classification module 13, configured to obtain subject sample data, input the subject sample data into an optimized Alzheimer's disease classification model, and output a classification result.

[0117] Based on the same inventive concept, the present invention further provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is configured to execute the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0118] It should be further noted that, based on the same inventive concept, the present invention further provides a computer storage medium, on which a computer program is stored, and the computer program, when run by a processor, executes the above method. The storage medium may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may, for example, but not be limited to, be an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0119] In the description of this specification, the descriptions referring to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0120] The above has shown and described the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and what is described in the above embodiments and the specification is only to illustrate the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure will have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure claimed.

Claims

1. An Alzheimer's disease classification method based on multimodal evidence deep learning, characterized by: The method comprises the following steps: Acquire an Alzheimer's disease multimodal data set; input samples in the Alzheimer's disease multimodal data set into a pre-established Alzheimer's disease classification model, and adjust the parameters of the Alzheimer's disease classification model to the optimum through an error back propagation algorithm to obtain an optimized Alzheimer's disease classification model; wherein the processing process of the pre-established Alzheimer's disease classification model includes data processing, evidence extraction, category credibility and classification result uncertainty estimation, and decision fusion; Obtain subject sample data, input the subject sample data into the optimized Alzheimer's disease classification model, and output the classification result.

2. The Alzheimer's disease classification method based on multimodal evidence deep learning according to claim 1 is characterized in that: The samples in the Alzheimer's disease multimodal dataset include imaging data of magnetic resonance imaging (MRI) and positron emission tomography (PET), non-imaging data of clinical cognitive test scales, biomarkers and genes, and corresponding Alzheimer's disease category labels.

3. The Alzheimer's disease classification method based on multimodal evidence deep learning according to claim 1, characterized in that: The process of processing Alzheimer's disease multimodal data includes: The input MRI image is subjected to head motion correction, neck removal, image registration, skull stripping, brain tissue extraction, bias field correction and slicing processing to obtain MRI samples. The input PET image is subjected to image registration, skull stripping, bias field correction and slicing processing to obtain PET samples. The input non-imaging data of clinical cognitive test scales, biomarkers and genes are directly vectorized and spliced ​​to obtain non-imaging samples.

4. The Alzheimer's disease classification method based on multimodal evidence deep learning according to claim 1, characterized in that: The process of evidence extraction includes: MRI evidence extraction, PET evidence extraction and non-imaging evidence extraction: MRI evidence extraction uses deep neural network and Softplus activation function to extract MRI evidence supporting the kth category from MRI samples The superscript M in represents the MRI modality, and the subscript k represents the index number of the Alzheimer's disease category, k = 1, 2, ..., K, where K is the number of Alzheimer's disease categories. The larger the value, the higher the confidence that the MRI modality supports the kth category; PET evidence extraction uses a deep neural network and Softplus activation function to extract PET evidence supporting the kth category from PET samples. e k P The superscript P in the table represents the PET modality, and the subscript k represents the index of the Alzheimer's disease category, k = 1, 2, ..., K. The larger the value, the higher the confidence of the PET modality in supporting the kth category; non-imaging evidence extraction uses a deep neural network and Softplus activation function to extract non-imaging evidence supporting the kth category from non-imaging samples The superscript D in the table represents the non-imaging modality, and the subscript k represents the index number of the Alzheimer's disease category, k = 1, 2, ..., K, The larger the value of , the higher the confidence that the non-imaging modality supports the kth category.

5. The Alzheimer's disease classification method based on multimodal evidence deep learning according to claim 4 is characterized in that: The class credibility and classification result uncertainty estimation use the extracted evidence of each modality supporting the kth class Generate the credibility of the Alzheimer's disease category provided by the corresponding modality and the classification result uncertainty u m ; Evidence from each modality supporting the kth category The more, the more credibility the corresponding modality provides for the Alzheimer's disease category The higher the sum of evidence supporting all categories of each modality, the greater the uncertainty u of the classification result provided by the corresponding modality. m The lower it is, the more reliable the classification result is. ≥0,u m ≥0, and u m The superscript m in the formula represents the modality category, m∈{M,P,D}, m=M represents MRI modality, m=P represents PET modality, m=D represents non-imaging modality, k=1,2,…,K; Use the extracted evidence from each modality to support the kth category Generate the credibility of the Alzheimer's disease category provided by the corresponding modality and the classification result uncertainty u m The process is as follows: The extracted evidence of each modality supporting the kth category Convert to Dirichlet distribution concentration parameter The calculation formula is: in, and The superscript m in represents the modal category, m∈{M,P,D}, k=1,2,…,K; For each sample in the K categories, assign K Alzheimer’s disease category credibility of each modality and the classification result uncertainty u m , and satisfy: in, u m ≥0; Evidence supporting the kth category based on each modality Calculate the credibility of the Alzheimer's disease category provided by the corresponding modality and the classification result uncertainty u m , the calculation formulas are: Among them, S m is the Dirichlet intensity, S m The superscript m in represents the modal category, m∈{M,P,D}, k=1,2,…,K.

6. The Alzheimer's disease classification method based on multimodal evidence deep learning according to claim 5, characterized in that: The decision fusion uses the confidence of the Alzheimer's disease category provided by each modality and the classification result uncertainty u m According to the Dempster combination rule, the similarity and conflict between the evidence of each modality are evaluated, and the credibility b of the Alzheimer's disease category after multimodal fusion is obtained. k And the overall classification result uncertainty u, output the final classification result; The steps to output the final classification results according to the Dempster combination rule are as follows: Using the credibility of Alzheimer's disease category provided by each modality and the classification result uncertainty u m , and obtain subjective opinions of MRI modality, PET modality, and non-imaging modality respectively According to Dempster's combination rule, the subjective opinion after the fusion of the three modalities is calculated The calculation formulas are: Where C is the normalization factor, calculated as Calculate the evidence e supporting the kth category after the fusion of the three modalities k 、Dirichlet distribution concentration parameter α k And Dirichlet intensity S, the calculation formulas are: The predicted probability p that the input sample belongs to the kth category k Equal to the mean of the Dirichlet distribution, calculated as: Among them, k = 1, 2, ..., K, and the category corresponding to the maximum prediction probability is taken as the final classification result.

7. The Alzheimer's disease classification method based on multimodal evidence deep learning according to claim 1, characterized in that: The loss function used for inputting samples in the Alzheimer's disease multimodal dataset into the pre-established Alzheimer's disease classification model for training is defined as: Among them, L(α M )、L(α P ) and L(α D ) represent the independent loss functions of MRI, PET and non-imaging modalities respectively; λ is a hyperparameter for adjusting the weight of single modality loss, 0<λ<1; L(α) represents the comprehensive loss function of the three modalities after evidence fusion; L(α M )、L(α P )、L(α D ) and L(α) are calculated as follows: Among them, y k Indicates that the training sample is assigned the true category label of the kth category. When one-hot encoding is used, if the training sample is assigned the kth category, then y k =1, otherwise y k =0; Ψ(·) is the Digamma function, which increases monotonically on (0, +∞).

8. An Alzheimer's disease classification system based on multimodal evidence deep learning, characterized by: include: A model building module is used to build an Alzheimer's disease classification model based on multimodal evidence deep learning. The model includes a data processing module, an evidence extraction module, a category credibility and classification uncertainty estimation module, and a decision fusion module; A model optimization module is used to input samples in the Alzheimer's disease multimodal data set into a pre-established Alzheimer's disease classification model, and adjust the parameters of the Alzheimer's disease classification model to the optimal value through an error back propagation algorithm to obtain an optimized Alzheimer's disease classification model; The classification module is used to obtain subject sample data, input the subject sample data into the optimized Alzheimer's disease classification model, and output the classification results.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: The memory stores a computer program that can be run on a processor. When the processor loads and executes the computer program, the Alzheimer's disease classification method based on multimodal evidence deep learning described in any one of claims 1 to 8 is adopted.

Images