Alzheimer's disease category prediction method and imaging method based on image fusion

Abstract

The application discloses an Alzheimer's disease category prediction method based on image fusion, comprising the following steps: acquiring existing and corresponding brain sMRI images and PET images, and preprocessing to obtain a training data set; constructing an Alzheimer's disease preliminary category prediction model and training to obtain an Alzheimer's disease category prediction model by using the training data set; inputting actual brain sMRI images and PET images into the Alzheimer's disease category prediction model to complete the category prediction of Alzheimer's disease based on image fusion. The application also discloses an imaging method comprising the Alzheimer's disease category prediction method based on image fusion. Through the application of the image fusion theory and the construction and training of the innovative category prediction model, the application not only realizes the category prediction and imaging of Alzheimer's disease based on multi-modal image fusion, but also has higher reliability and better accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image processing, specifically relating to an Alzheimer's disease category prediction method and imaging method based on image fusion. Background Technology

[0002] With the development of economy and technology and the improvement of people's living standards, people are paying more and more attention to health.

[0003] In recent years, Alzheimer's disease has received widespread attention and research. The classification of Alzheimer's disease is of great significance for both clinical and basic medical research.

[0004] Different brain images (commonly structural magnetic resonance imaging (sMRI) and positron emission tomography (PET) images) can provide different brain information; and based on different brain information, different predictions of Alzheimer's disease categories may be obtained.

[0005] Therefore, researchers have already used image fusion to predict the classification results of Alzheimer's disease. Existing category prediction schemes include image data-level fusion, image feature-level fusion, and decision-level fusion: image data-level fusion integrates images from different modalities into a unified image and uses a neural network algorithm for category prediction; image feature-level fusion integrates features from different modalities using different strategies to create a unified feature vector and uses a neural network algorithm for category prediction; and decision-level fusion uses different neural network classifiers to predict the classification results of images from different modalities, then fuses the predicted classification results to make a comprehensive decision.

[0006] Existing methods all assume that the importance of different modalities in all samples is fixed, and assign or learn a fixed weight for each modality to fuse the data and ultimately predict the category. However, the image quality of each sample may vary due to different acquisition methods, and the feature information from images of different qualities has varying impacts on the final classification decision. Therefore, existing Alzheimer's disease category prediction schemes all suffer from poor reliability and accuracy. Summary of the Invention

[0007] One of the objectives of this invention is to provide a highly reliable and accurate image fusion-based method for predicting Alzheimer's disease categories.

[0008] The second objective of this invention is to provide an imaging method that includes the aforementioned image fusion-based Alzheimer's disease category prediction method.

[0009] The image fusion-based Alzheimer's disease category prediction method provided by this invention includes the following steps:

[0010] S1. Obtain existing and corresponding brain sMRI and PET images;

[0011] S2. Preprocess the images obtained in step S1 to obtain the training dataset;

[0012] S3. Based on variational autoencoders, Dirichlet distribution, and Dempster-Shafer theory, a preliminary category prediction model for Alzheimer's disease is constructed.

[0013] S4. Using the training dataset obtained in step S2, train the preliminary Alzheimer's disease category prediction model constructed in step S3 to obtain the Alzheimer's disease category prediction model.

[0014] S5. Input the actual brain sMRI and PET images into the Alzheimer's disease category prediction model obtained in step S4 to complete the category prediction of Alzheimer's disease based on image fusion.

[0015] Step S2, which involves preprocessing the image obtained in step S1, specifically includes the following steps:

[0016] For the acquired sMRI images, the sMRI images are registered to a standard brain template space to unify the coordinate space; and the registered images are segmented to remove the influence of the skull region.

[0017] For the acquired PET images, an interpolation algorithm is used to adjust the PET images to the same resolution as the registered sMRI images;

[0018] Finally, random augmentation and voxel normalization were performed on the processed sMRI and PET images to complete the preprocessing.

[0019] Step S3, which involves constructing a preliminary Alzheimer's disease category prediction model based on variational autoencoders, Dirichlet distribution, and Dempster-Shafer theory, specifically includes the following steps:

[0020] The constructed preliminary category prediction model for Alzheimer's disease includes a first encoder module, a first channel attention module, a first variational sampling module, a first decoder module, a second encoder module, a second channel attention module, a second variational sampling module, a second decoder module, a category probability prediction module, a probability processing module, and a category prediction module.

[0021] The first encoder module, the first channel attention module, the first variational sampling module, and the first decoder module are connected in series; the second encoder module, the second channel attention module, the second variational sampling module, and the second decoder module are connected in series; the category probability prediction module, the probability processing module, and the category prediction module are connected in series; the output terminals of the first variational sampling module and the second variational sampling module are both connected to the input terminal of the category probability prediction module.

[0022] The first encoder module is used to encode features of the input sMRI image; the first channel attention module is used to enhance the features output by the first encoder module; the first variational sampling module is used to perform variational sampling on the features output by the first channel attention module; the first decoder module is used to decode the features output by the first variational sampling module, generate a generated sMRI image corresponding to the input sMRI image, and use it for training the preliminary Alzheimer's disease category prediction model.

[0023] The second encoder module is used to encode features of the input PET image; the second channel attention module is used to enhance features of the features output by the second encoder module; the second variational sampling module is used to perform variational sampling on features output by the second channel attention module; the second decoder module is used to decode features of the features output by the second variational sampling module, generate a generated PET image corresponding to the input PET image, and use it for training the preliminary Alzheimer's disease category prediction model.

[0024] The category probability prediction module is used to map the output of the first variational sampling module and the output of the second variational sampling module to obtain the category prediction probability of sMRI images for Alzheimer's disease and the category prediction probability of PET images for Alzheimer's disease.

[0025] The probability processing module is used to predict the probability of Alzheimer's disease category from sMRI images and PET images, based on the Dirichlet distribution, and to perform confidence and uncertainty calculations.

[0026] The category prediction module is used to fuse the confidence calculation results and uncertainty calculation results of the two categories of predicted probabilities based on Dempster-Shafer theory to obtain the final category prediction result of Alzheimer's disease.

[0027] The first encoder module and the second encoder module have the same structure, both including a first encoder layer, a second encoder layer, a third encoder layer, a fourth encoder layer and a fifth encoder layer; the first encoder layer, the second encoder layer, the third encoder layer, the fourth encoder layer and the fifth encoder layer are connected in series.

[0028] The first encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has one input channel, four output channels, five convolutional kernels, a kernel span of 1, and adds 2 padding units to all six sides of the 3D input during the convolution operation. The max pooling layer has two pooling kernels with a kernel span of 2. The batch normalization layer has four features.

[0029] The second encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 4 input channels, 8 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 8 features.

[0030] The third encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 8 input channels, 16 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 16 features.

[0031] The fourth encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 16 input channels, 32 output channels, 5 kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 32 features.

[0032] The fifth encoder layer includes one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 32 input channels, 64 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 64 features.

[0033] The first-channel attention module and the second-channel attention module have the same structure, both including an adaptive average pooling layer and two fully connected layers;

[0034] The adaptive average pooling layer is used to compress the spatial dimension of the features output by the corresponding encoder module to 1, so as to reduce the calculation of spatial dimension in subsequent processing.

[0035] The first fully connected layer is used to reduce the number of feature channels of the features output by the adaptive average pooling layer from 64 to 8;

[0036] The second fully connected layer is used to restore the number of feature channels of the features output by the first fully connected layer to 64, thereby obtaining the channel weight factor;

[0037] Finally, the channel weight factor is multiplied by the feature output by the corresponding encoder module to obtain the enhanced feature.

[0038] The first variational sampling module and the second variational sampling module have the same structure, including a first flat layer, a first fully connected layer, a second flattened layer, a second fully connected layer, and a sampling layer;

[0039] The first flattened layer and the first fully connected layer are connected; the second flattened layer and the second fully connected layer are connected; the outputs of the first fully connected layer and the second fully connected layer are both connected to the input of the sampling layer.

[0040] The enhanced features output by the corresponding channel attention module are flattened through the first flattening layer to obtain a 1-dimensional feature vector. The length of the obtained 1-dimensional feature vector is then mapped to 256 using the first fully connected layer. To avoid overfitting, a random dropout probability is set for the output of the neurons in the first fully connected layer during the feature vector calculation process. The output value of the dropped neurons is directly set to 0.

[0041] The enhanced features output by the corresponding channel attention module are flattened through a second flattening layer to obtain a 1D feature vector. A second fully connected layer is then used to map the length of this 1D feature vector to 256. To avoid overfitting, a random dropout probability is set for the output of neurons in the second fully connected layer during feature vector calculation; the output value of dropped neurons is directly set to 0.

[0042] The 256-length 1D feature vector output by the first fully connected layer is used as the mean parameter of the Gaussian distribution, and the 256-length 1D feature vector output by the second fully connected layer is used as the variance parameter of the Gaussian distribution; the sampling layer samples the Gaussian distribution to obtain variational sampling features.

[0043] The first decoder module and the second decoder module have the same structure, both including a recovery layer, a first decoder layer, a second decoder layer, a third decoder layer, a fourth decoder layer, and a fifth decoder layer;

[0044] The recovery layer, the first decoder layer, the second decoder layer, the third decoder layer, the fourth decoder layer, and the fifth decoder layer are connected in series.

[0045] The recovery layer consists of one fully connected layer and one Rectified Activation Unit (ReLU). The recovery layer is used to restore the size of the input features to the same size as the enhanced features output by the corresponding channel attention module through the fully connected layer and the ReLU.

[0046] The first decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 64 input channels, 32 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 32 features.

[0047] The second decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one Rectified Activation Unit (ReLU). The 3D convolutional layer has 32 input channels, 16 output channels, 5 kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 16 features.

[0048] The third decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 16 input channels, 8 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 8 features.

[0049] The fourth decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 8 input channels, 4 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 4 features.

[0050] The fifth decoder layer includes one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 4 input channels, 1 output channel, 5 convolutional kernels, and a kernel span of 1. The padding added to all six sides of the 3D input during the convolution operation is 2. The max pooling layer has 2 pooling kernels and a kernel span of 2. The batch normalization layer has 1 feature.

[0051] The category probability prediction module includes a fully connected layer, a dropout layer, and an activation function layer;

[0052] For the features output by the first variational sampling module, a fully connected layer is used to map the size of the features output by the first variational sampling module from 256 to 2, and a dropout layer is used to prevent overfitting with a dropout rate of 0.2. Finally, the activation function layer uses the Softplus activation function to process the features output by the fully connected layer to obtain the predicted probability of the sMRI image for Alzheimer's disease category.

[0053] For the features output by the second variational sampling module, a fully connected layer is used to map the size of the features output by the first variational sampling module from 256 to 2, and a dropout layer is used to prevent overfitting with a dropout rate of 0.2. Finally, the activation function layer uses the Softplus activation function to process the features output by the fully connected layer to obtain the predicted probability of the PET image for Alzheimer's disease category.

[0054] The probability processing module includes the following steps:

[0055] Predicting the k-th category of Alzheimer's disease from sMRI images The Dirichlet distribution parameters of the sMRI images were calculated. for Predicting the k-th category of Alzheimer's disease from PET images The Dirichlet distribution parameters of the PET image were calculated. for

[0056] For sMRI images, the confidence level of the predicted probability of the k-th category of Alzheimer's disease by the sMRI image is calculated using the following formula. and uncertainty

[0057]

[0058]

[0059] In the formula The Dirichlet intensity of the k-th class in the sMRI image and K represents the total number of categories of Alzheimer's disease;

[0060] For PET images, the confidence level of the predicted probability of the k-th category of Alzheimer's disease by the PET images is calculated using the following formula. and uncertainty

[0061]

[0062]

[0063] In the formula The Dirichlet intensity of the k-th class in the PET image and

[0064] The category prediction module includes the following steps in its processing:

[0065] Based on the Dempster-Shafer theory, the following fusion steps are designed:

[0066] The following formula is used to assess the confidence level of sMRI images in predicting the probability of the k-th category of Alzheimer's disease. The reliability of PET images in predicting the probability of the k-th category of Alzheimer's disease Perform credibility fusion:

[0067]

[0068] In the formula b k The confidence fusion value is the predicted probability of the k-th category of Alzheimer's disease; C is the conflict value between the confidence of sMRI images and the confidence of PET images.

[0069] The following formula is used to calculate the uncertainty in predicting the k-th category of Alzheimer's disease using sMRI images. Uncertainty in predicting the probability of Alzheimer's disease by PET images for the k-th category Uncertainty fusion:

[0070]

[0071] In the formula u k The uncertainty fusion value for predicting the probability of the k-th category of Alzheimer's disease;

[0072] The following formula is used to calculate the predicted probability e of the k-th category of Alzheimer's disease. k :

[0073] e k =b k ×S

[0074] In the formula, S is an intermediate variable and

[0075] Finally, the category with the highest predicted category probability is selected as the final category prediction result for Alzheimer's disease.

[0076] The training described in step S4 specifically includes the following steps:

[0077] The following formula is used as the loss function for sMRI images.

[0078]

[0079] In the formula σ 1 μ is the feature vector output by the first fully connected layer of the first variational sampling module. 1 x is the feature vector output by the second fully connected layer of the first variational sampling module; 1 The original input sMRI image, x 1 ' is the sMRI image generated by the first decoder module; || ||2 is the L2 norm, used to construct the L2 norm loss function, i.e., the least squares error;

[0080] The following formula is used as the loss function for PET images.

[0081]

[0082] In the formula σ 2 μ is the feature vector output by the first fully connected layer of the second variational sampling module. 2 x is the feature vector output by the second fully connected layer of the second variational sampling module; 2 The original input PET image, x 2 'PET images generated for the second decoder module;'

[0083] Constructing the feature extraction loss function L gen for

[0084] The following formula is used as the fusion classification loss function L(α) i ):

[0085]

[0086]

[0087]

[0088] In the formula L ace (α i ) represents the category prediction loss; α i Let be the parameters of the Dirichlet distribution and α i =e i +1, α i α of several categories ik Composition; λ is the weight coefficient and its value is the ratio of the current training iteration to the total training iterations; For KL divergence loss; y ikLet S be the true label of the k-th class in the i-th image; ψ() is the bigamma function; S i Γ is the Dirichlet intensity; Γ() is the gamma function; For the updated Dirichlet distribution parameters;

[0089] The following formula is used as the classification loss function for sMRI images.

[0090]

[0091]

[0092]

[0093] In the formula Category prediction loss for sMRI images; The Dirichlet distribution parameters of the sMRI image; KL divergence loss for sMRI images; Let be the true label of the k-th class in the i-th sMRI image; The Dirichlet distribution intensity of the sMRI image; Dirichlet distribution parameters for the updated sMRI images;

[0094] The following formula is used as the classification loss function for PET images.

[0095]

[0096]

[0097]

[0098] In the formula Loss for class prediction of PET images; The Dirichlet distribution parameters of the PET image; KL divergence loss for PET images; The true label for the k-th category in the i-th PET image; The Dirichlet distribution intensity of the PET image; The Dirichlet distribution parameters for the updated PET images;

[0099] Construct the overall classification loss function L cls for

[0100] Finally, the total loss function L is constructed as L = L gen +L cls ;

[0101] During training, the model parameters are trained using an adaptive moment estimation optimizer based on the constructed total loss function.

[0102] Step S5 involves inputting the actual brain sMRI and PET images into the Alzheimer's disease category prediction model obtained in step S4 to complete the image fusion-based Alzheimer's disease category prediction. This specifically includes the following steps:

[0103] Preprocessing of actual brain sMRI and PET images:

[0104] For actual sMRI images, the sMRI images are registered to a standard brain template space to unify the coordinate space; and the registered images are segmented to remove the influence of the skull region.

[0105] For the actual PET images, an interpolation algorithm is used to adjust the PET images to the same resolution as the registered sMRI images;

[0106] Delete the first and second decoders of the Alzheimer's disease category prediction model obtained in step S4 to obtain the Alzheimer's disease category prediction trimming model.

[0107] The preprocessed brain sMRI and PET images are input into the Alzheimer's disease category prediction pruning model to obtain the final Alzheimer's disease category prediction results.

[0108] The present invention also provides an imaging method including the aforementioned image fusion-based Alzheimer's disease category prediction method, comprising the following steps:

[0109] A. Obtain sMRI and PET images of the brain to be classified;

[0110] B. Using the image fusion-based Alzheimer's disease category prediction method described above, the brain sMRI and PET images to be classified obtained in step A are used to predict the Alzheimer's disease category.

[0111] C. The category prediction results obtained in step B are used for secondary imaging of the brain sMRI and PET images to be classified obtained in step A to obtain brain sMRI and PET images with Alzheimer's disease category prediction results.

[0112] The image fusion-based Alzheimer's disease category prediction and imaging method provided by this invention, through the application of image fusion theory and the construction and training of an innovative category prediction model, not only realizes Alzheimer's disease category prediction and imaging based on multimodal image fusion, but also has higher reliability and better accuracy. Attached Figure Description

[0113] Figure 1 This is a schematic diagram of the method flow for the category prediction method of the present invention.

[0114] Figure 2 This diagram illustrates a comparison of the prediction results of the category prediction method of this invention with existing category prediction methods.

[0115] Figure 3 This is a schematic diagram of the imaging method of the present invention. Detailed Implementation

[0116] like Figure 1 The diagram shows a flowchart of the category prediction method of the present invention: The Alzheimer's disease category prediction method based on image fusion provided by the present invention includes the following steps:

[0117] S1. Obtain existing and corresponding brain sMRI and PET images;

[0118] S2. Preprocess the images obtained in step S1 to obtain the training dataset; specifically, this includes the following steps:

[0119] For the acquired sMRI images, the sMRI images are registered (preferably using the FLS tool) to the standard brain template space MNI152 to unify the coordinate space. At this time, the resolution of the sMRI images is consistent at 181×217×181. The registered images are then segmented (preferably using the FLS tool) to remove the skull from the images to eliminate the influence of the skull region.

[0120] For the acquired PET images, an interpolation algorithm (preferably cubic spline interpolation) is used to adjust the PET images to the same resolution as the registered sMRI images;

[0121] Finally, random augmentation and voxel normalization were performed on the obtained processed sMRI and PET images to complete the preprocessing process.

[0122] S3. Based on variational autoencoders, Dirichlet distribution, and Dempster-Shafer theory, construct a preliminary category prediction model for Alzheimer's disease; specifically including the following steps:

[0123] The constructed preliminary category prediction model for Alzheimer's disease includes a first encoder module, a first channel attention module, a first variational sampling module, a first decoder module, a second encoder module, a second channel attention module, a second variational sampling module, a second decoder module, a category probability prediction module, a probability processing module, and a category prediction module.

[0124] The first encoder module, the first channel attention module, the first variational sampling module, and the first decoder module are connected in series; the second encoder module, the second channel attention module, the second variational sampling module, and the second decoder module are connected in series; the category probability prediction module, the probability processing module, and the category prediction module are connected in series; the output terminals of the first variational sampling module and the second variational sampling module are both connected to the input terminal of the category probability prediction module.

[0125] The first encoder module is used to encode features of the input sMRI image; the first channel attention module is used to enhance the features output by the first encoder module; the first variational sampling module is used to perform variational sampling on the features output by the first channel attention module; the first decoder module is used to decode the features output by the first variational sampling module, generate a generated sMRI image corresponding to the input sMRI image, and use it for training the preliminary Alzheimer's disease category prediction model.

[0126] The second encoder module is used to encode features of the input PET image; the second channel attention module is used to enhance features of the features output by the second encoder module; the second variational sampling module is used to perform variational sampling on features output by the second channel attention module; the second decoder module is used to decode features of the features output by the second variational sampling module, generate a generated PET image corresponding to the input PET image, and use it for training the preliminary Alzheimer's disease category prediction model.

[0127] The category probability prediction module is used to map the output of the first variational sampling module and the output of the second variational sampling module to obtain the category prediction probability of sMRI images for Alzheimer's disease and the category prediction probability of PET images for Alzheimer's disease.

[0128] The probability processing module is used to predict the probability of Alzheimer's disease category from sMRI images and PET images, based on the Dirichlet distribution, and to perform confidence and uncertainty calculations.

[0129] The category prediction module is used to fuse the confidence calculation results and uncertainty calculation results of the two categories of predicted probabilities based on Dempster-Shafer theory to obtain the final category prediction result of Alzheimer's disease.

[0130] In practice:

[0131] The first encoder module and the second encoder module have the same structure, both including a first encoder layer, a second encoder layer, a third encoder layer, a fourth encoder layer and a fifth encoder layer; the first encoder layer, the second encoder layer, the third encoder layer, the fourth encoder layer and the fifth encoder layer are connected in series.

[0132] The first encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has one input channel, four output channels, five convolutional kernels, a kernel span of 1, and adds 2 padding units to all six sides of the 3D input during the convolution operation. The max pooling layer has two pooling kernels with a kernel span of 2. The batch normalization layer has four features.

[0133] The second encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 4 input channels, 8 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 8 features.

[0134] The third encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 8 input channels, 16 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 16 features.

[0135] The fourth encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 16 input channels, 32 output channels, 5 kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 32 features.

[0136] The fifth encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 32 input channels, 64 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 64 features.

[0137] In practice:

[0138] The first and second channel attention modules have the same structure, both including an adaptive average pooling layer and two fully connected layers; the channel attention modules are used to further enhance the representativeness of features.

[0139] The adaptive average pooling layer is used to compress the spatial dimension of the features output by the corresponding encoder module to 1, so as to reduce the calculation of spatial dimension in subsequent processing.

[0140] The first fully connected layer is used to reduce the number of feature channels of the features output by the adaptive average pooling layer from 64 to 8;

[0141] The second fully connected layer is used to restore the number of feature channels of the features output by the first fully connected layer to 64, thereby obtaining the channel weight factor;

[0142] Finally, the channel weight factor is multiplied by the feature output by the corresponding encoder module to obtain the enhanced feature;

[0143] In practice:

[0144] The first variational sampling module and the second variational sampling module have the same structure, including a first flat layer, a first fully connected layer, a second flattened layer, a second fully connected layer, and a sampling layer;

[0145] The first flattened layer and the first fully connected layer are connected; the second flattened layer and the second fully connected layer are connected; the outputs of the first fully connected layer and the second fully connected layer are both connected to the input of the sampling layer.

[0146] The enhanced features output by the corresponding channel attention module are flattened through the first flattening layer to reduce the dimensionality to 1-dimensional feature vectors. The length of the 1-dimensional feature vectors is then mapped to 256 using the first fully connected layer. To avoid overfitting, a random dropout probability of 0.2 is set for the output of the neurons in the first fully connected layer during the feature vector calculation process. The output value of the dropped neurons is set to 0.

[0147] The enhanced features output by the corresponding channel attention module are flattened through the second flattening layer to reduce the dimensionality to 1-dimensional feature vectors. The length of the 1-dimensional feature vectors is then mapped to 256 using the second fully connected layer. To avoid overfitting, a random dropout probability of 0.2 is set for the output of the neurons in the second fully connected layer during the feature vector calculation process. The output value of the dropped neurons is set to 0.

[0148] The 256-length 1D feature vector output by the first fully connected layer is used as the mean parameter of the Gaussian distribution, and the 256-length 1D feature vector output by the second fully connected layer is used as the variance parameter of the Gaussian distribution; the sampling layer samples the Gaussian distribution to obtain variational sampling features;

[0149] In practice:

[0150] The first decoder module and the second decoder module have the same structure, both including a recovery layer, a first decoder layer, a second decoder layer, a third decoder layer, a fourth decoder layer, and a fifth decoder layer;

[0151] The recovery layer, the first decoder layer, the second decoder layer, the third decoder layer, the fourth decoder layer, and the fifth decoder layer are connected in series.

[0152] The recovery layer consists of one fully connected layer and one Rectified Activation Unit (ReLU). The recovery layer is used to restore the size of the input features to the same size as the enhanced features output by the corresponding channel attention module through the fully connected layer and the ReLU.

[0153] The first decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 64 input channels, 32 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 32 features.

[0154] The second decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one Rectified Activation Unit (ReLU). The 3D convolutional layer has 32 input channels, 16 output channels, 5 kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 16 features.

[0155] The third decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 16 input channels, 8 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 8 features.

[0156] The fourth decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 8 input channels, 4 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 4 features.

[0157] The fifth decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 4 input channels, 1 output channel, 5 convolutional kernels, and a kernel span of 1. The padding added to all six sides of the 3D input during the convolution operation is 2. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 1 feature.

[0158] The decoder module of the present invention functions to restore the variational sampling features to the original input image, and improve the representational ability of the variational sampling features by constraining the image generated by the encoder, thereby improving the accuracy of class probability prediction.

[0159] In practice:

[0160] The category probability prediction module includes a fully connected layer, a dropout layer, and an activation function layer;

[0161] For the features output by the first variational sampling module, a fully connected layer is used to map the size of the features output by the first variational sampling module from 256 to 2, and a dropout layer is used to prevent overfitting with a dropout rate of 0.2. Finally, the activation function layer uses the Softplus activation function to process the features output by the fully connected layer to obtain the predicted probability of the sMRI image for Alzheimer's disease category.

[0162] For the features output by the second variational sampling module, a fully connected layer is used to map the size of the features output by the first variational sampling module from 256 to 2, and a dropout layer is used to prevent overfitting with a dropout rate of 0.2. Finally, the activation function layer uses the Softplus activation function to process the features output by the fully connected layer to obtain the predicted probability of the PET image for Alzheimer's disease category.

[0163] In practice:

[0164] The probability processing module's processing steps include the following:

[0165] Predicting the k-th category of Alzheimer's disease from sMRI images The Dirichlet distribution parameters of the sMRI images were calculated. for Predicting the k-th category of Alzheimer's disease from PET images The Dirichlet distribution parameters of the PET image were calculated. for

[0166] For sMRI images, the confidence level of the predicted probability of the k-th category of Alzheimer's disease by the sMRI image is calculated using the following formula. and uncertainty

[0167]

[0168]

[0169] In the formula The Dirichlet intensity of the k-th class in the sMRI image and K represents the total number of categories of Alzheimer's disease;

[0170] For PET images, the confidence level of the predicted probability of the k-th category of Alzheimer's disease by the PET images is calculated using the following formula. and uncertainty

[0171]

[0172]

[0173] In the formula The Dirichlet intensity of the k-th class in the PET image and

[0174] The higher the predicted probability for class k, the higher the belief quality b assigned to class k. k The higher the probability of a class prediction, the greater the overall uncertainty u; conversely, the lower the predicted probabilities for all classes, the greater the overall uncertainty u; when the class prediction probability e is 0, then the belief quality b k The value is also 0, and the total uncertainty u is 1;

[0175] In practice:

[0176] The category prediction module's processing steps include the following:

[0177] Based on the Dempster-Shafer theory, the following fusion steps are designed:

[0178] The following formula is used to assess the confidence level of sMRI images in predicting the probability of the k-th category of Alzheimer's disease. The reliability of PET images in predicting the probability of the k-th category of Alzheimer's disease Perform credibility fusion:

[0179]

[0180] In the formula b kThe confidence fusion value is the predicted probability of the k-th category of Alzheimer's disease; C is the conflict value between the confidence of sMRI images and the confidence of PET images.

[0181] The following formula is used to calculate the uncertainty in predicting the k-th category of Alzheimer's disease using sMRI images. Uncertainty in predicting the probability of Alzheimer's disease by PET images for the k-th category Uncertainty fusion:

[0182]

[0183] In the formula u k The uncertainty fusion value for predicting the probability of the k-th category of Alzheimer's disease;

[0184] The following formula is used to calculate the predicted probability e of the k-th category of Alzheimer's disease. k :

[0185] e k =b k ×S

[0186] In the formula, S is an intermediate variable and

[0187] Finally, the category with the highest predicted probability is selected as the final category prediction result for Alzheimer's disease;

[0188] S4. Using the training dataset obtained in step S2, train the preliminary Alzheimer's disease category prediction model constructed in step S3 to obtain the Alzheimer's disease category prediction model.

[0189] In practice, the training includes the following steps:

[0190] The following formula is used as the loss function for sMRI images.

[0191]

[0192] In the formula σ 1 μ is the feature vector output by the first fully connected layer of the first variational sampling module. 1 x is the feature vector output by the second fully connected layer of the first variational sampling module; 1 The original input sMRI image, x 1 ' is the sMRI image generated by the first decoder module; || ||2 is the L2 norm, used to construct the L2 norm loss function, i.e., the least squares error;

[0193] The following formula is used as the loss function for PET images.

[0194]

[0195] In the formula σ 2 μ is the feature vector output by the first fully connected layer of the second variational sampling module. 2 x is the feature vector output by the second fully connected layer of the second variational sampling module; 2 The original input PET image, x 2 'PET images generated for the second decoder module;'

[0196] Constructing the feature extraction loss function L gen for This loss function serves two purposes: first, to constrain the latent spatial distribution, making it approach a standard normal distribution; and second, to ensure that the reconstructed output obtained by the decoder can restore the original input image to the greatest extent possible.

[0197] The following formula is used as the fusion classification loss function L(α) i ):

[0198]

[0199]

[0200]

[0201] In the formula L ace (α i ) represents the category prediction loss; α i Let be the parameters of the Dirichlet distribution and α i =e i +1, α i α of several categories ik Composition; λ is the weight coefficient and its value is the ratio of the current training iteration to the total training iterations; For KL divergence loss; y ik Let S be the true label of the k-th class in the i-th image; ψ() is the bigamma function; S i Γ is the Dirichlet intensity; Γ() is the gamma function; For the updated Dirichlet distribution parameters;

[0202] The dynamic selection of the weight coefficient λ is used to avoid focusing too much on KL divergence in the early stages of training, which would cause misclassified samples to converge to a uniform distribution too early.

[0203] The following formula is used as the classification loss function for sMRI images.

[0204]

[0205]

[0206]

[0207] In the formula Category prediction loss for sMRI images; The Dirichlet distribution parameters of the sMRI image; KL divergence loss for sMRI images; Let be the true label of the k-th class in the i-th sMRI image; The Dirichlet distribution intensity of the sMRI image; Dirichlet distribution parameters for the updated sMRI images;

[0208] The following formula is used as the classification loss function for PET images.

[0209]

[0210]

[0211]

[0212] In the formula Loss for class prediction of PET images; The Dirichlet distribution parameters of the PET image; KL divergence loss for PET images; The true label for the k-th category in the i-th PET image; The Dirichlet distribution intensity of the PET image; The Dirichlet distribution parameters for the updated PET images;

[0213] Construct the overall classification loss function L cls for

[0214] Finally, the total loss function L is constructed as L = L gen +L cls ;

[0215] During training, the model parameters are trained using an adaptive moment estimator based on the constructed total loss function;

[0216] The advantage of using the above loss function is that it constrains feature extraction from two aspects, trains the model to obtain significant features, and thus improves the accuracy of Alzheimer's disease category prediction. genConstraints are imposed from the perspective of feature reconstruction of the original image, so that the extracted features can contain as much salient information as possible from the original image; L cls Constraints are imposed from a classification perspective to maximize intra-class similarity and inter-class differences in extracted features, while also considering the differences in image quality between different individuals and modalities, thereby improving the accuracy and reliability of classification results.

[0217] S5. Input the actual brain sMRI and PET images into the Alzheimer's disease category prediction model obtained in step S4 to complete the image fusion-based Alzheimer's disease category prediction; specifically including the following steps:

[0218] Preprocessing of actual brain sMRI and PET images:

[0219] For actual sMRI images, the sMRI images are registered to a standard brain template space to unify the coordinate space; and the registered images are segmented to remove the influence of the skull region.

[0220] For the actual PET images, an interpolation algorithm is used to adjust the PET images to the same resolution as the registered sMRI images;

[0221] Delete the first and second decoders of the Alzheimer's disease category prediction model obtained in step S4 to obtain the Alzheimer's disease category prediction trimming model.

[0222] The preprocessed brain sMRI and PET images are input into the Alzheimer's disease category prediction pruning model to obtain the final Alzheimer's disease category prediction results.

[0223] like Figure 2 The diagram illustrates a comparison of the prediction results of the category prediction method of this invention with existing category prediction methods: The category prediction method of this invention is compared with existing category prediction methods that apply other image fusion strategies, including feature stitching (corresponding to 1) and decision-weighted summation (corresponding to 2). The category prediction method of this invention applies a reliable fusion strategy (corresponding to 3). Figure 2 The classification performance of the category prediction method of this invention was compared with that of existing category prediction methods on the categories of individuals with mild cognitive impairment (MCI) and normal (NC) and individuals with Alzheimer's disease (AD) and normal (NC). Performance metrics included: accuracy (ACC), specificity (PRE), sensitivity (SEN), precision (SPE), and F1 score. Figure 2As can be seen, the category prediction method of this invention outperforms two other existing category prediction methods in the classification tasks. This is because the category prediction method of this invention uniquely considers the confidence and uncertainty of the initial class prediction probability and designs a unique fusion strategy to integrate the confidence and uncertainty quality of the prediction probabilities of different images, rather than directly fusing them through feature concatenation and decision weighted summation. This helps to improve the accuracy of classification decisions.

[0224] like Figure 3 The diagram shown is a flowchart of the imaging method of the present invention: The imaging method disclosed in this invention, which includes the aforementioned image fusion-based Alzheimer's disease category prediction method, includes the following steps:

[0225] A. Obtain sMRI and PET images of the brain to be classified;

[0226] B. Using the image fusion-based Alzheimer's disease category prediction method described above, the brain sMRI and PET images to be classified obtained in step A are used to predict the Alzheimer's disease category.

[0227] C. The category prediction results obtained in step B are used for secondary imaging of the brain sMRI and PET images to be classified obtained in step A to obtain brain sMRI and PET images with Alzheimer's disease category prediction results.

[0228] The imaging method provided by this invention can be applied to medical devices that can simultaneously acquire brain sMRI and PET images, or can be set in a third device (such as a host computer).

[0229] If the imaging method of the present invention is applied to a medical device capable of simultaneously acquiring brain sMRI and PET images, the medical device first operates normally to acquire brain sMRI and PET images, then uses the image fusion-based Alzheimer's disease category prediction method provided by the present invention to perform corresponding category prediction, and then uses the imaging method provided by the present invention to directly perform secondary imaging on the acquired brain sMRI and PET images; finally, the medical device outputs the original brain sMRI and PET images, as well as brain sMRI and PET images with Alzheimer's disease category prediction results;

[0230] If the imaging method of the present invention is applied to a third device (such as a host computer), the third device needs to first acquire brain sMRI images and PET images, then use the image fusion-based Alzheimer's disease category prediction method provided by the present invention to perform corresponding category prediction, and then use the imaging method provided by the present invention to directly perform secondary imaging on the acquired brain sMRI images and PET images; finally, the third device outputs the original brain sMRI images and PET images, as well as brain sMRI images and PET images with Alzheimer's disease category prediction results;

[0231] The imaging method of this invention enables medical devices that can simultaneously acquire brain sMRI and PET images, or third devices that can simultaneously output normal brain sMRI and PET images, as well as brain sMRI and PET images with Alzheimer's disease category prediction results, thereby greatly facilitating staff and those being tested.

Claims

1. An image fusion-based method for predicting Alzheimer's disease categories, comprising the following steps: S1. Obtain existing and corresponding brain sMRI and PET images; S2. Preprocess the images obtained in step S1 to obtain the training dataset; S3. Based on variational autoencoders, Dirichlet distribution, and Dempster-Shafer theory, a preliminary category prediction model for Alzheimer's disease is constructed. The constructed preliminary category prediction model for Alzheimer's disease includes a first encoder module, a first channel attention module, a first variational sampling module, a first decoder module, a second encoder module, a second channel attention module, a second variational sampling module, a second decoder module, a category probability prediction module, a probability processing module, and a category prediction module. S4. Using the training dataset obtained in step S2, train the preliminary Alzheimer's disease category prediction model constructed in step S3 to obtain the Alzheimer's disease category prediction model. The training specifically includes the following steps: The following formula is used as the loss function for sMRI images. : In the formula This is the feature vector output by the first fully connected layer of the first variational sampling module; This is the feature vector output by the second fully connected layer of the first variational sampling module; The original input sMRI image, sMRI images generated for the first decoder module; It is an L2 norm; The following formula is used as the loss function for PET images. : In the formula This is the feature vector output by the first fully connected layer of the second variational sampling module; This is the feature vector output by the second fully connected layer of the second variational sampling module; The original input PET image, PET images generated for the second decoder module; Constructing a feature extraction loss function for ; The following formula is used as the fusion classification loss function. : In the formula Predict loss for each category; The parameters of the Dirichlet distribution are and =e i +1, Composed of several categories composition; This is a weighting coefficient, and its value is the ratio of the current training iteration to the total training iterations. For KL divergence loss; Let be the true label of the k-th category in the i-th image; It is a double gamma function; Dirichlet strength; It is a gamma function; For the updated Dirichlet distribution parameters; The following formula is used as the classification loss function for sMRI images. : In the formula Loss for class prediction of sMRI images; The Dirichlet distribution parameters of the sMRI image; KL divergence loss for sMRI images; Let be the true label of the k-th class in the i-th sMRI image; The Dirichlet distribution intensity of the sMRI image; Dirichlet distribution parameters for the updated sMRI images; The following formula is used as the classification loss function for PET images. : In the formula Loss for class prediction of PET images; The Dirichlet distribution parameters of the PET image; KL divergence loss for PET images; The true label for the k-th category in the i-th PET image; The Dirichlet distribution intensity of the PET image; The Dirichlet distribution parameters for the updated PET images; Construct the overall classification loss function for ; Finally, construct the total loss function. for ; During training, the model parameters are trained using an adaptive moment estimator based on the constructed total loss function; S5. Input the actual brain sMRI and PET images into the Alzheimer's disease category prediction model obtained in step S4 to complete the category prediction of Alzheimer's disease based on image fusion.

2. The Alzheimer's disease category prediction method based on image fusion according to claim 1, characterized in that... Step S3, which involves constructing a preliminary Alzheimer's disease category prediction model based on variational autoencoders, Dirichlet distribution, and Dempster-Shafer theory, specifically includes the following steps: The first encoder module, the first channel attention module, the first variational sampling module, and the first decoder module are connected in series; the second encoder module, the second channel attention module, the second variational sampling module, and the second decoder module are connected in series; the category probability prediction module, the probability processing module, and the category prediction module are connected in series; the output terminals of the first variational sampling module and the second variational sampling module are both connected to the input terminal of the category probability prediction module. The first encoder module is used to perform feature encoding on the input sMRI image; The first channel attention module is used to enhance the features output by the first encoder module. The first variational sampling module is used to perform variational sampling on the features output by the first channel attention module; the first decoder module is used to perform feature decoding on the features output by the first variational sampling module, generate a generated sMRI image corresponding to the input sMRI image, and use it for training the preliminary category prediction model for Alzheimer's disease. The second encoder module is used to perform feature encoding on the input PET image; The second channel attention module is used to enhance the features output by the second encoder module. The second variational sampling module is used to perform variational sampling on the features output by the second channel attention module; the second decoder module is used to perform feature decoding on the features output by the second variational sampling module, generate a generated PET image corresponding to the input PET image, and use it for training the preliminary category prediction model for Alzheimer's disease. The category probability prediction module is used to map the output of the first variational sampling module and the output of the second variational sampling module to obtain the category prediction probability of sMRI images for Alzheimer's disease and the category prediction probability of PET images for Alzheimer's disease. The probability processing module is used to predict the probability of Alzheimer's disease category from sMRI images and PET images, based on the Dirichlet distribution, and to perform confidence and uncertainty calculations. The category prediction module is used to fuse the confidence calculation results and uncertainty calculation results of the two categories of predicted probabilities based on Dempster-Shafer theory to obtain the final category prediction result of Alzheimer's disease.

3. The Alzheimer's disease category prediction method based on image fusion according to claim 2, characterized in that... The first encoder module and the second encoder module have the same structure, both including a first encoder layer, a second encoder layer, a third encoder layer, a fourth encoder layer and a fifth encoder layer; the first encoder layer, the second encoder layer, the third encoder layer, the fourth encoder layer and the fifth encoder layer are connected in series. The first encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has one input channel, four output channels, five convolutional kernels, a kernel span of 1, and adds 2 padding units to all six sides of the 3D input during the convolution operation. The max pooling layer has two pooling kernels with a kernel span of 2. The batch normalization layer has four features. The second encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 4 input channels, 8 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 8 features. The third encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 8 input channels, 16 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 16 features. The fourth encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 16 input channels, 32 output channels, 5 kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 32 features. The fifth encoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 32 input channels, 64 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 64 features. The first decoder module and the second decoder module have the same structure, both including a recovery layer, a first decoder layer, a second decoder layer, a third decoder layer, a fourth decoder layer, and a fifth decoder layer; The recovery layer, the first decoder layer, the second decoder layer, the third decoder layer, the fourth decoder layer, and the fifth decoder layer are connected in series. The recovery layer consists of one fully connected layer and one ReLU (Rectified Activated Unit). The recovery layer is used to restore the size of the input features to the same size as the enhanced features output by the corresponding channel attention module, through a fully connected layer and a linear rectified activation unit (ReLU). The first decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 64 input channels, 32 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 32 features. The second decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one Rectified Activation Unit (ReLU). The 3D convolutional layer has 32 input channels, 16 output channels, 5 kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 16 features. The third decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 16 input channels, 8 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 8 features. The fourth decoder layer consists of one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 8 input channels, 4 output channels, 5 convolutional kernels, a kernel span of 1, and 2 padding units added to all six sides of the 3D input during the convolution operation. The max pooling layer has 2 pooling kernels with a kernel span of 2. The batch normalization layer has 4 features. The fifth decoder layer includes one 3D convolutional layer, one max pooling layer, one batch normalization layer, and one ReLU linear rectified activation unit. The 3D convolutional layer has 4 input channels, 1 output channel, 5 convolutional kernels, and a kernel span of 1. The padding added to all six sides of the 3D input during the convolution operation is 2. The max pooling layer has 2 pooling kernels and a kernel span of 2. The batch normalization layer has 1 feature.

4. The Alzheimer's disease category prediction method based on image fusion according to claim 3, characterized in that... The first-channel attention module and the second-channel attention module have the same structure, both including an adaptive average pooling layer and two fully connected layers; The adaptive average pooling layer is used to compress the spatial dimension of the features output by the corresponding encoder module to 1, so as to reduce the calculation of spatial dimension in subsequent processing. The first fully connected layer is used to reduce the number of feature channels of the features output by the adaptive average pooling layer from 64 to 8; The second fully connected layer is used to restore the number of feature channels of the features output by the first fully connected layer to 64, thereby obtaining the channel weight factor; Finally, the channel weight factor is multiplied by the feature output by the corresponding encoder module to obtain the enhanced feature.

5. The Alzheimer's disease category prediction method based on image fusion according to claim 4, characterized in that... The first variational sampling module and the second variational sampling module have the same structure, including a first flat layer, a first fully connected layer, a second flattened layer, a second fully connected layer, and a sampling layer; The first flattened layer and the first fully connected layer are connected; the second flattened layer and the second fully connected layer are connected; the outputs of the first fully connected layer and the second fully connected layer are both connected to the input of the sampling layer. The enhanced features output by the corresponding channel attention module are flattened through the first flattening layer to reduce the dimensionality to 1-dimensional feature vectors. The length of the 1-dimensional feature vectors is then mapped to 256 using the first fully connected layer. To avoid overfitting, a random dropout probability is set for the output of neurons in the first fully connected layer. The output value of the dropped neurons is set to 0. The enhanced features output by the corresponding channel attention module are flattened through the second flattening layer to obtain a 1-dimensional feature vector. The length of the obtained 1-dimensional feature vector is then mapped to 256 using the second fully connected layer. To avoid overfitting, a random dropout probability is set for the output of the neurons in the second fully connected layer. The output value of the dropped neurons is directly set to 0. The 256-length 1D feature vector output by the first fully connected layer is used as the mean parameter of the Gaussian distribution, and the 256-length 1D feature vector output by the second fully connected layer is used as the variance parameter of the Gaussian distribution; the sampling layer samples the Gaussian distribution to obtain variational sampling features.

6. The Alzheimer's disease category prediction method based on image fusion according to claim 5, characterized in that... The category probability prediction module includes a fully connected layer, a dropout layer, and an activation function layer; For the features output by the first variational sampling module, a fully connected layer is used to map the size of the features output by the first variational sampling module from 256 to 2, and a dropout layer is used to prevent overfitting with a dropout rate of 0.

2. Finally, the activation function layer uses the Softplus activation function to process the features output by the fully connected layer to obtain the predicted probability of the sMRI image for Alzheimer's disease category. For the features output by the second variational sampling module, a fully connected layer is used to map the size of the features output by the first variational sampling module from 256 to 2, and a dropout layer is used to prevent overfitting with a dropout rate of 0.

2. Finally, the activation function layer uses the Softplus activation function to process the features output by the fully connected layer to obtain the predicted probability of the PET image for Alzheimer's disease category.

7. The Alzheimer's disease category prediction method based on image fusion according to claim 6, characterized in that... Step S2, which involves preprocessing the image obtained in step S1, specifically includes the following steps: For the acquired sMRI images, the sMRI images are registered to a standard brain template space to unify the coordinate space; and the registered images are segmented to remove the influence of the skull region. For the acquired PET images, an interpolation algorithm is used to adjust the PET images to the same resolution as the registered sMRI images; Finally, random augmentation and voxel normalization were performed on the obtained processed sMRI and PET images to complete the preprocessing process. Step S5 involves inputting the actual brain sMRI and PET images into the Alzheimer's disease category prediction model obtained in step S4 to complete the image fusion-based Alzheimer's disease category prediction. This specifically includes the following steps: Preprocessing of actual brain sMRI and PET images: For actual sMRI images, the sMRI images are registered to a standard brain template space to unify the coordinate space; and the registered images are segmented to remove the influence of the skull region. For the actual PET images, an interpolation algorithm is used to adjust the PET images to the same resolution as the registered sMRI images; Delete the first and second decoders of the Alzheimer's disease category prediction model obtained in step S4 to obtain the Alzheimer's disease category prediction trimming model. The preprocessed brain sMRI and PET images are input into the Alzheimer's disease category prediction pruning model to obtain the final Alzheimer's disease category prediction results.

8. An imaging method comprising the image fusion-based Alzheimer's disease category prediction method according to any one of claims 1 to 7, characterized in that... Includes the following steps: A. Obtain the brain sMRI and PET images to be classified; B. Using the image fusion-based Alzheimer's disease category prediction method as described in any one of claims 1 to 7, perform Alzheimer's disease category prediction on the brain sMRI and PET images to be classified obtained in step A; C. The category prediction results obtained in step B are used for secondary imaging of the brain sMRI and PET images to be classified obtained in step A to obtain brain sMRI and PET images with category prediction results for Alzheimer's disease.

Images