Alzheimer's disease risk prediction method based on deep learning

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an Alzheimer's disease risk prediction method based on deep learning. According to the method, through a multi-view feature extraction stage, capturing of spatial information and extraction of multi-view slice-level features under different view angles are realized by using a method of combining 3D and 2D convolutional layers; through a multi-view double-alignment stage, the consistency among different views is enhanced, and complementary information is effectively reserved; through a private deviation filtering stage, identifying and removing specific view deviations which are useless for the classification task; and finally, through a multi-view fusion classification stage, basic features of multiple views are integrated, information redundancy is avoided, and the efficiency and accuracy of AD early risk prediction are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and specifically relates to a method for predicting Alzheimer's disease risk based on deep learning. Background Art

[0002] As a powerful machine learning technology, deep learning has made significant progress in image recognition and classification tasks, and has shown great potential in the field of early risk prediction of Alzheimer's disease (AD). AD is a progressive neurodegenerative disease, and early accurate prediction is crucial to delay the progression of the disease. Based on 3D structural magnetic resonance imaging (sMRI) data, deep learning methods can automatically learn complex feature representations, thereby improving the accuracy and efficiency of early risk prediction of AD.

[0003] Traditional machine learning methods usually rely on manually designed feature extraction and classification algorithms in early AD risk prediction. These methods require a lot of manual participation and expertise, and due to the complexity and diversity of medical images, traditional methods face great challenges in processing large-scale data and complex cases. In addition, manually designed features may not fully capture subtle changes related to AD, limiting the generalization ability of the model and the accuracy of prediction.

[0004] Among single-view methods, 2D-based methods can simplify computational complexity by selecting specific slices from 3D sMRI images as model input, but ignore the spatial association of 3D, which may lead to the loss of important anatomical information. Although 3D-based methods can capture more comprehensive spatial information, they are computationally expensive and have the risk of overfitting. In addition, these methods ignore the inherent multi-view features of 3D sMRI, which may provide complementary information and improve the robustness and prediction performance of the model.

[0005] Among the multi-view methods, although 2D-based multi-view methods attempt to exploit multiple perspectives, they often fail to fully explore spatial information. In order to exploit spatial information at a lower computational cost, 2D-3D hybrid multi-view methods combine 3D representations with 2D multi-view representations, but these methods usually adopt simple splicing or self-attention mechanisms to achieve multi-view fusion. In addition, these methods only use basic cross-entropy loss to guide the end-to-end training process, which may not effectively mine the deep complementary and consistent relationships between different views, and may even lead to feature redundancy, thereby reducing the model prediction performance. Summary of the invention

[0006] In view of the above shortcomings, the present invention provides a method for predicting the risk of Alzheimer's disease based on deep learning. The present invention uses a combination of 3D and 2D convolutional layers in the multi-view feature extraction stage to capture spatial information from different perspectives and extract multi-view slice-level features; through the multi-view double alignment stage, the consistency between different views is enhanced and complementary information is effectively retained; through the private bias filtering stage, specific view biases that are not beneficial to the classification task are identified and removed; finally, through the multi-view fusion classification stage, the basic features of multiple views are integrated to avoid information redundancy, which is conducive to improving the efficiency and accuracy of early AD risk prediction.

[0007] The technical solution adopted by the present invention is:

[0008] A method for predicting the risk of Alzheimer's disease based on deep learning, characterized by comprising the following steps:

[0009] S1. Obtain structural MRI images from the Alzheimer's Disease Neuroimaging Project dataset and obtain the required training dataset after preprocessing.

[0010] S2, building a risk prediction model, including a multi-view feature extraction module, a multi-view dual alignment module, a private bias filtering module, a feature fusion and prediction network;

[0011] The input of the multi-view feature extraction module is training data, which is processed to obtain slice-level representations of multiple views. The specific processing method is to input the training data into the multi-view feature extraction module in batches, and each batch of data is defined as X∈R N×1×H×W×D , where N is the batch size, H, W, and D represent the height, width, and depth of each structural MRI image, respectively. Let X pass through the 3D convolution module to obtain a 3D representation I 3D , then I 3D Through three parallel 2D sub-networks and applying a global average pooling operation in the spatial dimension, the slice-level representation of multiple views is obtained, which is defined as Where S m represents the number of slices in the mth view, d represents the feature dimension, and M represents the number of views;

[0012] The input of the multi-view dual alignment module is I m , the attention weights of each slice are calculated through M multi-layer perceptrons, and these weights are used to weight and aggregate the slice-level features to generate multi-view confusion features The features are normalized by semantic alignment constraints and attention alignment constraints at the same time; where:

[0013]

[0014] A m =σ(fMLPm (I m ))

[0015] Where σ(·) is the sigmoid activation function, represents the slice-level attention of the m-th view, ⊙ represents the Hadamard product with broadcast mechanism; represents a multi-layer perceptron;

[0016] The input of the private deviation filter module is H m , first extract the multi-view private deviation information through M independent two-layer MLPs to obtain In this process, the contrast loss is used to constrain the deviation features, and then according to P m The direction of H m Perform orthogonal decomposition to obtain the essential features that remove private bias information

[0017]

[0018] The feature fusion and prediction network is first based on the globally shared classification layer f SFC (·) Calculate E m and H m The difference in logical values ​​between them is constrained by cross entropy loss in this process, and finally a classification layer is used to achieve the final risk prediction, which is also constrained by cross entropy loss.

[0019] S3. The constructed risk prediction model is trained using the obtained training data set. The training method is to use adaptive moment estimation and batch normalization to train the model. The loss function used is expressed as:

[0020]

[0021] in,

[0022]

[0023]

[0024]

[0025]

[0026]

[0027] in, is the cross entropy loss of the classification layer, Y n,c ∈{0,1} indicates whether the nth sample belongs to class c. If it does, it is 1, otherwise it is 0. FC(·) indicates a fully connected layer; is the semantic alignment constraint, Z m =f SFC (H m ) indicates H m The corresponding logical value, f SFC (·) indicates the globally shared classification layer, C is the total number of categories, s(·) is the softmax function, To take care of the alignment constraints, represents the probability that all slices in the mth view belong to the true class of the sample, Represents the index corresponding to the true class label of the nth sample, Yes m The results of the remodeling; is the contrast loss, d(·,·) represents the cosine similarity; To calculate the essential feature E m and H m The cross entropy loss is used for the difference in logical values ​​between ; Indicates the essential characteristic E m ; α and β are two hyperparameters used to balance the contrastive loss and the cross entropy loss used as a semantic guarantee.

[0028] The beneficial effects of the present invention are as follows: compared with the existing technology, by combining 3D and 2D convolution techniques, the loss of underlying spatial information is effectively avoided when extracting multi-view representations; a multi-view dual alignment module is designed and utilized, which significantly enhances the consistency between features of different views, retains and normalizes the complementary information unique to each view, thereby enhancing the representation capability of multi-view features; based on the orthogonal decomposition of cross-view contrast loss and semantic regularization, private bias information within the view is effectively identified and filtered, further promoting the fusion of multi-view features and the accuracy of the final prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0030] Figure 2 Schematic diagram of the multi-view dual alignment submodule in the present invention.

[0031] Figure 3 Schematic diagram of the private deviation filtering submodule in the present invention. DETAILED DESCRIPTION

[0032] The present invention is described in detail below with reference to the accompanying drawings and embodiments:

[0033] Example

[0034] Figure 1is a flow chart of a method for predicting early Alzheimer's disease risk based on multi-view alignment, filtering and fusion of the present invention, such as Figure 1 As shown, this example includes the following steps:

[0035] Step 1: Structural MRI image preprocessing.

[0036] Given the annotated sMRIs, we first convert the DCM format files into NIFTI format using the SimpleITK library.

[0037] Next, the fslreorient2std tool was applied to normalize the image orientation and ensure consistency with a standard template.

[0038] Then, the sMRIs were cropped using the robustfov tool to remove non-target regions of the neck and lower head.

[0039] Furthermore, the FLIRT tool was used to align all sMRI images with the Colin27 template to ensure that all images had the same resolution (i.e., 1 × 1 × 1 mm 3 ) and dimensions (i.e. 191×217×191).

[0040] Finally, the BET tool was used to perform skull and dura peeling to further improve image quality.

[0041] To reduce the influence of background, we cropped all experimental sMRIs to 160 × 192 × 160. In addition, we augmented the training images by translation and mirroring operations to increase the diversity of training data. Specifically, the translation operation randomly selects one of the six directions (up, down, front, back, left, right) and moves a voxel with equal probability, and the mirroring operation symmetrically transforms the left and right brains of sMRIs with a probability of 0.5.

[0042] This example uses sMRIs for a total of 1828 samples, including 848 normal (NC) samples, 396 AD samples and 584 mild cognitive impairment (MCI) samples. MCI samples can be further divided into 323 stable mild cognitive impairment (sMCI) samples and 261 progressive mild cognitive impairment (pMCI) samples.

[0043] The experiment is divided into three types of tasks, namely, two-classification task (AD-NC), three-classification task (AD-MCI-NC) and four-classification task (AD-pMCI-sMCI-NC). During the experiment, five-fold cross validation was used to divide the training set (accounting for 80%) and the test set (accounting for 20%), and the average results were reported.

[0044] Step 2: Extract multi-view slice-level features.

[0045] Combining 3D and 2D convolutional layers extracts multi-view slice-level features while capturing the underlying spatial information. In this embodiment, the feature extraction network consists of six convolutional modules with the number of output channels being 16, 32, 64, 64, 128, and 128, respectively. The first three modules use 3D convolutions, while the last three modules consist of three parallel, structurally identical 2D convolutional blocks. All convolution kernels have a size of 3, a stride of 1, and a padding of 1, which does not change the size of the feature map. There is a maximum pooling layer between every two convolutional modules with a kernel size of 2, a stride of 2, and no padding to achieve 2x downsampling.

[0046] Given a batch of data X∈R N×1×H×W×D Where N is the batch size, H, W, and D represent the height, width, and depth of each sMRI image, respectively. In this embodiment, N = 4, H = D = 160, and W = 192. First, the three-dimensional representation is obtained by the first three three-dimensional convolution modules. Then, the batch dimension is merged with any spatial dimension to obtain a multi-view input We further obtain slice-level representations of multiple views by utilizing three parallel 2D sub-networks and applying GAP operations on the spatial dimension. In this embodiment Dimension d=128.

[0047] Step 3: Aggregate slice features to obtain mixed features and impose alignment constraints.

[0048] Accept multi-view slice-level features as input and output aggregated multi-view confusion features, and impose consistency constraints from two perspectives: attention activation and feature semantics. Figure 2 shown.

[0049] Given multi-view image-level features First, it is input into the attention alignment module, and the attention weights of each slice are calculated through M multi-layer perceptrons (MLPs), and these weights are used to weight and aggregate the slice-level features to generate multi-view confusion features. Right now:

[0050]

[0051]

[0052] Based on multi-view confusion features Semantic alignment constraints are designed to ensure semantic consistency between features from different perspectives, and the feature representation of complementary information is standardized by unifying the decision space. Specifically,

[0053]

[0054] Where Z m =f SFC (H m ) indicates H m The corresponding logical value. SFC (·) represents the globally shared classification layer. C∈{2,3,4} is the total number of categories. s(·) is the softmax function, Y n,c ∈{0,1} indicates whether the nth sample belongs to class c, if it belongs to that class, it is 1, otherwise it is 0.

[0055] Based on semantic alignment, attention alignment constraints are further implemented to promote the correct activation of multi-view regions with similar discriminative semantics in a consistent decision space while avoiding the wrong activation of trivial regions. The specific calculation method is

[0056]

[0057] in Represents the probability that all slices in the mth view belong to the true class of the sample. Represents the index corresponding to the true class label of the nth sample. Yes m The result of remodeling.

[0058] Step 4: Identify and isolate view-private deviations.

[0059] Identify and separate view-specific bias information from mixed features, thereby enhancing the discriminability of features and facilitating the fusion of multi-view features, such as Figure 3 shown.

[0060] First, based on the mixed features Utilize m independent two-layer MLPs to extract multi-view private bias information

[0061] Afterwards, the cross-view contrast loss is calculated to constrain the bias features Ensure that the deviations within the same view remain consistent and that the deviations between different views remain orthogonal, i.e.

[0062]

[0063] where d(·,·) represents the cosine similarity.

[0064] Next, according to The direction of Perform orthogonal decomposition to obtain the essential features without private bias information:

[0065]

[0066] Finally, based on the globally shared classification layer f SFC (·) Calculate essential characteristics and mixed features The difference in logical values ​​between them is constrained based on the cross entropy loss to ensure the semantic correctness of the essential features. The calculation formula is as follows:

[0067]

[0068] in Indicates the essential characteristic E m The logical value of .

[0069] Step 5: Construction of risk prediction model.

[0070] Based on the mean aggregation of essential features, a classification layer is introduced to achieve the final AD early risk prediction:

[0071]

[0072] The final objective function is calculated and T rounds of training are performed. In this embodiment, T=100. The objective function is calculated as:

[0073]

[0074] Where α and β are two hyperparameters used to balance the horizontal contrast loss and semantic constraints. In this implementation example, the optimal setting for a binary classification task is a=b=0.4. The optimal setting for a ternary classification task is a=1.0, b=0.6. The optimal setting for a quaternary classification task is a=0.8, b=0.6.

Claims

1. A method for predicting Alzheimer's disease risk based on deep learning, characterized in that: The following steps are involved: S1. Obtain structural MRI images from the Alzheimer's Disease Neuroimaging Project dataset and obtain the required training dataset after preprocessing. S2, building a risk prediction model, including a multi-view feature extraction module, a multi-view dual alignment module, a private bias filtering module, a feature fusion and prediction network; The input of the multi-view feature extraction module is training data, which is processed to obtain slice-level representations of multiple views. The specific processing method is to input the training data into the multi-view feature extraction module in batches, and each batch of data is defined as X∈R N ×1×H×W×D , where N is the batch size, H, W, and D represent the height, width, and depth of each structural MRI image, respectively. Let X pass through the 3D convolution module to obtain a 3D representation I 3D , then I 3D Through three parallel 2D sub-networks and applying a global average pooling operation in the spatial dimension, the slice-level representation of multiple views is obtained, which is defined as Where S m represents the number of slices in the mth view, d represents the feature dimension, and M represents the number of views; The input of the multi-view dual alignment module is I m , the attention weights of each slice are calculated through M multi-layer perceptrons, and these weights are used to weight and aggregate the slice-level features to generate multi-view confusion features The features are normalized by semantic alignment constraints and attention alignment constraints at the same time; where: Where σ(·) is the sigmoid activation function, represents the slice-level attention of the m-th view, ⊙ represents the Hadamard product with broadcast mechanism; represents a multi-layer perceptron; The input of the private deviation filter module is H m , first extract the multi-view private deviation information through M independent two-layer MLPs to obtain In this process, the contrast loss is used to constrain the deviation features, and then according to P m The direction of H m Perform orthogonal decomposition to obtain the essential features that remove private bias information The feature fusion and prediction network is first based on the globally shared classification layer f SFC (·) Calculate E m and H m The difference in logical values ​​between them is constrained by cross entropy loss in this process, and finally a classification layer is used to achieve the final risk prediction, which is also constrained by cross entropy loss. S3. The constructed risk prediction model is trained using the obtained training data set. The training method is to use adaptive moment estimation and batch normalization to train the model. The loss function used is expressed as: in, in, is the cross entropy loss of the classification layer, Y n，c ∈{0, 1} indicates whether the nth sample belongs to class c. If it does, it is 1, otherwise it is 0. FC (·) indicates a fully connected layer; is the semantic alignment constraint, Z m =f SFC (H m ) indicates H m The corresponding logical value, f SFC (·) indicates the globally shared classification layer, C is the total number of categories, s(·) is the softmax function, To take care of the alignment constraints, represents the probability that all slices in the mth view belong to the true class of the sample, Represents the index corresponding to the true class label of the nth sample, Yes m The results of the remodeling; is the contrast loss, d(·,·) represents the cosine similarity; To calculate the essential feature E m and H m The cross entropy loss used for the difference in logical values ​​between Indicates the essential characteristic E m ; α and β are two hyperparameters used to balance the contrastive loss and the cross entropy loss used as a semantic guarantee.

Images