Alzheimer's disease analysis method based on attention multi-task interpretable graph neural network

Abstract

The invention provides a method for analyzing Alzheimer's disease based on a multi-task graph convolutional network (GCN) of an attention mechanism, and aims to improve the accuracy of disease diagnosis and clearly identify a brain region related to the Alzheimer's disease. Most existing Alzheimer disease diagnosis models only depend on medical image data and ignore clinical score information related to diseases, so that brain areas affected by the diseases cannot be effectively recognized; optimization is achieved through the following steps that firstly, segmentation based on tissue types is conducted on brain areas, and initial weights are randomly distributed to each area; thirdly, a multi-task attention unit is introduced, and cross interaction between medical images and clinical scores is promoted by capturing shared feature information between brain areas and between tasks; finally, a specific task layer is designed for each task, and accurate prediction of the Alzheimer's disease and related brain areas is carried out. Experimental results show that the accuracy of the method is superior to that of an existing method on an Alzheimer's disease neuroimaging proposal (ADNI) data set, and the brain area related to the disease can be clearly recognized.

Description

Technical Field

[0001] The present invention belongs to the technical field of Alzheimer's disease analysis based on deep learning, and more specifically, relates to an Alzheimer's disease analysis method based on an attention-based multi-task interpretable graph neural network. Background Art

[0002] Alzheimer's disease (AD) is a neurodegenerative disease that may significantly reduce the quality of life of the elderly. Early intervention of the disease is an effective way to prevent its deterioration. Neuroimaging techniques, such as magnetic resonance imaging (MRI), are widely used in medical diagnosis because they can provide precise information about the condition of the brain. At the same time, the Alzheimer's Disease Cognitive Assessment Scale-Cognitive Subscale (ADAS-Cog) and the Mini-Mental State Examination (MMSE) are common cognitive assessment methods for diagnosing AD. In recent years, machine learning methods have been widely used in AD analysis, but most studies have focused on improving the AD classification performance based on MRI. In actual Alzheimer's disease (AD), the classification of the brain is not easy to detect. )In the diagnosis process, it is not enough to consider only medical images, because clinical scores related to AD will also have a certain impact on the disease, such as ADAS-Cog and MMSE; since ADAS-Cog and MMSE scores are usually difficult to obtain, an MRI-based method that can consider the association between AD tasks is proposed to solve this problem; previous studies have shown that multi-task models are able to exploit the association between AD diagnosis and clinical score prediction tasks; however, most of such methods are based on a multi-task framework, use hard parameter sharing or soft parameter sharing to capture shared features between tasks, and use graph convolutional networks (GCNs) for feature extraction in a single task.

[0003] However, this approach has two problems; first, hard parameter sharing first uses the underlying common layers to share parameters, and then outputs the prediction results of different tasks through task-specific layers; the limitation of hard sharing is that the proportion of information sharing between tasks cannot be adjusted; when the information of one task is not completely useful or even partially unfavorable to another task, the prediction results may be suboptimal.

[0004] To solve this problem, more and more existing research has turned to the use of soft parameter sharing; this method allows each task to have its own specific parameters and controls the proportion of information sharing between different tasks by regularizing the parameters between tasks; however, existing soft sharing methods require manual adjustment of the information sharing ratio between tasks, which is time-consuming and labor-intensive.

[0005] Secondly, using only graph convolutional networks (GCNs) cannot provide the corresponding important brain areas for each task while outputting the results, which may lead to relatively low credibility of the model. To solve this problem, existing solutions rely on a posteriori explanation methods, that is, training an explanation model after the classification model is trained to identify key features in the diagnosis process. However, the limitation of this type of a posteriori explanation method is that the explanation performance is completely dependent on the classification model, and it is impossible to use interpretable results to optimize model training. Therefore, it is necessary to discover the most important features during the training process. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an Alzheimer's disease analysis method based on an attention-based multi-task interpretable graph neural network; the present invention proposes a multi-task interpretable graph convolutional network (AMTI-GCN) based on an attention mechanism, which uses the attention mechanism to automatically learn shared information between tasks and identify the most important brain areas for each task respectively; specifically, the present invention first embeds the L2,1 norm into the graph convolution to achieve the selection of important features of the task during the learning process; then uses the attention mechanism to automatically find the optimal ratio of feature information sharing between tasks; finally, the present invention designs task-specific layers for each task to obtain their respective outputs.

[0007] The present invention has the following beneficial effects:

[0008] 1) This paper proposes a multi-task attention mechanism to learn the shared correlation information between multiple tasks;

[0009] 2) This paper proposes a new GCN method to identify important brain regions through joint training on specific tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flowchart of a specific implementation of the Alzheimer's disease analysis method based on the attention-based multi-task interpretable graph neural network of the present invention;

[0011] Figure 2 is a flow chart of the interpretability module in the present invention;

[0012] Figure 3 It is a flow chart of the feature sharing module in the present invention;

[0013] Figure 4 It is a task specific module flow chart in the present invention. DETAILED DESCRIPTION

[0014] The specific implementation modes of the present invention are described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be ignored here.

[0015] Example

[0016] Figure 1 is a flowchart of a specific implementation of the Alzheimer's disease analysis method based on the attention-based multi-task interpretable graph neural network of the present invention; Figure 1 As shown, the specific steps of the Alzheimer's disease analysis method based on the attention-based multi-task interpretable graph neural network of the present invention include:

[0017] S101: Build training and test datasets:

[0018] According to actual needs, MRI images of M patients and ADAS-Cog and MMSE score samples are collected to form the training data set X.

[0019] S102: Interpretability module; segment brain regions according to tissue type and randomly assign initial weights to each region:

[0020] The present invention uses a specific feature extraction method to segment the brain area of ​​MRI images; since each feature contributes differently to specific task training, treating all features equally often leads to suboptimal prediction results; therefore, the present invention assigns weights to each feature and uses sparsity to reduce the impact of redundant features, thereby obtaining better prediction performance and interpretability.

[0021] S103: Feature sharing module; feature extraction using GCN and attention mechanism:

[0022] The present invention uses GCN to transfer and fuse information between samples, and uses an inter-task feature fusion method based on the attention mechanism to fuse features between different tasks; this effectively captures the shared feature information between brain regions and between tasks, and realizes the cross-interaction of medical images and clinical scores.

[0023] S104: Task-specific module; prediction for specific tasks (classification, regression)

[0024] The present invention designs a task-specific layer for each task to achieve the prediction of Alzheimer's disease and its corresponding brain regions.

[0025] Figure 2 is a flow chart of the interpretability module in the present invention; Figure 2 As shown, the specific steps performed by the interpretability module in the present invention include:

[0026] S201: Initialize the weight matrix:

[0027] In order to assign different weights to different features according to the importance of the task, the present invention initializes a weight matrix; specifically, considering the matrix Where k∈{1,2,...,m} is the weight matrix of the kth task, which is a trainable matrix and randomly initialized.

[0028] S202: Sparse feature weights:

[0029] In order to reduce the impact of damaged features, when the i-th feature is not important, we hope that the corresponding weight w i,k becomes smaller or even zero; therefore, the L0 norm is the most appropriate constraint condition, and its mathematical expression is ‖W k ‖0; however, L0 norm regularization is non-convex, so the gradient descent method cannot be used to find the optimal solution; therefore, the present invention uses L 2,1 norm, which is the minimum convex hull of the L0 norm, used for row sparseness, and the mathematical expression is ‖W k ‖ 2,1 .

[0030] S203: assign corresponding weights:

[0031] The present invention treats the sparse weight matrix W k Perform the Hadamard product operation with the feature matrix X, and the expression is as follows: Where ⊙ represents the Hadamard product, that is, W k Multiply X element-wise, is the original feature matrix, which is the same for all tasks; then the weighted feature matrix Input feature sharing module to learn other task information that is beneficial to its own training.

[0032] Figure 3 is a flow chart of the feature sharing module in the present invention; Figure 3 As shown, the specific steps performed by the feature sharing module in the present invention include:

[0033] S301: Graph convolution operation:

[0034] In order to make the subsequent attention mechanism better share feature information, the present invention will And the adjacency matrix A is input into the graph convolution layer to learn the characteristics and structural information of the nodes; the specific formula is as follows:

[0035]

[0036] Where l∈{0,...,L-1}; is the output after graph convolution; is a diagonal matrix, and is the trainable parameter matrix of the lth layer in the kth task, σ(·) represents the activation function; after L layers of graph convolution, we can get the embedded feature matrix

[0037] S302: Attention-based feature sharing between tasks.

[0038] The input of the attention unit is the embedded feature after L layers of graph convolution And its output is a shared representation This representation is a linear combination of the m task embedding feature matrices; the attention-based feature sharing process can be described by the following formula:

[0039]

[0040] in, is the jth feature of the i-th sample in the k-th task after t times of feature sharing through the attention unit, t∈{1,2,...,T}; is a trainable parameter, which indicates the proportion of information obtained by the i-th task from the j-th task, and Satisfies the following formula: The larger the value of , the higher the degree of sharing between task i and task j. It means that task i does not absorb information from task j.

[0041] The present invention places the attention unit after the graph convolution network; specifically, after each feature extraction through graph convolution, the embedded feature matrix is ​​input into the attention unit to perform information interaction; after layer graph convolution and T times of sharing, the obtained features are input into the task-specific module.

[0042] Figure 4 is a flowchart of a task-specific module in the present invention; Figure 4 As shown, the specific steps performed by the task-specific module in the present invention include:

[0043] S401: Classification task prediction:

[0044] After the feature sharing module, the feature matrix of m tasks can be obtained If the kth task is a classification task, the present invention embeds the feature matrix Input to the softmax layer to obtain the probability that the i-th sample belongs to the j-th class The specific expression is as follows:

[0045]

[0046] In order to obtain the probability of each sample corresponding to a category, the present invention uses the highest probability index of the category to which each sample belongs as the prediction result. So as to get the final predicted labels of all samples

[0047] S402: Regression task prediction:

[0048] If the task is a regression task, after the feature sharing module, the feature matrix of the task can be obtained This matrix is ​​then fed into a multi-layer perceptron (MLP) to output the predicted values Its mathematical expression is as follows:

[0049] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A multi-task interpretable graph convolution method based on attention mechanism for Alzheimer's disease, characterized by: The following steps are involved: S101: Build training and test datasets: According to actual needs, collect MRI images of M patients and ADAS-Cog and MMSE score samples to form a training data set X; S102: Interpretability module: Segment brain regions according to tissue type and randomly assign initial weights to each region; S102 uses a specific feature extraction method to segment the brain regions of MRI images. Since each feature contributes differently to specific task training, treating all features equally often leads to suboptimal prediction results. S102 assigns weights to each feature and uses sparsity to reduce the impact of redundant features, thereby achieving better prediction performance and interpretability. S103: Feature sharing module, using GCN and attention mechanism for feature extraction: S103 uses GCN to transfer and fuse information between samples, and uses an inter-task feature fusion method based on the attention mechanism to fuse the features between different tasks. This effectively captures the shared feature information between brain regions and tasks, and realizes the cross-interaction between medical images and clinical scores. S104: Task-specific module, prediction of specific tasks (classification, regression): S104 designed a task-specific network layer to achieve the prediction of Alzheimer's disease and its corresponding brain regions.

2. According to the method for multi-task interpretable graph convolution based on attention mechanism for Alzheimer's disease according to claim 1, the specific steps performed by the interpretability module in step S102 include: S201: Initialize the weight matrix: In order to assign different weights to different features according to the importance of the task, S201 initializes a weight matrix. Consider the matrix Where k∈{1,2,...,m} is the weight matrix of the kth task, which is a trainable matrix and randomly initialized; S202: Sparse feature weights: In order to reduce the impact of damaged features, when the i-th feature is not important, we hope that the corresponding weight w i,k Become smaller or even zero; S202 uses L 2,1 norm, which is the minimum convex hull of the L0 norm, used for row sparseness, and the mathematical expression is ‖W k ‖ 2,1 ; S203: assign corresponding weights: S203 sparse weight matrix W k Perform the Hadamard product operation with the feature matrix X, and the expression is as follows: Where ⊙ represents the Hadamard product, that is, W k Multiply X element-wise, is the original feature matrix, which is the same for all tasks; then the weighted feature matrix Input feature sharing module to learn other task information that is beneficial to its own training.

3. According to the method for multi-task interpretable graph convolution based on attention mechanism for Alzheimer's disease in claim 1, the specific steps performed by the feature sharing module in step S103 include: S301: Graph convolution operation: In order to enable the subsequent attention mechanism to better share feature information, S301 And the adjacency matrix A is input into the graph convolution layer to learn the characteristics and structural information of the nodes; The specific formula is as follows: in, is the output after graph convolution, is a diagonal matrix, and It is The trainable parameter matrix of the layer in the kth task, σ(·) represents the activation function; after L layers of graph convolution, we can get the embedded feature matrix S302: Attention-based feature sharing between tasks: The input of the attention unit is the embedded feature after L layers of graph convolution And its output is a shared representation This representation is a linear combination of the m task embedding feature matrices. The attention-based feature sharing process can be described by the following formula: in, is the jth feature of the ith sample in the kth task after t times of feature sharing through the attention unit, t∈{1,2,...,T}, is a trainable parameter, which indicates the proportion of information obtained by the i-th task from the j-th task, and Satisfies the following formula: The larger the value of , the higher the degree of sharing between task i and task j. It means that task i does not absorb information from task j; S302 places the attention unit after the graph convolution network; specifically, after each feature extraction through graph convolution, the embedded feature matrix is ​​input into the attention unit to perform information interaction, and after layer graph convolution and T times of sharing, the obtained features are input into the task-specific module.

4. According to the method for multi-task interpretable graph convolution based on attention mechanism for Alzheimer's disease in claim 1, the specific steps performed by the task-specific module in S104 include: S401: Classification task prediction: After the feature sharing module, the feature matrix of m tasks can be obtained If the kth task is a classification task, S401 embeds the feature matrix Input to the softmax layer to obtain the probability that the i-th sample belongs to the j-th class The specific expression is as follows: In order to obtain the probability of each sample corresponding to a category, S401 takes the highest probability index of the category to which each sample belongs as the prediction result. So as to get the final predicted labels of all samples S402: Regression task prediction: If the task is a regression task, after the feature sharing module, the feature matrix of the task can be obtained S402 inputs the matrix into a multi-layer perceptron (MLP) to output a predicted value Its mathematical expression is

Images