Early AD brain tissue morphology characterization and classification method of attention 3D dense connection convolutional network
Through the attention mechanism, 3D densely connected convolutional network, the problems of insufficient feature representation ability and poor interpretability in early diagnosis of Alzheimer's disease are solved, and high-precision Alzheimer's disease classification and brain morphological evolution trajectory are achieved. Visualization improves early diagnosis capabilities.
Patent Information
- Application Number
- CN202411312480.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-12-06
AI Technical Summary
The existing early diagnosis method of Alzheimer's disease based on deep learning has insufficient feature characterization ability, poor interpretability, data leakage problems, and insufficient utilization of magnetic resonance image features. It is difficult to reveal the dynamic atrophy process of the brain, affecting early diagnosis ability. .
Adopting attention mechanism 3D densely connected convolutional networks, through magnetic resonance brain image preprocessing, data fusion and network construction, combining convolution, spatial attention, pooling, dense connection and full connection modules, the Alzheimer's disease is achieved Automatic classification diagnosis of early brain morphology and generate interpretable attention maps.
It improves the classification performance of Alzheimer's disease, provides attention maps that can be used to explore the evolution of brain morphology, assists AD diagnosis and treatment, and significantly improves diagnostic accuracy and interpretability.
Smart Images

Figure CN119090865A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of brain science, and in particular to an early AD brain morphology characterization and classification method of an attention mechanism 3D densely connected convolutional network. Background Art
[0002] Alzheimer's disease (AD) is a progressive neurodegenerative disease characterized by the loss and destruction of brain neurons, and its disease process is irreversible. The main clinical features of the disease include memory, cognitive and other functional disorders. Its incidence is high in the elderly and has become an important burden on society. Mild cognitive impairment (MCI) is a transitional stage from cognitively normal (CN) to AD. During the CN->MCI->AD transformation process, the patient's subcortical structures and cortex atrophy, and the ventricles expand. It is very important to explore and discover reliable and effective biomarkers for early AD such as MCI, and then use them to accurately predict AD and intervene in the disease process.
[0003] Early diagnosis of AD is a key link to timely intervene and prevent or delay the progression of the disease, and magnetic resonance imaging (MRI) has been widely used in early diagnosis of AD. Traditional MRI-based voxel morphology AD analysis methods usually only focus on local information at the voxel level and cannot fully consider the interaction between global brain voxels or regions. In recent years, deep learning technology has been widely used in AD classification problems based on multi-voxel joint analysis of magnetic resonance images. Convolutional neural networks and their variants have been favored by researchers in AD classification as mainstream deep learning technologies. This type of method extracts features from the magnetic resonance images of the input subjects through convolution, and then connects these extracted features to the fully connected layer for AD classification. The advantages of this AD classification method based on convolutional neural networks are fast speed and good performance. However, this type of method has problems such as insufficient representation ability of the extracted features and lack of interpretability. At present, the AD classification method of convolutional neural networks based on attention mechanisms has received more and more attention from researchers. This method combines attention mechanisms with convolutional neural networks to improve the representation ability and interpretability of AD features. The advantages of this AD classification method of convolutional neural networks based on attention mechanisms are good performance. However, the AD features extracted by this type of method are not very interpretable.
[0004] Although the above-mentioned deep learning-based AD classification methods have achieved high classification accuracy, these methods have potential data leakage problems and insufficient use of input network magnetic resonance image features. In addition, the AD features extracted by these methods are not very interpretable. Further research is needed to explore the dynamic atrophy process of the subjects' brains and reveal the morphological biomarkers of early AD in order to improve the early diagnosis of AD. Summary of the invention
[0005] In response to the problems existing in the above-mentioned deep learning-based AD characterization and classification methods, the present invention proposes an early AD brain morphology characterization and classification method based on an attention mechanism 3D densely connected convolutional network. The invention can realize automatic classification diagnosis between any two categories of CN, MCI, and AD based on three-dimensional magnetic resonance imaging, and explain the longitudinal evolution of brain morphology during the transformation of subjects from normal cognition to mild cognitive impairment and then to Alzheimer's disease.
[0006] To achieve the above objectives, the present invention provides an early AD brain morphology characterization and classification method based on an attention mechanism 3D densely connected convolutional network, comprising the following steps:
[0007] Step S1, MRI brain image preprocessing: image preprocessing is performed using CAT12, namely, uneven field correction, brain tissue segmentation, spatial standardization registration, and spatial normalization modulation are performed on the MRI brain image.
[0008] Step S2, three-channel data fusion and data set division: First, the gray matter (GM), white matter (WM) and cerebrospinal fluid (CSF) images obtained by step S1 preprocessing are converted into numpy array format, and then channel fusion is performed to generate 3-channel image feature data, and this data is used as the input of the attention mechanism 3D densely connected convolutional network proposed in the present invention. The data of each category is randomly divided into a training set and a test set in a ratio of 9:1 to prepare for model training.
[0009] Step S3, attention mechanism 3D densely connected convolutional network construction: including convolution module, spatial attention module, pooling module, dense connection module + Transition module and fully connected module in sequence. Specifically, in the attention mechanism 3D densely connected convolutional network, the front end is the convolution module, which is used for primary image feature extraction; a spatial attention module is connected after the convolution module, which performs voxel-by-voxel weighting on the feature map extracted after the convolution operation in the spatial dimension to capture the global information of each channel feature map, so that the network can learn more discriminative features for classification, while suppressing or ignoring non-discriminative features; the spatial attention module is followed by a pooling module, which uses the maximum pooling operation to reduce the size of the input feature map. Subsequently, the first densely connected module is connected, which contains 10 dense layers. The output feature map of each layer will be integrated into the next layer through splicing, and the network representation ability will be improved through feature map reuse. After this densely connected module is the Transition module, which contains a convolution layer and an average pooling layer, which compress the number of channels and the size of the feature map of the input feature map respectively. Next, the second dense connection module and the Transition module are connected. The structures and parameters of these two modules are the same as those of the first dense connection module and the Transition module. Finally, the end of the network is a fully connected module, which consists of three fully connected layers. Among them, the third fully connected layer is a binary classification output layer, where the Softmax classifier is used to output the final classification result.
[0010] Step S4, network parameter training and classification test: Build a densely connected network according to step S3, input the training set and test set data obtained in step S2 into the network for training and testing, and adjust the network parameters according to the results until the network converges. After obtaining a better model weight, re-divide the data set to conduct ten cross-validation experiments to test the model performance.
[0011] Step S5, attention map statistical analysis: For the obtained attention map of the test subject, a statistical test is performed to test whether the position feature of each voxel in the attention map has an important contribution to AD classification.
[0012] The beneficial effects of the present invention are:
[0013] (1) The present invention adopts three strategies, such as densely connected network as the backbone network, multi-view brain tissue images (GM, WM, CSF) as network input, and online sampling training of all pixels, which effectively improves the classification performance of the designed network for AD.
[0014] (2) The attention mechanism is introduced on the basis of densely connected networks, which not only further improves the classification performance, but also provides an attention map that can be used to explore the evolution trajectory of AD brain morphology, which can assist in AD diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.
[0016] Figure 1 It is a schematic diagram of the overall process of the method of the present invention;
[0017] Figure 2 Schematic diagram of the attention 3D densely connected convolutional network of the present invention;
[0018] Figure 3 The classification accuracy results of the present invention in three classification tasks are shown;
[0019] Figure 4 The statistical results of the attention maps of the test subjects in three classification tasks of the present invention are displayed. DETAILED DESCRIPTION
[0020] The present invention is further described in detail and clearly in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0021] like Figure 1 As shown, the present invention proposes a method for biological characterization and classification of MCI and AD neuroimaging using a 3D densely connected convolutional network with an attention mechanism, and the specific implementation method includes the following steps:
[0022] Step S1: Image preprocessing: Use CAT12 to perform image preprocessing, that is, perform uneven field correction, brain tissue segmentation, MNI standard space registration, and spatial normalization compensation modulation on brain MRI. This method uniformly adjusts the size of the MRI image without the skull to 141×180×150.
[0023] Step S2: The segmented gray matter, white matter, and cerebrospinal fluid images are fused in channel dimension into a 3-channel image data with a size of 3×141×180×150, and the data is divided into a training set and a test set.
[0024] Step S3: Construct an attention 3D densely connected convolutional network: including convolution module, spatial attention module, pooling module, dense connection module, transition module, and fully connected module in sequence.
[0025] like Figure 2 As shown in the figure, the convolutional layer of the convolutional module contains 16 convolution kernels of size 3×3×3, the convolution step is 2, and the padding is 1. It is followed by a rectified linear unit (ReLU) activation layer and a batch normalization (BN) layer; the attention module consists of a convolutional layer, two pooling layers, and a ReLU activation layer, where the convolution kernel size of the convolutional layer is 1×1×1, the number of convolution kernels is 1, the convolution step is 3, and the padding is 1. The average pooling layer and the maximum pooling layer, both average pooling and maximum pooling are pooled in the channel dimension. The activation layer contains two activation layers, ReLU and Tanh, in sequence; the pooling module consists of a maximum pooling layer, the pooling kernel size is 3×3×3, the pooling step is 3, and the padding is 1.
[0026] The dense connection module consists of 10 Dense layers, each of which contains two convolutional layers, two ReLU layers, and two BN layers. The detailed structure of each Dense layer is that the BN layer and the ReLU layer are in the front, followed by a convolutional layer (the convolution kernel size is 1×1×1, the number of convolution kernels is 4×24, and the convolution step is 1). After the convolutional layer, a BN layer, a ReLU layer, and a convolutional layer (the convolution kernel size is 3×3×3, the number of convolution kernels is 24, the convolution step is 1, and the padding is 1) are connected in series. After each Denseblock, an identical Transition module is connected in series. The series-connected Transition module consists of a convolutional layer (the convolution kernel size is 1×1×1, the number of convolution kernels is 128, and the convolution step is 1) and an average pooling layer (the pooling kernel size is 1×1×1, and the pooling step is 2). According to the above design of the backbone network, for a feature map with an input size of N×C×H×W×D, after passing through the second or second densely connected module, the output size becomes a feature map of N×(C+24*10)×H×W×D, and after passing through the subsequent Transition module, the output size becomes a feature map of N×((C+24*10) / 2)×(H / 2)×(W / 2)×(D / 2). Here N is the batch size, C is the number of channels, and H×W×D is the size of the 3D feature map.
[0027] The end of the network is a fully connected module, which is mainly composed of 3 fully connected layers. The number of input nodes of the first fully connected layer is determined by the size of the features after the downsampling module is converted into a one-dimensional feature vector, and the output node is 1000. The number of input nodes of the second fully connected layer is 1000, and the number of output nodes is 100. The number of input nodes of the last fully connected layer is 100, and the last output node is the number of classification categories 2. In addition, each fully connected layer is followed by a BN layer and a ReLU activation layer.
[0028] Specifically, the specific classification process of the network is described in detail by taking a single subject input data into the network as an example: the 3-channel fused image (image size 3×141×180×150) obtained in step S2 is input into the convolution module, and the output feature map (16×71×90×75) is obtained after convolution, and then the output feature map is simultaneously passed through the average pooling layer and the maximum pooling layer in the spatial attention module. Both pooling layers perform channel dimension pooling on the feature map to obtain two output feature maps (both with a size of 1×71×90×75), and then the two feature maps are spliced to obtain a feature map with a size of 2×71×90×75. This feature map is passed through the convolution layer in the spatial attention module to obtain a feature with a size of 1×71×90×75, and the feature is then passed through the two activation layers of ReLU and Tanh in sequence to obtain the final spatial attention weight. The obtained weight is multiplied pixel by pixel with the feature map size of 16×71×90×75 that initially passes through the convolution module (spatial weighted processing), and finally an output feature map of unchanged size (16×71×90×75) is obtained.
[0029] The weighted feature map is then input into the pooling module, which is the maximum pooling module, and the feature map after pooling is (16×24×30×25). The feature map after pooling is sent to the dense connection module. Each dense connection module consists of 10 dense connection layers, and each dense connection layer contains two convolutional layers. After passing through the two convolutional layers in a dense connection layer, the number of channels of the feature map will become 24 (growth rate = 24), but due to the structure of the dense connection module, the output features of each layer will be spliced with the output features of the previous layer, that is, after a dense connection layer with 10 layers, the number of channels of the output features is 16+24×10=256. The size of the feature map after the first dense connection module is (256×24×30×25). There are two dense connection modules in the network structure, and each dense connection module is followed by a downsampling module. The downsampling module is mainly composed of a convolution layer and a pooling layer. The function of the convolution layer is to halve the number of channels of the feature map. Before passing through the convolution layer, the feature map first passes through a BN layer and a ReLU activation layer, and then passes through the convolution layer. The feature map size becomes 128@24×30×25, and finally passes through the end of the downsampling module, the pooling layer. The pooling layer is average pooling, the pooling kernel size is 2×2×2, and the pooling step is 2. After pooling, the feature map size is 128@12×15×12. After that, it passes through the next dense connection module to obtain a feature map of size 368@12×15×12. And similarly, it passes through the downsampling module, first passing through the BN layer in the downsampling module and then passing through the ReLU activation layer, and then passing through the convolution layer, the number of channels is halved to obtain a feature map (184@12×15×12), and finally passing through the pooling layer to obtain the output feature map (184@6×7×6).
[0030] Before the three-dimensional feature vector is input into the fully connected layer, it needs to be converted and flattened into a one-dimensional vector. After the output feature map of the downsampling module passes through the BN layer and the ReLU activation layer, it is flattened by the Flatten layer and then converted into a one-dimensional feature vector of size 1×46368 by passing through a dropout layer (dropout_rate=0.5) to prevent overfitting. Then, the one-dimensional vector is input into the first fully connected layer, with an output node of 1000. After the first fully connected layer, a BN layer and a ReLU activation layer are connected to obtain a one-dimensional vector (1×1000). Then, the one-dimensional vector is input into the second fully connected layer, with an output node of 100. Similarly, after passing through the BN layer and the ReLU activation layer, a one-dimensional feature vector of size 1×100 is obtained. The last fully connected layer is the classification result output layer, with an output node of 2. The one-dimensional feature vector (1×100) is input into the fully connected layer, and the final classification result is obtained by the softmax activation function.
[0031] Step S4: After building the network according to step S3, first randomly shuffle the data into 10 equal parts by category, use the 10-fold cross-validation method to optimize the model parameters and save the attention weight map corresponding to each fold during testing. When dividing the data, in order to ensure data balance, first divide the subjects of different categories into 10 equal parts, of which 9 parts are used as training data each time, and the remaining 1 part is used for testing. Then randomly combine images of different categories to generate 10-fold data. During network training, 8 subject images are input as a small batch (i.e., batchsize=8), and the initial learning rate is 1e -3 As the number of iterations increases, the learning rate gradually decreases, and the calculation formula is learning rate = 1e -3 ×(1-epoch / epochs). The total number of iterations is 150 (ie, epochs = 150), and epoch is the current iteration.
[0032] Step S5: In the classifications of CNvs.MCI, MCIvs.AD, and CNvs.AD, statistical tests are performed on the attention maps obtained from the test set subjects. In statistical tests, common t-tests and variance analysis require that the samples conform to the normal distribution. However, the voxels of the attention map here are all non-negative values and do not conform to the normal distribution, so the present invention uses a non-parametric permutation test to perform statistical analysis on the attention map. The non-parametric permutation test randomly permutes the voxels in the order of the attention maps of the two groups of samples (CNvs.MCI, CNvs.AD or MCIvs.AD), and recalculates the statistical test quantity, and the number of random permutations is set to 1000.
[0033] When multiple statistical tests are performed voxel by voxel between each two groups of samples, multiple comparison correction is required. The present invention uses the false discovery rate (FDR) for correction, and the attention map is balanced between false positives and false negatives through correction, that is, the false / true positive ratio is controlled within a certain range, the significance level is set to p=0.05, and the number of cluster voxels is set to be greater than 500.
[0034] The results of the ten-fold cross validation of the three classification tasks of AD vs MCI, AD vs CN and MCI vs CN are as follows: the average classification accuracy of AD vs MCI is 98.60%, the average classification accuracy of AD vs CN is 97.63%, and the average classification accuracy of MCI vs CN is 98.37%. From the above results, it can be seen that the attention 3D densely connected convolutional network designed by the present invention has achieved high accuracy in the three classification tasks. Figure 3As shown in the figure, ablation comparison experiments were conducted on the network with and without the spatial attention module through one-fold verification, verifying the positive impact of the spatial attention module on the performance of the 3D densely connected convolutional network. In the AD vs MCI task, the 3D densely connected convolutional network with attention achieved 100% accuracy, and the dense convolutional network without attention had an accuracy of 99.09%. The accuracy was improved by 1% after the attention module was added to the network. In the two classification tasks of AD vs CN and MCI vs CN, although the classification accuracy of the network with the attention module did not improve compared with the network without the attention module, the classification process of the network can be explained by visualizing the feature map of the spatial attention module.
[0035] like Figure 4 As shown in Figure 2, the method of the present invention not only improves the AD classification performance, but also provides an attention map for exploring the evolution trajectory of AD brain morphology. In the classification of CN vs. MCI, MCI vs. AD, and CN vs. AD, the attention maps obtained on the test set subjects were statistically tested to obtain the distribution maps of brain regions that have significant contributions to the classification (such as Figure 3 (as shown in the figure), as follows. (1) During the transformation of CN to MCI, brain morphological abnormalities occurred in subcortical structures such as the hippocampus, parahippocampal gyrus, amygdala, thalamus, lentiform nucleus, putamen, caudate nucleus, globus pallidus, and fusiform gyrus. (2) During the transformation of MCI to AD, brain morphological abnormalities also occurred in newly added cortical structures such as the medial superior frontal gyrus, intraorbital superior frontal gyrus, middle temporal gyrus, inferior temporal gyrus, temporal pole, middle occipital gyrus, fusiform gyrus, precentral gyrus, postcentral gyrus, precuneus, angular gyrus, supramarginal gyrus, and paracingulate gyrus. (3) In the process of direct transformation from CN to AD, brain morphological abnormalities occur in structures such as the medial superior frontal gyrus, middle frontal gyrus, inferior frontal gyrus, triangular part of inferior frontal gyrus, middle temporal gyrus, inferior temporal gyrus, middle occipital gyrus, fusiform gyrus, lingual gyrus, cuneus, superior parietal gyrus, precentral gyrus, postcentral gyrus, precuneus, angular gyrus, supramarginal gyrus, insula, hippocampus, parahippocampal gyrus, amygdala, thalamus, lenticular nucleus, putamen, caudate nucleus, globus pallidus, medial cingulate gyrus, paracingulate gyrus, and cerebellum. In other words, the gray matter morphology of almost the entire brain, including subcortical structures, cortical structures, and cerebellum, is abnormal.
[0036] In the early stages of AD (i.e. the MCI stage here), the present invention discovered subcortical structures that had earlier abnormal brain morphology, such as the hippocampus, amygdala, parahippocampal gyrus, thalamus, and lentiform nucleus. Among them, the medial temporal lobe, such as the hippocampus and amygdala, is the earliest brain structure found to have pathological changes of neurofibrillary tangles and amyloid deposition in Alzheimer's disease, i.e. AD. These brain structures play an important role in the encoding and retrieval of situational and spatial memories, i.e. the formation of new memories and the recovery of old memories, and are therefore called the "gate of memory". It can be seen that the situational and spatial memory defects of AD are closely related to the damage of these brain structures.
Claims
1. A method for characterizing and classifying early AD brain tissue morphology using an attention 3D densely connected convolutional network, characterized in that: Here are the steps: Step S1, brain MRI image preprocessing: Cat12 toolbox is used for image preprocessing, and each subject's MRI image is subjected to inhomogeneous field correction, brain tissue segmentation, MNI standard spatial registration, and spatial normalization compensation modulation in turn. Step S2, the segmented gray matter, white matter and cerebrospinal fluid are fused to generate a 3-channel spliced 4D image. Step S3, construct an attention 3D densely connected convolutional network: including convolution module, spatial attention module, pooling module, dense connection module, downsampling module, and fully connected layer module in sequence. Specifically, in the 3D densely connected convolutional network, the front end is the convolution module, which is used for image feature extraction. A spatial attention module is connected after the convolution module, which performs pixel weighting of the spatial dimension on the features after convolution to capture global information while ignoring most useless information. After the spatial attention module is the pooling module, which is the maximum pooling module to reduce the input feature size. Subsequently, the first densely connected module is connected. Each densely connected module has 10 dense layers. The feature output of each layer will be used as input to the next layer to achieve feature reuse and enhance feature extraction. After the first densely connected module is the first downsampling module, which contains a convolution layer and an average pooling layer, which respectively reduces the channel dimension and feature size of the input features. After the first downsampling module, the second densely connected module is connected, which has the same composition as the first densely connected module. Correspondingly, after the second dense connection module, there is the second downsampling module. Finally, at the end of the network structure is the fully connected layer module, which is mainly composed of 3 fully connected layers. The last layer is the classification task output layer, which uses the Softmax activation function for classification. Step S4, model parameter training and classification test: After building the network according to step S3, a 10-fold cross-validation method is used to perform network training and testing.
2. The early AD brain morphology characterization and classification method of the attention 3D densely connected convolutional network according to claim 1 is characterized in that: The input image in step S2 is a channel fusion of the gray matter, white matter, and cerebrospinal fluid images that have been segmented after preprocessing, so as to enhance the feature representation capability of the input image by imitating a 3-channel input feature image.
3. The early AD brain morphology characterization and classification method of the attention 3D densely connected convolutional network according to claim 1 is characterized in that: The structure of the attention 3D densely connected convolutional network in step S3 is designed as follows: a spatial attention module is embedded in the front end of the network, which performs voxel weighting on the features after convolution in the spatial dimension to capture global information while ignoring most of the useless feature information.
4. The method for characterizing and classifying early AD brain morphology using an attention 3D densely connected convolutional network according to claim 1, characterized in that: The spatial attention module used by the attention 3D densely connected convolutional network in step S3 performs a non-parametric permutation test on the weight map of the attention module to test whether each voxel value of the attention map is statistically significantly different from 0, that is, to test whether the voxel position feature has an important contribution to AD classification, thereby discovering brain morphological features closely related to AD.
