Alzheimer disease onset risk prediction method based on CTA framework
Through the 3D convolutional neural network and Transformer attention module of the CTA framework, combined with the total mean square variance loss, the inaccuracy problem of Alzheimer's disease incidence risk prediction is solved, and more accurate brain image feature representation and clinical score prediction are achieved. , reducing subjective bias.
Patent Information
- Application Number
- CN202411057709.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-08-02
AI Technical Summary
The prior art has inaccuracy in the prediction of the incidence risk of Alzheimer's disease, mainly due to subjective bias in the assessment of cognitive impairment caused by doctors' subjective factors and individual differences, and the prediction methods based on sMRI cannot be effectively optimized.
Using a CTA framework method, brain image features are extracted through 3D convolutional neural network, adaptive information adjustment is performed by combining Transformer attention module, and joint prediction is carried out through fully connected regression layer and total mean square variance loss to realize Alzheimer's disease Predictions of the risk of disease.
This method can automatically identify specific discriminant locations in the whole brain image, seamlessly learn high-level feature representations, improve the prediction accuracy of clinical cognitive scores, reduce subjective bias, and achieve more accurate Alzheimer's disease risk assessment.
Smart Images

Figure CN118983099A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of medical image and intelligent information processing, and specifically relates to a method for predicting the risk of Alzheimer's disease based on a CTA framework. Background Art
[0002] Alzheimer's disease (AD) is one of the most common types of dementia, characterized by a progressive decline in cognitive abilities and daily functions. This neurodegenerative disease usually affects the elderly and is irreversible. The onset of AD severely disrupts the lives of patients and their families, and imposes a heavy burden on society and the economy. Currently, there are no effective drugs or clinical treatments that can stop or reverse the progression of AD. Therefore, early diagnosis and assessment of AD are crucial to slowing its progression. However, due to the complexity of AD, it is difficult for humans to fully understand its pathogenesis, which poses many challenges to diagnosis and assessment. Nowadays, the diagnosis of AD is mainly carried out through a comprehensive assessment of cognitive behavioral assessments and imaging biomarkers. Among them, cognitive scales are used to assess the phenotypic characteristics of the subject's cognitive state, such as the Mini-Mental State Examination (MMSE), the Clinical Dementia Rating Scale (CDRS), and the Alzheimer's Disease Assessment Scale (ADAS). Due to problems such as the subjective factors of doctors and individual differences of patients, there are also many subjective biases in the assessment of cognitive impairment.
[0003] With the advancement of artificial intelligence and the accumulation of brain imaging data, accurate estimation of clinical scores through brain imaging can help understand the pathological stage of AD and predict its progression. As one of the most important imaging techniques in clinical practice, sMRI is widely used in the diagnosis and evaluation of AD. Therefore, with the help of AI technology, AD assessment based on brain sMRI can circumvent subjective bias caused by various factors such as doctors or subjects. At present, many machine learning-based methods usually pre-select dementia-related areas from sMRI data for feature extraction and prediction model construction. However, these methods may not be able to achieve optimization due to the potential heterogeneity between different steps and the individual specificity in the progression of dementia. Combining deep learning methods for feature encoding and model construction can achieve task-oriented feature learning more effectively. Summary of the invention
[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for predicting the risk of Alzheimer's disease based on a CTA framework, which solves the problem of inaccurate prediction of the onset of Alzheimer's disease in the prior art.
[0005] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a method for predicting the risk of Alzheimer's disease based on the CTA framework, comprising the following steps:
[0006] S1, input the brain image into the 3D convolutional neural network for feature extraction to obtain a brain image feature map;
[0007] S2, input the brain image feature map into the Transformer attention module, adaptively adjust the information between different positions of the brain image feature map, and obtain the final attention feature map;
[0008] S3. The final attention feature map is convolved with the brain image feature map. The convolution result is input into the fully connected regression layer after global pooling. The fully connected regression layer is combined with the total mean square error loss to jointly predict several cognitive scores and obtain the predicted probability of Alzheimer's disease.
[0009] Further: in said S1, the 3D convolutional neural network includes first to fourth convolutional subnetworks connected in sequence;
[0010] Among them, the structures of the first to third convolutional sub-networks are the same, including the first convolutional layer, the second convolutional layer and the 2×2×2 maximum pooling layer connected in sequence, and the convolution kernels of the first convolutional layer and the second convolutional layer are both 3×3×3;
[0011] The fourth convolutional subnetwork includes the third convolutional layer, the fourth convolutional layer and the 1×1×1 global average pooling layer connected in sequence. The convolution kernel of the third convolutional layer is 3×3×3, and the convolution kernel of the fourth convolutional layer is 1×1×1.
[0012] The outputs of the first to fourth convolutional layers are sequentially connected to the batch normalization layer and the ReLU activation function layer to extract and represent the spatial features of brain images.
[0013] Further: In S2, the Transformer attention module includes an encoder, a decoder, a connection layer and a softmax activation function layer connected in sequence;
[0014] The encoder includes a first multi-head attention layer and a first forward position feedback network layer, and the input end of the first forward position feedback network layer is connected to the output end and the input end of the first multi-head attention layer by normalization and residual connection;
[0015] The decoder includes a masked multi-head attention layer, a second multi-head attention layer and a second forward position feedback network layer. The input end of the second multi-head attention layer is connected to the output end and input end of the first forward position feedback network layer and the output end and input end of the masked multi-head attention layer by normalization and residual connection. The input end of the second forward position feedback network layer is connected to the output end and input end of the second multi-head attention layer by normalization and residual connection. The input end of the connection layer is connected to the output end and input end of the second forward position feedback network layer by normalization and residual connection.
[0016] Furthermore: the masked multi-head attention layer, the first multi-head attention layer and the second multi-head attention layer have the same structure, including three parallel first convolutions, several scaled dot product attention modules and second convolutions, the input end of each scaled dot product attention module is connected to the three parallel first convolutions, and the output end of each scaled dot product attention module is connected to the second convolution.
[0017] Further: S2 comprises the following sub-steps:
[0018] S21, embedding the brain image feature map into the 3D spatial position information through position encoding to obtain an input feature map;
[0019] S22, input the input feature map into the first multi-head attention layer to obtain a first attention feature map;
[0020] S23, inputting the first attention feature map and the input feature map into the first forward position feedback network layer to obtain an attention feature map of the encoder;
[0021] S24, input the input feature map into the masked multi-head attention layer to obtain a second attention feature map;
[0022] S25, inputting the second attention feature map, the input feature map, and the attention feature map of the encoder into the second multi-head attention layer to obtain a third attention feature map;
[0023] S26 inputs the third attention feature map and the second attention feature map into the second forward position feedback network layer to obtain an attention feature map of the decoder;
[0024] S27, input the attention feature map of the decoder and the third attention feature map into the softmax activation function layer through the connection layer to obtain an attention weight map;
[0025] S28. Fuse the attention weight map and the brain image feature map to obtain the final attention feature map.
[0026] Further: In S21, the position encoding is the same as the five-dimensional shape parameters of the brain image feature map, and the five-dimensional shape parameters include batch size, depth, height, width and number of channels. The expression of the position encoding PE is specifically:
[0027] PE = sin(pos×div_term)
[0028] Where sin(·) is the sine function, pos is the position index tensor, and div_term is the scaling factor.
[0029] Further: the masked multi-head attention layer, the first multi-head attention layer and the second multi-head attention layer calculate the output feature map according to the input feature map in the same way, specifically:
[0030] SA1, maps the input feature map into query, key and value through three parallel first convolutions;
[0031] SA2. Input the query, key, and value into all scaled dot product attention modules to obtain the calculation results of each scaled dot product attention module.
[0032] SA3. Concatenate the calculation results of all scaled dot product attention modules through the second convolution to obtain the output feature map.
[0033] Further: In the SA2, the calculation result head of the i-th scaled dot product attention module is obtained i The specific expression is:
[0034] head i =Conv3D(Attention(Q,K,V))
[0035] Where Conv3D(·) is a 3D convolution operation, Attention(Q i ,K i ,V i ) is the self-attention dot product matrix, and its specific expression is:
[0036]
[0037] Where softmax(·) is the softmax activation function, Q is the query, K is the key, V is the value, T is the transposed symbol, and d k is the dimension of the key vector;
[0038] In SA3, the expression of MultiHead(Q, K, V) after the calculation results of all scaled dot product attention modules are concatenated is as follows:
[0039] MultiHead(Q,K,V)=Conv3D(concat(head1,...,head h ))
[0040] In the formula, concat(·) is the concatenation operation, head1 is the calculation result of the first scaled dot product attention module, and head h is the calculation result of the h-th scaled dot product attention module.
[0041] Further: In S3, the total mean square error loss L reg The specific expression is:
[0042] L reg =MSE ADAS11 +MSE CDRSB +MSE MMSE
[0043] In the formula, MSE ADAS11 is the mean square error of the cognitive score with the score label ADAS11, MSE CDRSB is the mean square error of the cognitive score with the score label CDRSB, MSE MMSE is the mean square error of the cognitive score with the score label MMSE, where the expression for calculating the mean square error MSE of any cognitive score is:
[0044]
[0045] Where N is the number of samples, y i is the true score value of the nth sample, is the predicted score value of the nth sample.
[0046] The beneficial effects of the present invention are as follows: the present invention provides a method for predicting the risk of Alzheimer's disease based on a CTA framework, which can automatically identify the discriminant position of a specific participant in a whole-brain image, and seamlessly learn high-level feature representations to construct a regression model for estimating clinical cognitive scores. The specific contributions of the present invention are as follows:
[0047] (1) Encoding brain images using 3D convolutional neural networks;
[0048] (2) The use of the Transformer attention module allows adaptive fusion of information between different positions or features, and dynamically adjusts weights according to importance. This means that during encoding of brain images, the attention mechanism can adaptively fuse information from different brain regions to better represent the structural characteristics of the entire brain.
[0049] (3) Multiple cognitive scores are jointly predicted through a fully connected regression layer combined with the total mean square error loss, ultimately improving the prediction accuracy of clinical scores. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 This is a flow chart of a method for predicting the risk of Alzheimer's disease based on a CTA framework according to the present invention.
[0051] Figure 2 This is a deep learning architecture diagram that integrates CNN and Transformer attention.
[0052] Figure 3 This is a structural diagram of the 3D convolutional neural network of the present invention.
[0053] Figure 4 This is the structure diagram of the Transformer self-attention module of the present invention.
[0054] Figure 5 This is a structural diagram of the multi-head attention layer in the Transformer self-attention module of the present invention. DETAILED DESCRIPTION
[0055] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.
[0056] Embodiment 1:
[0057] like Figure 1 As shown, in one embodiment of the present invention, a method for predicting the risk of Alzheimer's disease based on a CTA framework comprises the following steps:
[0058] S1, input the brain image into the 3D convolutional neural network for feature extraction to obtain a brain image feature map;
[0059] S2, input the brain image feature map into the Transformer attention module, adaptively adjust the information between different positions of the brain image feature map, and obtain the final attention feature map;
[0060] S3. The final attention feature map is convolved with the brain image feature map. The convolution result is input into the fully connected regression layer after global pooling. The fully connected regression layer is combined with the total mean square error loss to jointly predict several cognitive scores and obtain the predicted probability of Alzheimer's disease.
[0061] like Figure 2 As shown, in this embodiment, the present invention designs a deep learning architecture that integrates CNN and Transformer Attention (CTA) for estimating multiple cognitive scores from brain images scanned by SMRI, and the deep learning architecture includes a 3D convolutional neural network, a Transformer attention module, and a fully connected regression layer for estimating multiple cognitive scores.
[0062] The brain image sampling method is divided into multiple training batches. The sampling strategy of each training batch is as follows: N subjects are randomly selected from the training data set to form a batch training data containing N brain images. Then, N brain images of size 121×145×121 are sent to the 3D convolutional neural network for training, and then the output brain image feature map size is 9×12×9.
[0063] In S1, the 3D convolutional neural network includes first to fourth convolutional subnetworks connected in sequence;
[0064] Among them, the structures of the first to third convolutional sub-networks are the same, including the first convolutional layer, the second convolutional layer and the 2×2×2 maximum pooling layer connected in sequence, and the convolution kernels of the first convolutional layer and the second convolutional layer are both 3×3×3;
[0065] The fourth convolutional subnetwork includes the third convolutional layer, the fourth convolutional layer and the 1×1×1 global average pooling layer connected in sequence. The convolution kernel of the third convolutional layer is 3×3×3, and the convolution kernel of the fourth convolutional layer is 1×1×1.
[0066] The outputs of the first to fourth convolutional layers are sequentially connected to the batch normalization layer and the ReLU activation function layer to extract and represent the spatial features of brain images.
[0067] In this embodiment, the structure of the 3D convolutional neural network is as follows: Figure 3 As shown in the figure, batch normalization (BatchNorm) and ReLU activation functions are used after each convolution layer of the 3D convolutional neural network to effectively extract and represent the spatial features of brain images. Specifically, the convolution layer uses a 3×3×3 convolution kernel to capture local features in three-dimensional space, and the final 1×1×1 convolution kernel convolution layer is used to integrate the information in the feature map. The 2×2×2 maximum pooling layer reduces the size of the feature map by downsampling while retaining important spatial information. The 1×1×1 global average pooling (GAP) layer summarizes the global information of each feature map to provide a compact feature representation; a batch normalization (BatchNorm) layer is used after each convolution layer to stabilize the training process and accelerate convergence. The ReLU activation function is applied after each convolution layer to introduce nonlinearity, thereby enhancing the expressive power of the model. This architectural design not only ensures that the depth and complexity of the model are moderate, but also retains sufficient feature extraction capabilities, which helps to better capture and represent key information in brain images.
[0068] In S2, the Transformer attention module includes an encoder, a decoder, a connection layer and a softmax activation function layer connected in sequence;
[0069] The encoder includes a first multi-head attention layer and a first forward position feedback network layer, and the input end of the first forward position feedback network layer is connected to the output end and the input end of the first multi-head attention layer by normalization and residual connection;
[0070] The decoder includes a masked multi-head attention layer, a second multi-head attention layer and a second forward position feedback network layer. The input end of the second multi-head attention layer is connected to the output end and input end of the first forward position feedback network layer and the output end and input end of the masked multi-head attention layer by normalization and residual connection. The input end of the second forward position feedback network layer is connected to the output end and input end of the second multi-head attention layer by normalization and residual connection. The input end of the connection layer is connected to the output end and input end of the second forward position feedback network layer by normalization and residual connection.
[0071] In this embodiment, the structure of the Transformer attention module is as follows Figure 4 As shown in the figure, in the multi-head attention layer of the encoder, the input feature map is mapped into query, key and value through three parallel convolutions, and the attention-based image feature representation is obtained by calculating the attention weight of each head. In order to enhance the stability and representation ability of the model, these output features are batch normalized, activated and residually connected. In the decoder, feature fusion and restoration are performed by receiving the attention feature map from the encoder and the original input feature map as input. In the multi-head attention layer of the decoder, the features are calculated and fused, and then mapped back to the high-dimensional space through restoration and deconvolution operations to generate a feature map of the same size as the input image. In this process, the decoder uses skip connections to fuse shallow and deep features to obtain more discriminative feature representations. Through this multi-level feature fusion and deconvolution operation, the decoder not only achieves feature fusion and restoration, but also optimizes the feature representation, enhancing the performance of the model in MRI analysis.
[0072] like Figure 5 As shown, the structures of the masked multi-head attention layer, the first multi-head attention layer and the second multi-head attention layer are the same, including three parallel first convolutions, several scaled dot product attention modules and second convolutions. In this embodiment, h number of them are set, and the input end of each scaled dot product attention module is connected to the three parallel first convolutions, and the output end of each scaled dot product attention module is connected to the second convolution.
[0073] The S2 comprises the following sub-steps:
[0074] S21, embedding the brain image feature map into the 3D spatial position information through position encoding to obtain an input feature map;
[0075] S22, input the input feature map into the first multi-head attention layer to obtain a first attention feature map;
[0076] S23, inputting the first attention feature map and the input feature map into the first forward position feedback network layer to obtain an attention feature map of the encoder;
[0077] S24, input the input feature map into the masked multi-head attention layer to obtain a second attention feature map;
[0078] S25, inputting the second attention feature map, the input feature map, and the attention feature map of the encoder into the second multi-head attention layer to obtain a third attention feature map;
[0079] S26 inputs the third attention feature map and the second attention feature map into the second forward position feedback network layer to obtain an attention feature map of the decoder;
[0080] S27, input the attention feature map of the decoder and the third attention feature map into the softmax activation function layer through the connection layer to obtain an attention weight map;
[0081] S28. Fuse the attention weight map and the brain image feature map to obtain the final attention feature map.
[0082] In S21, the position encoding is the same as the five-dimensional shape parameters of the brain image feature map, and the five-dimensional shape parameters include batch size N, depth D, height H, width W and number of channels C. The expression of the position encoding PE is specifically:
[0083] PE = sin(pos×div_term)
[0084] Where sin(·) is the sine function, pos is the position index tensor, which ranges from 0 to C / 2-1, and div_term is the scaling factor, which is calculated by the exponential function.
[0085] The position code PE can also be calculated by the cosine function, as shown in the following formula:
[0086] PE = cos(pos × div_term)
[0087] The masked multi-head attention layer, the first multi-head attention layer and the second multi-head attention layer calculate the output feature map according to the input feature map in the same way, specifically:
[0088] SA1, maps the input feature map into query, key and value through three parallel first convolutions;
[0089] SA2: Input the query, key, and value into all scaled dot product attention modules to obtain the calculation results of each scaled dot product attention module.
[0090] SA3. Concatenate the calculation results of all scaled dot product attention modules through the second convolution to obtain the output feature map.
[0091] In this embodiment, the scaled dot product attention module can capture the query, key, and value, which are reduced to a spatial dimension of 32 after the 3D convolution operation.
[0092] In SA2, the calculation result head of the i-th scaled dot product attention module is obtained i The specific expression is:
[0093] head i =Conv3D(Attention(Q,K,V))
[0094] Where Conv3D(·) is a 3D convolution operation, Attention(Q i ,K i ,V i ) is the self-attention dot product matrix, and its specific expression is:
[0095]
[0096] Where softmax(·) is the softmax activation function, Q is the query, K is the key, V is the value, T is the transposed symbol, and d k is the dimension of the key vector;
[0097] In SA3, the expression of MultiHead(Q, K, V) after the calculation results of all scaled dot product attention modules are concatenated is as follows:
[0098] MultiHead(Q,K,V)=Conv3D(concat(head1,...,head h ))
[0099] In the formula, concat(·) is the concatenation operation, head1 is the calculation result of the first scaled dot product attention module, and head h is the calculation result of the h-th scaled dot product attention module.
[0100] Due to the characteristics of 3D images, the Transformer attention module does not flatten the 3D image into two dimensions for calculation. Instead, it uses 3D convolution kernels instead of weight matrices. The head in each scaled dot product attention module performs multiple convolutions using different convolution kernels, and the size of each convolution kernel is set to 1. After completing the 3D convolution calculations of Q, K, and V, a deconvolution operation is performed to restore the original dimensions of the image features.
[0101] In S3, the total mean square error loss L reg The specific expression is:
[0102] L reg =MSE ADAS11 +MSE CDRSB +MSE MMSE
[0103] In the formula, MSE ADAS11 is the mean square error of the cognitive score with the score label ADAS11, MSE CDRSB is the mean square error of the cognitive score with the score label CDRSB, MSE MMSE is the mean square error of the cognitive score with the score label MMSE, where the expression for calculating the mean square error MSE of any cognitive score is:
[0104]
[0105] Where N is the number of samples, y i is the true score value of the nth sample, is the predicted score value of the nth sample.
[0106] In S4, during the joint prediction of each cognitive score, the final attention feature map input includes samples of N subjects, represented by {(X n ,y n )} N , where X n represents the nth sample, Represents T cognitive scores. To achieve this goal, the total mean square error loss L is used. reg It is the sum of the mean square errors of cognitive scores of three different score labels in the regression task. The total mean square error loss designed by the present invention will optimize the CTA network through back propagation and integrate its effect into the Transformer attention to assist network learning.
[0107] Embodiment 2:
[0108] This example provides experimental data for Example 1 to verify the effectiveness of the present invention in improving the accuracy of clinical score prediction.
[0109] In this example, the experiment was conducted using the PyTorch platform. The network training used the Adam optimizer, with a batch size of 4 and a dropout rate of 0.3. During the training phase, a total of 70 cycles were performed, using three score labels (ADAS11, CDRSB, MMSE) for the regression task of score estimation.
[0110] In the ADNI database, the irregularity of cognitive impairment diseases leads to significant differences in neuroimaging between different subjects. Therefore, in order to perform score estimation, two independent sub-datasets (dataset1 and dataset2) were used for validation. Specifically, two sub-datasets were constructed from the obtained ADNI database and split into training-validation-test at a ratio of 70%, 15%, and 15%. In addition, there was no overlap in the subjects in the two test sets, which enhanced the generalization ability assessment of the method.
[0111] The present invention evaluates the performance of the proposed method by evaluating three clinical cognitive scores including CDRSB, ADAS11 and MMSE. The prediction performance is quantified using correlation coefficient (CC) and root mean square error (RMSE). The present invention compares six different methods, including ResNet and CNN backbone baseline networks. In addition, spatial attention blocks are applied to these baseline networks to construct Sa-ResNet and Sa-CNN methods respectively. The present invention also includes Transformer attention blocks without position encoding methods (CTA without PE) and the proposed method (CTA with PE). The ResNet method uses 3D ResNet10 as a CNN encoder for brain images, and then performs regression prediction through three fully connected layers. The CNN method uses a 3D CNN backbone without attention blocks to extract brain image features, and then performs regression prediction through three fully connected layers. In Sa-ResNet, Sa-CNN and CTA methods, the attention block is applied to the last layer of the 3D CNN backbone. In the above attention-based methods, the feature map output by the backbone network is encoded by the attention block and then fused with the original feature map through dot product fusion. Through these comparative experiments, we can evaluate the effectiveness of the proposed Transformer block in improving the accuracy of cognitive score estimation.
[0112] Table 1 Prediction results of the model on dataset 1
[0113]
[0114] Table 2 Prediction results of the model on dataset 2
[0115]
[0116] Table 3 Comparison results including traditional learning methods and deep learning methods based on ADNI dataset
[0117]
[0118] The ablation experimental results of score estimation on two sub-datasets of the ADNI dataset are shown in Tables 1 and 2. By comparing and analyzing the experimental data obtained from six different methods, the performance of the proposed method under different configurations can be evaluated, and the following conclusions are drawn.
[0119] First, the experimental results clearly show that the methods that integrate the attention mechanism in the deep learning model outperform those that do not use the attention mechanism on both datasets. This finding suggests that applying the attention module to the feature map after feature extraction can effectively enhance the model's ability to represent the spatial information of MRI images, thereby helping to build a more accurate regression model. Second, the performance of the CTA without PE method exceeds that of the spatial attention-based method. This shows that Transformer attention can supplement local information and improve model performance by effectively focusing on specific areas of brain images. Finally, the performance of the CTA method with the addition of spatial position encoding is better than that of the CTA without PE method. This result emphasizes the importance of spatial position encoding in enhancing the efficiency of the Transformer attention module in processing 3D MRI. Spatial position encoding not only retains the three-dimensional spatial information of the image, enhances the model's understanding of the spatial structure and contextual relationship of the image, but also improves the ability of feature representation. Therefore, the introduction of this mechanism significantly enhances the model's ability to capture and utilize SMRI spatial information, thereby improving the accuracy of cognitive score prediction.
[0120] In addition, in Table 3, the method of the present invention is compared with some traditional learning methods and deep learning methods based on the ADNI dataset to demonstrate the effectiveness of the proposed method. The experimental results in Table 3 represent the average results of two different test sets.
[0121] The beneficial effects of the present invention are as follows: the present invention provides a method for predicting the risk of Alzheimer's disease based on a CTA framework, which can automatically identify the discriminant position of a specific participant in a whole-brain image, and seamlessly learn high-level feature representations to construct a regression model for estimating clinical cognitive scores. The specific contributions of the present invention are as follows:
[0122] (1) Encoding brain images using 3D convolutional neural networks;
[0123] (2) The use of the Transformer attention module allows adaptive fusion of information between different positions or features, and dynamically adjusts weights according to importance. This means that during encoding of brain images, the attention mechanism can adaptively fuse information from different brain regions to better represent the structural characteristics of the entire brain.
[0124] (3) Multiple cognitive scores are jointly predicted through a fully connected regression layer combined with the total mean square error loss, ultimately improving the prediction accuracy of clinical scores.
[0125] In the description of the present invention, it is necessary to understand that the orientation or positional relationship indicated by the terms "center", "thickness", "upper", "lower", "horizontal", "top", "bottom", "inner", "outer", "radial", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", and "third" are used only for descriptive purposes, and cannot be understood as indicating or implying the relative importance or the number of implicitly specified technical features. Therefore, the features defined by "first", "second", and "third" may explicitly or implicitly include one or more of the features.
Claims
1. A method for predicting the risk of Alzheimer's disease based on the CTA framework, characterized in that: The following steps are involved: S1, input the brain image into the 3D convolutional neural network for feature extraction to obtain a brain image feature map; S2, input the brain image feature map into the Transformer attention module, adaptively adjust the information between different positions of the brain image feature map, and obtain the final attention feature map; S3. The final attention feature map is convolved with the brain image feature map. The convolution result is input into the fully connected regression layer after global pooling. The fully connected regression layer is combined with the total mean square error loss to jointly predict several cognitive scores and obtain the predicted probability of Alzheimer's disease.
2. The method for predicting the risk of Alzheimer's disease based on the CTA framework according to claim 1, characterized in that: In S1, the 3D convolutional neural network includes first to fourth convolutional subnetworks connected in sequence; Among them, the structures of the first to third convolutional sub-networks are the same, including the first convolutional layer, the second convolutional layer and the 2×2×2 maximum pooling layer connected in sequence, and the convolution kernels of the first convolutional layer and the second convolutional layer are both 3×3×3; The fourth convolutional subnetwork includes the third convolutional layer, the fourth convolutional layer and the 1×1×1 global average pooling layer connected in sequence. The convolution kernel of the third convolutional layer is 3×3×3, and the convolution kernel of the fourth convolutional layer is 1×1×1. The outputs of the first to fourth convolutional layers are sequentially connected to the batch normalization layer and the ReLU activation function layer to extract and represent the spatial features of brain images.
3. The method for predicting the risk of Alzheimer's disease based on the CTA framework according to claim 1, characterized in that: In S2, the Transformer attention module includes an encoder, a decoder, a connection layer and a softmax activation function layer connected in sequence; The encoder includes a first multi-head attention layer and a first forward position feedback network layer, and the input end of the first forward position feedback network layer is connected to the output end and the input end of the first multi-head attention layer by normalization and residual connection; The decoder includes a masked multi-head attention layer, a second multi-head attention layer and a second forward position feedback network layer. The input end of the second multi-head attention layer is connected to the output end and input end of the first forward position feedback network layer and the output end and input end of the masked multi-head attention layer by normalization and residual connection. The input end of the second forward position feedback network layer is connected to the output end and input end of the second multi-head attention layer by normalization and residual connection. The input end of the connection layer is connected to the output end and input end of the second forward position feedback network layer by normalization and residual connection.
4. The method for predicting the risk of Alzheimer's disease based on the CTA framework according to claim 3, characterized in that: The masked multi-head attention layer, the first multi-head attention layer and the second multi-head attention layer have the same structure, including three parallel first convolutions, several scaled dot product attention modules and second convolutions, the input end of each scaled dot product attention module is connected to the three parallel first convolutions, and the output end of each scaled dot product attention module is connected to the second convolution.
5. The method for predicting the risk of Alzheimer's disease based on the CTA framework according to claim 4, characterized in that: The S2 comprises the following sub-steps: S21, embedding the brain image feature map into the 3D spatial position information through position encoding to obtain an input feature map; S22, input the input feature map into the first multi-head attention layer to obtain a first attention feature map; S23, inputting the first attention feature map and the input feature map into the first forward position feedback network layer to obtain an attention feature map of the encoder; S24, input the input feature map into the masked multi-head attention layer to obtain a second attention feature map; S25, inputting the second attention feature map, the input feature map, and the attention feature map of the encoder into the second multi-head attention layer to obtain a third attention feature map; S26 inputs the third attention feature map and the second attention feature map into the second forward position feedback network layer to obtain an attention feature map of the decoder; S27, input the attention feature map of the decoder and the third attention feature map into the softmax activation function layer through the connection layer to obtain an attention weight map; S28. Fuse the attention weight map and the brain image feature map to obtain the final attention feature map.
6. The method for predicting the risk of Alzheimer's disease based on the CTA framework according to claim 5, characterized in that: In S21, the position encoding is the same as the five-dimensional shape parameters of the brain image feature map, and the five-dimensional shape parameters include batch size, depth, height, width and number of channels. The expression of the position encoding PE is specifically: PE = sin(pos×div_term) Where sin(·) is the sine function, pos is the position index tensor, and div_term is the scaling factor.
7. The method for predicting the risk of Alzheimer's disease based on the CTA framework according to claim 6, characterized in that: The masked multi-head attention layer, the first multi-head attention layer, and the second multi-head attention layer calculate the output feature map according to the input feature map in the same way, specifically: SA1, maps the input feature map into query, key and value through three parallel first convolutions; SA2. Input the query, key, and value into all scaled dot product attention modules to obtain the calculation results of each scaled dot product attention module. SA3. Concatenate the calculation results of all scaled dot product attention modules through the second convolution to obtain the output feature map.
8. The method for predicting the risk of Alzheimer's disease based on the CTA framework according to claim 7, characterized in that: In SA2, the calculation result head of the i-th scaled dot product attention module is obtained i The specific expression is: head i =Conv3D(Attention(Q,K,V)) Where Conv3D(·) is a 3D convolution operation, Attention(Q i ,K i ,V i ) is the self-attention dot product matrix, and its specific expression is: Where softmax(·) is the softmax activation function, Q is the query, K is the key, V is the value, T is the transposed symbol, and d k is the dimension of the key vector; In SA3, the expression of MultiHead(Q, K, V) after the calculation results of all scaled dot product attention modules are concatenated is as follows: MultiHead(Q,K,V)=Conv3D(concat(head1,...,head h )) In the formula, concat(·) is the concatenation operation, head1 is the calculation result of the first scaled dot product attention module, and head h is the calculation result of the h-th scaled dot product attention module.
9. The method for predicting the risk of Alzheimer's disease based on the CTA framework according to claim 1, characterized in that: In S3, the total mean square error loss L reg The specific expression is: L reg =MSE ADAS11 +MSE CDRSB +MSE MMSE In the formula, MSE ADAS11 is the mean square error of the cognitive score with the score label ADAS11, MSE CDRSB is the mean square error of the cognitive score with the score label CDRSB, MSE MMSE is the mean square error of the cognitive score with the score label MMSE, where the expression for calculating the mean square error MSE of any cognitive score is: Where N is the number of samples, y i is the true score value of the nth sample, is the predicted score value of the nth sample.
Citation Information
Patent Citations
Alzheimer's disease classification prediction method based on visual Transform algorithm
CN113951834A
Hyperspectral image classification method fusing CNN and ViT spatial spectrum features
CN116824220A
Three-dimensional medical image recognition method and apparatus, device, storage medium, and product
US20240046471A1
Cited By
Long-term individual AD risk prediction model and prediction method based on multi-scale deep learning
CN119943388A
