Early Alzheimer's disease classification method based on multi-view comparative learning

Abstract

The invention relates to the technical field of medical image classification, in particular to an early Alzheimer's disease classification method based on multi-view comparative learning. The method comprises the following steps: preprocessing three-dimensional MRI scanning data to obtain images of different views, and performing data enhancement on the images of different views to construct a positive sample pair data set; pre-training the feature extraction network containing the two branches by using the positive sample pair data set, and calculating the comparison loss of the output of the two branches; taking an image encoder in a second branch of the pre-trained feature extraction network as a feature extraction module of a logistic regression model, and performing downstream task fine tuning on the logistic regression model by using the test set to obtain an early Alzheimer's disease classification model; and preprocessing target three-dimensional MRI scanning data, inputting the preprocessed target three-dimensional MRI scanning data into the early Alzheimer's disease classification model, and outputting a classification result of the early Alzheimer's disease.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image classification, and in particular to an early Alzheimer's disease classification method based on multi-view contrast learning. Background Art

[0002] Alzheimer's disease (AD) is a chronic degenerative brain disease. Mild cognitive impairment (MCI) is the early stage of AD, characterized by less severe cognitive and functional impairments, but 10%-15% of MCI patients progress to AD each year. Early Alzheimer's disease detection aims to use computers combined with brain imaging to identify cognitive dysfunction at an early stage, so that relevant measures can be taken as early as possible.

[0003] Deep learning methods have been widely used in medical image analysis for neurodegenerative diseases. Such methods can learn complex feature representations in medical image data and automatically extract effective features. Compared with traditional machine learning techniques, they reduce manual feature extraction operations and improve detection efficiency. Using deep learning methods to analyze magnetic resonance imaging (MRI) can be used for early detection of AD, which can improve doctors' detection efficiency and accuracy. However, the excellent performance of deep neural networks is largely due to the precise manual annotation of large amounts of raw data, which is very time-consuming, labor-intensive and expensive to obtain, and is therefore impractical in real-world deployment. The difficulty in obtaining detailed expert annotations for MRI data has seriously hindered the scalability and availability of early AD detection methods. In addition, in the current early Alzheimer's disease detection research, most of the contrastive learning methods use a single view, ignoring the potential connections between different views of MRI images, resulting in a lack of rich representation information for complex brain structures.

[0004] In summary, in the current Alzheimer's disease classification research, most deep learning methods are supervised, that is, they rely on a large amount of labeled data, but it is difficult to obtain large-scale medical imaging data with precise expert annotations, so it has certain limitations. In addition, the method based on self-supervised contrastive learning also has the problem of lack of correlation between multi-view visual information. Summary of the invention

[0005] In order to better solve the problem in the prior art that it is difficult to obtain a large amount of labeled data and lack of correlation between multi-view visual information, resulting in low efficiency in early Alzheimer's disease detection, the present invention provides an early Alzheimer's disease classification method based on multi-view contrastive learning, which converts unlabeled MRI data into different views through data processing, pre-trains the model through contrastive learning, and fine-tunes the downstream tasks to improve the overall performance of the model, thereby improving the efficiency and accuracy of early Alzheimer's disease detection.

[0006] In a first aspect, the present invention provides an early Alzheimer's disease classification method based on multi-view contrastive learning, comprising:

[0007] Step 1: Collect and preprocess three-dimensional MRI scan data to construct a multi-view dataset; divide the multi-view dataset into a training set and a test set;

[0008] Step 2: Perform data enhancement on the training set to generate an enhanced data set to construct a positive sample pair data set; wherein the positive sample pair data set includes the enhanced data set and the training set;

[0009] Step 3: pre-training a feature extraction network using the positive sample data set; the feature extraction network includes a first branch and a second branch; each branch includes an image encoder and an ACMF module;

[0010] The pre-training process is: inputting the enhanced data set and the training set into the first branch and the second branch respectively for training, and calculating the contrast loss of the outputs of the two branches;

[0011] Step 4: using the image encoder in the second branch of the pre-trained feature extraction network as a feature extraction module of a preset logistic regression model, and using the test set to fine-tune the logistic regression model for downstream tasks to obtain an early Alzheimer's disease classification model;

[0012] Step 5: Pre-process the target three-dimensional MRI scan data and input it into the early Alzheimer's disease classification model to output the classification result of early Alzheimer's disease.

[0013] Furthermore, the preprocessing includes performing registration, skull stripping, denoising, cropping and slicing operations on the original three-dimensional MRI data to obtain a multi-view data set.

[0014] Further, the multi-view dataset includes coronal plane, cross-sectional plane and sagittal plane having the same pathological mechanism.

[0015] Furthermore, the data enhancement includes random aspect ratio cropping, random horizontal inversion, and modifying brightness, contrast or saturation with a probability of 0.8.

[0016] Furthermore, in step 3, a contrast loss function is used to calculate the contrast loss of the outputs of the two branches, and the contrast loss function is:

[0017]

[0018] Among them, L con represents the contrast loss function, sim() represents the regularized dot product, τ represents the temperature parameter, zi and z j Respectively represent the features of the current two branch outputs; (z i ,z j ) represents the positive sample pair feature; (z i ,z k ) represents the negative sample pair feature, z k represents the negative sample feature, which is the output of the feature extraction network branch divided by z i and z j The remaining features of; N represents the number of samples in the training set.

[0019] The ACFM module includes a first attention module, a second attention module and a splicing module;

[0020] The ACMF module is used to perform feature fusion using the attention mechanism. The specific process includes:

[0021] Pairing the coronal plane feature vector, the cross-sectional feature vector and the sagittal plane feature vector obtained by the image encoder and inputting them into the first attention module and the second attention module respectively;

[0022] Calculating the first attention vector of the coronal plane feature vector and the cross-sectional feature vector through the first attention module and fusing the first attention vector with the coronal plane feature vector to obtain a first feature vector;

[0023]

[0024] Among them, softmax() represents the softmax function, Y c represents the coronal plane eigenvector, represents the first attention vector, Y a represents the cross-sectional eigenvector, Y c ′ represents the first eigenvector, d k represents the dimension of the embedding layer of the attention module;

[0025] Calculating the second attention vector of the cross-sectional feature vector and the sagittal feature vector through the second attention module and fusing the second attention vector with the sagittal feature vector to obtain a second feature vector;

[0026]

[0027] Among them, Y s represents the sagittal plane eigenvector, represents the second attention vector, Y s ′ represents the second eigenvector;

[0028] The first feature vector, the cross-section feature vector and the second feature vector are dimensionally spliced ​​by a splicing module to obtain a third feature vector;

[0029] y t ′=concatenate(Y c ′,Y a ,Y s ′) (7)

[0030] Among them, concatenate() represents the concatenation operation, y t ′ represents the third eigenvector.

[0031] Furthermore, each branch of the feature extraction network also includes a multilayer perceptron; the multilayer perceptron is used to further process and analyze the output of the ACMF module.

[0032] Furthermore, the image encoder adopts a ResNet50 network structure.

[0033] Furthermore, the pre-training further includes extracting the output of the intermediate layer of the image encoder respectively and calculating a compact loss function; wherein the output of the intermediate layer of the image encoder is the output of the third staggered block in the ResNet50 network, and the compact loss function is calculated by the following formula:

[0034]

[0035] Among them, C ii Denotes the diagonal elements of the cross-correlation matrix, C ij represents the off-diagonal elements of the cross-correlation matrix, λ represents the manually set parameter, and L barlow represents the compact loss function,

[0036]

[0037] Among them, b means that there are b positive sample pairs in the current training batch. represents the output of the middle layer of the image encoder corresponding to the i-th sample of the first input branch, Represents the output of the middle layer of the image encoder corresponding to the j-th sample of the second input branch.

[0038] Beneficial effects of the present invention:

[0039] The present invention converts unlabeled MRI data into different views through data processing, constructs an image encoder to extract implicit representation information in image data of different views, and pre-trains the image encoder through contrastive learning. Finally, it combines logical regression to fine-tune downstream tasks to improve the overall performance of the model and enhance the efficiency and accuracy of early Alzheimer's disease detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a flow chart of an early Alzheimer's disease classification method based on multi-view contrastive learning provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of a three-dimensional MRI scan data preprocessing process provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of a feature extraction network and pre-training process provided by an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of the ACMF module structure and data processing process provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Taking the data from the Alzheimer's Disease Neuroimaging Project as an example, the study found that three-dimensional MRI scans contain a large amount of visual information, such as brain structure, gray matter, and white matter atrophy. This information is related to the detection of Alzheimer's disease. Therefore, in order to represent this visual information, a multi-view contrastive learning-based early Alzheimer's disease classification method was proposed based on the contrastive learning method.

[0046] like Figure 1 As shown, an embodiment of the present invention provides an early Alzheimer's disease classification method based on multi-view contrast learning, comprising:

[0047] Step 1: Collect and preprocess 3D MRI scan data to construct a multi-view dataset; divide the multi-view dataset into a training set and a test set;

[0048] Specifically, Figure 2 As shown, the preprocessing includes registration, skull stripping, denoising, cropping and slicing of the original three-dimensional MRI data to obtain a multi-view dataset.

[0049] Specifically, the multi-view dataset includes coronal, cross-sectional, and sagittal planes with the same pathological mechanism.

[0050] Step 2: Perform data augmentation on the training set to generate an augmented data set to construct a positive sample pair data set; the positive sample pair data set includes an augmented data set and a training set; the training data and augmented data in each positive sample pair data set correspond one to one.

[0051] Specifically, data augmentation includes random aspect ratio cropping, random horizontal inversion, and modifying brightness, contrast, or saturation with a probability of 0.8.

[0052] It should be noted that the data enhancement performed in the embodiments of the present invention may be one or more of the above data enhancements.

[0053] Step 3: Pre-train the feature extraction network using the positive sample data set; the feature extraction network includes a first branch and a second branch; each branch includes an image encoder and an ACMF module;

[0054] Specifically, Figure 3 As shown in FIG, the pre-training process is as follows: the enhanced data set and the training set are respectively input into the first branch and the second branch for training, and the contrast loss is calculated for the outputs of the two branches.

[0055] It can be understood that the enhanced data set is input into the first branch of the feature extraction network, and the training set is input into the second branch of the feature extraction network for training.

[0056] Specifically, the contrast loss function is used to calculate the contrast loss of the outputs of the two branches. The contrast loss function is:

[0057]

[0058] Among them, L con represents the contrast loss function, sim() represents the regularized dot product, τ represents the temperature parameter, z i and z j Respectively represent the features of the current two branch outputs; (z i ,z j ) represents the positive sample pair feature; (z i ,z k ) represents the negative sample pair feature, z k Represents the negative sample feature, which is the output of the feature extraction network branch divided by z i and z j The remaining features of; N represents the number of samples in the training set.

[0059] It can be understood that the contrast loss between the output features of the two branches is calculated by the contrast loss function to adjust the model parameters of the feature extraction network.

[0060] Step 4: Use the image encoder in the second branch of the pre-trained feature extraction network as the feature extraction module of the preset logistic regression model, and use the test set to fine-tune the logistic regression model for downstream tasks to obtain an early Alzheimer's disease classification model;

[0061] It can be understood that the logistic regression model is a commonly used classification model in the field, and will not be elaborated again. The embodiment of the present invention uses the logistic regression model to classify the early Alzheimer's disease classification results into AD, MCI and health. Downstream task fine-tuning refers to adjusting the model parameters subsequent to the feature extraction module.

[0062] Step 5: Preprocess the target three-dimensional MRI scan data and input it into the early Alzheimer's disease classification model to output the classification results of early Alzheimer's disease.

[0063] The method proposed in the embodiment of the present invention converts unlabeled MRI data into different views through data processing, constructs an image encoder to extract implicit representation information in image data of different views, and pre-trains the image encoder through contrastive learning. Finally, it combines logical regression to fine-tune downstream tasks to improve the overall performance of the model and improve the efficiency and accuracy of early Alzheimer's disease detection.

[0064] In one possible embodiment, Figure 4 As shown, the ACMF module specifically includes a first attention module, a second attention module and a splicing module; wherein the attention module calculates the attention vector of different view feature vectors through Att(Q, K, V),

[0065]

[0066] Among them, Q represents the query vector, K represents the key vector, V represents the value vector, and d k Represents the dimension of the embedding layer of the attention module.

[0067] The specific process of ACFM module processing data includes:

[0068] Pair the coronal feature vector, the cross-sectional feature vector and the sagittal feature vector obtained by the feature extraction module and input them into the first attention module and the second attention module respectively;

[0069] Specifically, the coronal plane feature vector Y is obtained through the feature extraction module c , cross-sectional characteristic vector Y a and the sagittal eigenvector Y s , the cross section as the main view of the detection will be used as the query Q when calculating the attention vector, and the coronal and sagittal planes are paired with the cross section respectively, that is, <Y a ,Y c > and <Ya ,Y s >, thereby performing attention calculations between different views.

[0070] Calculate the coronal plane feature vector Y through the first attention module c and the cross-sectional characteristic vector Y a The first attention vector and the coronal plane eigenvector Y c Fusion is performed to obtain the first eigenvector Y c ′.

[0071] Specifically, the cross-sectional feature vector Y a As query Q, the coronal feature vector Y c As key K and value V, calculate the first attention vector

[0072]

[0073] Among them, softmax() represents the softmax function.

[0074] Through the jump connection structure, the coronal plane feature vector Y c and the first attention vector Fusion is performed to generate the first eigenvector Y c′ , which preserves both specific basic information and high-level semantic information:

[0075]

[0076] Calculate the cross-sectional feature vector Y through the second attention module a and the sagittal eigenvector Y s The second attention vector and the sagittal eigenvector Y s Fusion is performed to obtain the second eigenvector Y s ′.

[0077] Specifically, the cross-sectional feature vector Y a As query Q, the sagittal feature vector Y s As key K and value V, calculate the second attention vector

[0078]

[0079] Through the jump connection structure, the sagittal plane feature vector Y s and the second attention vector Fusion is performed to generate the second feature vector Y s ′, which retains both specific basic information and high-level semantic information:

[0080]

[0081] Through the concatenation module, the first feature vector Y c ′, cross-sectional characteristic vector Y a and the second eigenvector Y s ′ is concatenated in dimension to obtain the third eigenvector y t ′, which is the output of the ACMF module.

[0082] y t ′=concatenate(Y c ′,Y a ,Y s ′) (7)

[0083] Among them, concatenate() represents the concatenation operation.

[0084] In one possible embodiment, each branch of the image encoder further includes a multi-layer perceptron.

[0085] Specifically, the multilayer perceptron has powerful nonlinear mapping capabilities and can further process and analyze the output of the ACMF module.

[0086] In one possible embodiment, the image encoder adopts a ResNet50 network structure.

[0087] It is understood that the image encoder is used to extract features from the image, and other network structures with feature extraction can be used, which will not be elaborated in detail here. The embodiment of the present invention uses the ResNet50 network structure as an image encoder as a preferred method.

[0088] In one possible embodiment, Figure 3 As shown in the figure, the pre-training also includes extracting the output of the middle layer of the feature extraction module and calculating the compact loss function; the output of the middle layer of the feature extraction module is the output of the third staggered block in the ResNet50 network, and the compact loss function is calculated by the following formula:

[0089]

[0090] Among them, L barlow represents the compact loss function, C ii Denotes the diagonal elements of the cross-correlation matrix, C ij represents the off-diagonal elements of the cross-correlation matrix, λ represents an artificially set parameter, and in this embodiment, λ is set to 5·10 -3The first term is the "invariant" term, which attempts to equate the diagonal elements of the cross-correlation matrix to 1, forcing the model to learn an invariant representation corresponding to the image. The second term is called the "redundancy reduction" term, which attempts to decorrelate the different vector components represented by the off-diagonal elements. C is the cross-correlation matrix between the outputs of two identical networks along the batch dimension, calculated by the following formula:

[0091]

[0092] Among them, b means that there are b positive sample pairs in the current training batch. represents the output of the middle layer of the image encoder corresponding to the i-th sample of the first input branch, Represents the output of the middle layer of the image encoder corresponding to the j-th sample of the second input branch.

[0093] It can be understood that in this embodiment, the training data set is divided into different batches for training, and the number of data samples in each training is the same.

[0094] To verify the effectiveness of the method proposed in this invention, the early Alzheimer's disease diagnosis task was converted into a three-classification task (AD, MCI, and healthy) for training and evaluation. Single-view data and processed multi-view data were used for pre-training on the method proposed in this invention, and the pre-trained encoder was combined with logistic regression to fine-tune the downstream task. The experiment used accuracy, precision, recall, and F1 value as evaluation indicators. The experimental results are shown in Table 1. The experimental results show that the method of this invention is effective, and all evaluation indicators have been significantly improved.

[0095] Table 1 Experimental results

[0096]

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for early Alzheimer's disease classification based on multi-view contrastive learning, characterized in that: include: Step 1: Collect and preprocess 3D MRI scan data to construct a multi-view dataset; Dividing the multi-view dataset into a training set and a test set; Step 2: Perform data enhancement on the training set to generate an enhanced data set to construct a positive sample pair data set; wherein the positive sample pair data set includes the enhanced data set and the training set; Step 3: Pre-training the feature extraction network using the positive sample data set; The feature extraction network includes a first branch and a second branch; Each branch includes an image encoder and an ACMF module; The pre-training process includes: inputting the enhanced data set and the training set into the first branch and the second branch for training respectively, and calculating the contrast loss of the outputs of the two branches; Step 4: using the image encoder in the second branch of the pre-trained feature extraction network as a feature extraction module of the logistic regression model, and using the test set to fine-tune the logistic regression model for downstream tasks to obtain an early Alzheimer's disease classification model; Step 5: Pre-process the target three-dimensional MRI scan data and input it into the early Alzheimer's disease classification model to output the classification result of early Alzheimer's disease.

2. The early Alzheimer's disease classification method based on multi-view contrastive learning according to claim 1, characterized in that: The preprocessing includes performing registration, skull stripping, denoising, cropping and slicing operations on the original three-dimensional MRI data to obtain a multi-view data set.

3. The early Alzheimer's disease classification method based on multi-view contrastive learning according to claim 1, characterized in that: The multi-view dataset includes coronal, cross-sectional, and sagittal planes with the same pathological mechanism.

4. The early Alzheimer's disease classification method based on multi-view contrastive learning according to claim 1, characterized in that: The data augmentation includes random aspect ratio cropping, random horizontal inversion, and modifying brightness, contrast or saturation with a probability of 0.

8.

5. The early Alzheimer's disease classification method based on multi-view contrastive learning according to claim 1, characterized in that: In step 3, the contrast loss function is used to calculate the contrast loss of the outputs of the two branches. The contrast loss function is: Among them, L con represents the contrast loss function, sim() represents the regularized dot product, τ represents the temperature parameter, z i and z j Respectively represent the features of the current two branch outputs; (z i ,z j ) represents the positive sample pair feature; (z i ,z k ) represents the negative sample pair feature, z k represents the negative sample feature, which is the output of the feature extraction network branch divided by z i and z j The remaining features of; N represents the number of samples in the training set.

6. The early Alzheimer's disease classification method based on multi-view contrastive learning according to claim 3, characterized in that: The ACFM module includes a first attention module, a second attention module and a splicing module; The ACMF module is used to perform feature fusion using the attention mechanism. The specific process includes: Pairing the coronal plane feature vector, the cross-sectional feature vector and the sagittal plane feature vector obtained by the image encoder and inputting them into the first attention module and the second attention module respectively; Calculating the first attention vector of the coronal plane feature vector and the cross-sectional feature vector through the first attention module and fusing the first attention vector with the coronal plane feature vector to obtain a first feature vector; Among them, softmax() represents the softmax function, Y c represents the coronal plane eigenvector, represents the first attention vector, Y a represents the cross-sectional eigenvector, Y c ′ represents the first eigenvector, d k represents the dimension of the embedding layer of the attention module; Calculating the second attention vector of the cross-sectional feature vector and the sagittal feature vector through the second attention module and fusing the second attention vector with the sagittal feature vector to obtain a second feature vector; Among them, Y s represents the sagittal plane eigenvector, represents the second attention vector, Y s ′ represents the second eigenvector; The first feature vector, the cross-section feature vector and the second feature vector are dimensionally spliced ​​by a splicing module to obtain a third feature vector; and t ′=concatenate(AND c ',AND a ,AND s ′) (7) Among them, concatenate() represents the concatenation operation, y t ′ represents the third eigenvector.

7. The early Alzheimer's disease classification method based on multi-view contrastive learning according to claim 1, characterized in that: Each branch of the feature extraction network also includes a multilayer perceptron; the multilayer perceptron is used to further process and analyze the output of the ACMF module.

8. The early Alzheimer's disease classification method based on multi-view contrastive learning according to claim 1, characterized in that: The image encoder adopts the ResNet50 network structure.

9. The method for classifying early Alzheimer's disease based on multi-view contrastive learning according to claim 8, characterized in that: The pre-training further includes extracting the output of the intermediate layer of the image encoder respectively and calculating a compact loss function; wherein the output of the intermediate layer of the image encoder is the output of the third staggered block in the ResNet50 network, and the compact loss function is calculated by the following formula: Among them, C ii Denotes the diagonal elements of the cross-correlation matrix, C ij represents the off-diagonal elements of the cross-correlation matrix, λ represents the manually set parameter, and L barlow represents the compact loss function, Among them, b means that there are b positive sample pairs in the current training batch. represents the output of the middle layer of the image encoder corresponding to the i-th sample of the first input branch, Represents the output of the middle layer of the image encoder corresponding to the j-th sample of the second input branch.

Images