An alzheimer's disease pathological region positioning and classification prediction method

Abstract

The application discloses an Alzheimer's disease pathological region positioning and classification prediction method, comprising the following steps: constructing an Alzheimer's disease classification model; constructing a visual explanation model based on counterfactual reasoning; introducing a three-dimensional coordinate attention mechanism guided by a counterfactual graph to further enhance the classification model; and iterating the explanation model and the Alzheimer's disease classification model. The counterfactual reasoning explanation model divides fine pathological regions by using a counterfactual reasoning method, uses the position information of the generated pathological regions to guide the classification model, and enables the classification model to focus on learning the discriminant region related to the disease and to have higher sensitivity to the pathological region; the three-dimensional coordinate attention mechanism is used to obtain the dependency relationship between three-dimensional image regions and to retain the accurate position information in the three-dimensional space, the model is more likely to obtain the interested region, the whole brain structure is analyzed, and thus the training efficiency and the accuracy of the classification model are improved, and the model has higher robustness.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and visual computing, specifically to a method for locating and classifying pathological regions in Alzheimer's disease. Background Technology

[0002] Alzheimer's disease (AD) is the most common progressive neurodegenerative disease, characterized by progressive impairment of cognitive function that ultimately leads to irreversible neuronal loss. AD causes damage and destruction of nerve cells in brain regions associated with memory, language, and other cognitive functions, and it accounts for 60%–80% of dementia cases worldwide. Brain atrophy associated with AD manifests as a continuous progression from cognitively normal (NC) to mild cognitive impairment (MCI) and across the symptom spectrum of dementia.

[0003] Currently, among various brain imaging techniques, structural magnetic resonance imaging (sMRI) is the most extensively studied for AD diagnosis because it provides imaging biomarkers of neuronal loss in brain anatomy. Specifically, sMRI scans help detect and measure morphological changes in the brain, such as ventricular enlargement and localized atrophy, as well as anatomical differences between subjects. In recent years, with the continuous progress of deep learning research in medical image processing, substantial progress has been made in sMRI-based AD diagnosis, and it is expected to reach clinical application levels. However, the black-box nature of deep learning models limits their practical application in the medical field, especially in terms of fairness, accountability, and transparency. Therefore, the interpretability of model output and the interpretability of the model's internal workings are increasingly needed. From the perspective of end users (doctors, patients, etc.), the ability to interpret at the level of human cognition, as well as the interpretability of deep learning model output, is crucial. However, building predictive models that are both interpretable and high-performance remains an unsolved problem because it is difficult to strike a trade-off between them; that is, the performance of interpretable models is often lower than that of black-box models, especially in the field of medical vision.

[0004] Balancing model performance and interpretability has been a long-standing goal in the field of Explainable Artificial Intelligence (XAI). In the early days of XAI, researchers proposed various methods to discover or identify the regions that have the greatest impact on obtaining classifier results. The main purpose of these XAI methods was to answer the question: "For input X, which part influences the classifier's decision to label it as Y?" However, more recent XAI methods offer a more fundamental explanation in the sense of causality: "If input X were X′, would the result be Z instead of Y?" This explanation is defined as the root of counterfactual reasoning. Counterfactual reasoning provides explanations at the level of human cognition because it explains the model's decisions in hypothetical scenarios.

[0005] Therefore, a deep explanation model for Alzheimer's disease based on counterfactual reasoning can output a higher level of visual explanation. For example, a counterfactual graph conditioned on a target label (i.e., a hypothetical scenario) is generated and added to the input image, causing the transformed image to be diagnosed as the target type. For instance, when a counterfactual graph is added to the sMRI image of a subject with AD, it causes a change in the category of the input sMRI image, thus diagnosing them as a normal subject. During the training phase, the counterfactual graph is used as privileged information, that is, the locational information of the counterfactual graph affecting category determination is used to further optimize the diagnostic model, allowing the diagnostic model to focus on learning and discovering disease-related discriminant regions, thus combining the explanation model with the AD diagnostic model. Simultaneously, to obtain the dependencies between 3D image regions and preserve accurate 3D spatial location information, 3D coordinate attention is introduced into each convolutional layer of the classifier, further enhancing the classifier's generalization ability, thereby improving the overall model performance and interpretability. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method for locating and classifying the pathological regions of Alzheimer's disease, thus solving the problems mentioned in the background section.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for localizing and classifying Alzheimer's disease pathological regions, comprising the following steps:

[0008] S1. Obtain image data of three types of magnetic resonance imaging (MRI) from the ADNI database: Alzheimer's patients, normal subjects, and patients with mild cognitive impairment, and perform data preprocessing on the obtained image samples;

[0009] S2. Normalize the three types of preprocessed sMRI image data to zero mean and unit variance, normalize the quantiles, and downsample to adjust the image size to obtain downsampled image data.

[0010] S3. Using a 3D-Resnet18 network, construct an AD / MCI / NC classification diagnostic model for sMRI images and obtain the classification features of each layer of the image classification.

[0011] S4. Construct a visualization explanation model for multi-path counterfactual reasoning based on the classification model. Extract the corresponding classification features from the pre-trained classification model in S3 as the encoder in the counterfactual graph. Generate the multi-path counterfactual explanation model based on the recurrent generative adversarial network (CycleGAN).

[0012] S5. The counterfactual map generated by the counterfactual map generator is used as a guide for the location information of the pathological region. Based on the S3 classification model, it is further improved by introducing a three-dimensional coordinate attention mechanism to enhance the performance of the classification model.

[0013] S6. Input the features of the classification model trained in S5 into the model in S4, repeat S4 and S5 for iteration, calculate the accuracy of the prediction model after each iteration, and retain the model with the highest accuracy up to the current iteration; after reaching the set number of iterations, output the optimal model.

[0014] Optionally, the sMRI image data preprocessing and resizing method includes the following steps:

[0015] S11. Perform anterior compositing (AC)-posterior compositing (PC) correction on the acquired ADNI image data to obtain the data in the required standard space.

[0016] S12. Using the HD-BET brain dissection tool, dissect the skull and neck tissues and other non-brain tissues to obtain the dissected image data.

[0017] S13. Using the FLIRT tool in FSL software, the dissected image is registered to the standard MNI152 1mm brain template space through FLIRT linear transformation.

[0018] S14. The registered image data is subjected to bias field correction using the N4BiasFieldCorrection function of ANTs software, and finally the image data with image size of 193mm×229mm×193mm is obtained.

[0019] S15. Normalize the corrected data to zero mean and unit variance, and normalize the quantiles (set to 10% and 90%). Then, downsample the image to reduce its size by a factor of 2. The final sMRI image size input into the model is 96mm×114mm×96mm.

[0020] Optionally, the Alzheimer's disease deep explanation model based on counterfactual reasoning can output a higher level of visual explanation, including:

[0021] Based on a pre-trained Alzheimer's disease classification model, a visual explanation model based on counterfactual reasoning is constructed. This enables the counterfactual graph generator to transform image data into data of any target type through counterfactual mapping. For example, when a counterfactual graph is added to the sMRI image of a subject with AD, it will cause the input sMRI image category to change, thereby diagnosing the subject as a normal subject. Based on the CycleGAN framework, a multi-path counterfactual reasoning explanation model for Alzheimer's disease is constructed, which mainly includes three modules: counterfactual graph generator (CMG), outcome evaluator, and discriminator.

[0022] The counterfact graph generator is used to synthesize counterfact graphs (CMs) conditioned on the target label t. x,t It consists of an encoder and generator Composition, encoder and generator The network design is a variant based on U-Net, which combines the target labels with the encoder after tiling. The data features extracted from each layer are concatenated and passed to the generator via skip connections. In the middle, record Indicates encoder The output feature map of the l-th convolutional layer is tiled given the target label t to match its shape. To better extract relevant information from the target label, 3D convolution is used, which has a learnable 3×3×3 convolution kernel, a stride of 1 in each dimension, and zero padding. Then, a non-linear LReLU activation function is applied to obtain the following results:

[0023]

[0024] in, Operators representing channel concatenation, followed by feature mappings. The target label information is passed to the generator by skipping the connection. In the middle, then the generator Capable of responsibly synthesizing corresponding counterfactual graphs (CMs) from target labels and feature maps. x,t The results are shown below:

[0025]

[0026] in Finally, by synthesizing the counterfactual graph CM X,t The transformed image is produced by adding it to the input MRI image X. Right now It will be classified as the target label t by the subsequent result evaluator module, i.e., the pre-trained diagnostic model.

[0027] Optionally, counterfactual maps can be used as guidance for pathological region location information, and two-dimensional coordinate attention can be improved to make it suitable for training 3D networks using three-dimensional sMRI data, including:

[0028] Using counterfactual graphs as guidance enhances the effectiveness of the diagnostic model. Specifically, the target label for the generated input image is T. CN and T AD The counterfactual mapping, i.e., the brain images most normal and most similar to AD for the input sample, is used to construct the guidance map M by combining these two counterfactual maps. Guide This indicates brain regions sensitive to AD pathology:

[0029]

[0030] Where MinMax(*) represents the minimum-maximum normalization performed in voxel mode, therefore, the guiding graph M Guide The absolute value in the text refers to the attention value that is allowed to be used in the most normal brain imaging and the extreme cases most similar to AD brain imaging (because...). Negative values ​​highlight the most normal areas of the brain, while The positive value highlights the brain region most similar to AD. This guidance map modulates the layer's output through a subsequent three-dimensional coordinate attention mechanism, thereby enhancing the layer's output performance in the diagnostic model.

[0031] sMRI images are mapped at each layer of the classification model, using global attention, local attention, and 3D coordinate attention respectively. The 3D coordinate attention performs global average pooling on the three spatial dimensions, transposes the pooled tensors, concatenates them, and then performs 1*1*1 convolution, batch normalization, and activation functions. It is then split into three separate tensors along the spatial dimensions, each multiplied by a 1*1*1 convolution and a sigmoid function. The outputs of global and local attention are multiplied by the output of 3D coordinate attention, multiplied by the initially mapped data using a sigmoid function, and then summed. Finally, a ReLU activation function is applied to obtain the attention-enabled feature map.

[0032] This invention provides a method for localizing and classifying the pathological regions of Alzheimer's disease, which has the following beneficial effects:

[0033] 1. This invention employs a counterfactual reasoning method, constructing a multi-path counterfactual reasoning explanation model based on the CycleGAN framework. During the training phase, the counterfactual graph generated in the explanation model is used as privileged information, that is, the location information of the counterfactual graph affecting the category classification is used to guide the classification model, enabling the model to focus on learning and discovering disease-related discriminative regions. This makes the classification model more sensitive to pathological regions, thereby improving the training efficiency and accuracy of the classification model and making the model more robust.

[0034] 2. This invention introduces an attention mechanism into each layer of the feature extraction process in the classification model. It improves the two-dimensional coordinate attention mechanism to make it suitable for training 3D networks using 3D sMRI data. The three-dimensional coordinate attention mechanism further acquires the dependencies between 3D image regions and retains precise positional information in 3D space, making it easier for the model to acquire regions of interest, expanding the receptive field for whole-brain structure analysis, and effectively improving the accuracy of disease classification. The improved performance of the classification model makes it suitable as the encoder ε in the interpretation model. nThe ability to extract disease-related features from input images is enhanced, making the counterfactual graph generator (CMG) of the explanatory model easier and more robust to train. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the framework of the present invention;

[0036] Figure 2 This is a flowchart of the sMRI image data preprocessing and size adjustment method of the present invention;

[0037] Figure 3 This is a schematic diagram of the classification prediction model of the present invention;

[0038] Figure 4 This is a schematic diagram of the structure of the visual interpretation model of the present invention;

[0039] Figure 5 This is a schematic diagram of the counterfactual generator in the visual interpretation model of this invention;

[0040] Figure 6 This is a schematic diagram of the three-dimensional coordinate attention of the present invention. Detailed Implementation

[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0042] Please see Figures 1 to 6 This invention provides a technical solution: a method for localizing and classifying Alzheimer's disease pathological regions, comprising the following steps:

[0043] S1. Obtain image data of three types of magnetic resonance imaging (MRI) from the ADNI database: Alzheimer's patients, normal subjects, and patients with mild cognitive impairment, and perform data preprocessing on the obtained image samples;

[0044] S2. Normalize the three types of preprocessed sMRI image data to zero mean and unit variance, normalize the quantiles, and downsample to adjust the image size to obtain downsampled image data.

[0045] S3. Using a 3D-Resnet18 network, construct an AD / MCI / NC classification diagnostic model for sMRI images and obtain the classification features of each layer of the image classification.

[0046] S4. Construct a visual explanation model based on counterfactual reasoning for the classification model. Extract the corresponding classification features from the classification model pre-trained in S3 as the encoder in the counterfactual graph. Generate a multi-path counterfactual explanation model based on a recurrent generative adversarial network (CycleGAN).

[0047] S5. The counterfactual map generated by the counterfactual map generator is used as a guide for the location information of the pathological region. Based on the S3 classification model, it is further improved by introducing a three-dimensional coordinate attention mechanism to enhance the performance of the classification model.

[0048] S6. Input the features of the classification model trained in S5 into the model in S4, repeat S4 and S5 for iteration, calculate the accuracy of the prediction model after each iteration, and retain the model with the highest accuracy up to the current iteration; after reaching the set number of iterations, output the optimal model.

[0049] The sMRI image data preprocessing and resizing method includes the following steps:

[0050] S11. Perform anterior compositing (AC)-posterior compositing (PC) correction on the acquired ADNI image data to obtain the data in the required standard space.

[0051] S12. Using the HD-BET brain dissection tool, dissect the skull and neck tissues and other non-brain tissues to obtain the dissected image data.

[0052] S13. Using the FLIRT tool in FSL software, the dissected image is registered to the standard MNI152 1mm brain template space through FLIRT linear transformation.

[0053] S14. The registered image data is subjected to bias field correction using the N4BiasFieldCorrection function of ANTs software, and finally the image data with image size of 193mm×229mm×193mm is obtained.

[0054] S15. Normalize the corrected data to zero mean and unit variance, and normalize the quantiles (set to 10% and 90%). Then, downsample the image to reduce its size by a factor of 2. The final sMRI image size input into the model is 96mm×114mm×96mm.

[0055] The counterfactual reasoning-based deep explanation model for Alzheimer's disease can output higher-level visual explanations, including:

[0056] Based on a pre-trained Alzheimer's disease classification model, a visual explanation model based on counterfactual reasoning is constructed. This enables the counterfactual graph generator to transform image data into data of any target type through counterfactual mapping. For example, when a counterfactual graph is added to the sMRI image of a subject with AD, it will cause the input sMRI image category to change, thereby diagnosing the subject as a normal subject. Based on the CycleGAN framework, a multi-path counterfactual reasoning explanation model for Alzheimer's disease is constructed, which mainly includes three modules: counterfactual graph generator (CMG), outcome evaluator, and discriminator.

[0057] The counterfact graph generator is used to synthesize counterfact graphs (CMs) conditioned on the target label t. X,t It consists of an encoder and generator Composition, encoder and generator The network design is a variant based on U-Net, which combines the target labels with the encoder after tiling. The data features extracted from each layer are concatenated and passed to the generator via skip connections. In the middle, record Indicates encoder The output feature map of the l-th convolutional layer is tiled given the target label t to match its shape. To better extract relevant information from the target label, 3D convolution is used, which has a learnable 3×3×3 convolution kernel, a stride of 1 in each dimension, and zero padding. Then, a non-linear LReLU activation function is applied to obtain the following results:

[0058]

[0059] in, Operators representing channel concatenation, followed by feature mappings. The target label information is passed to the generator by skipping the connection. In the middle, then the generator Capable of responsibly synthesizing corresponding counterfactual graphs (CMs) from target labels and feature maps. X,t The results are shown below:

[0060]

[0061] in Finally, by synthesizing the counterfactual graph CM X,t The transformed image is produced by adding it to the input MRI image X. Right now It will be classified as the target label t by the subsequent result evaluator module, i.e., the pre-trained diagnostic model.

[0062] Using counterfactual maps as guidance for pathological region location information, the 2D coordinate attention method is improved to make it suitable for training 3D networks using 3D sMRI data, including:

[0063] Using counterfactual graphs as guidance enhances the effectiveness of the diagnostic model. Specifically, the target label for the generated input image is T. CN and T AD The counterfactual mapping, i.e., the brain images most normal and most similar to AD for the input sample, is used to construct the guidance map M by combining these two counterfactual maps. Guide This indicates brain regions sensitive to AD pathology:

[0064]

[0065] Where MinMax(*) represents the minimum-maximum normalization performed in voxel mode, therefore, the guiding graph M Guide The absolute value in the text refers to the attention value that is allowed to be used in the most normal brain imaging and the extreme cases most similar to AD brain imaging (because...). Negative values ​​highlight the most normal areas of the brain, while The positive value highlights the brain region most similar to AD. This guidance map modulates the layer's output through a subsequent three-dimensional coordinate attention mechanism, thereby enhancing the layer's output performance in the diagnostic model.

[0066] sMRI images are mapped at each layer of the classification model, using global attention, local attention, and 3D coordinate attention respectively. The 3D coordinate attention performs global average pooling on the three spatial dimensions, transposes the pooled tensors, concatenates them, and then performs 1*1*1 convolution, batch normalization, and activation functions. It is then split into three separate tensors along the spatial dimensions, each multiplied by a 1*1*1 convolution and a sigmoid function. The outputs of global and local attention are multiplied by the output of 3D coordinate attention, multiplied by the initially mapped data using a sigmoid function, and then summed. Finally, a ReLU activation function is applied to obtain the attention-enabled feature map.

[0067] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for localizing and classifying the pathological regions of Alzheimer's disease, characterized in that: Includes the following steps: S1. Obtain image data of three types of magnetic resonance imaging (MRI) from the ADNI database: Alzheimer's patients, normal subjects, and patients with mild cognitive impairment, and perform data preprocessing on the obtained image samples; S2. Normalize the three types of preprocessed sMRI image data to zero mean and unit variance, normalize the quantiles, and downsample to adjust the image size to obtain downsampled image data. S3. Using a 3D-Resnet18 network, construct an AD / MCI / NC classification diagnostic model for sMRI images and obtain the classification features of each layer of the image classification. S4. Construct a visualization explanation model for multi-path counterfactual reasoning based on a classification model. Extract the corresponding classification features from the pre-trained classification model in S3 as the encoder in the counterfactual graph. Generate the multi-path counterfactual explanation model based on a recurrent generative adversarial network. S5. The counterfactual map generated by the counterfactual map generator is used as a guide for the location information of the pathological region. Based on the S3 classification model, it is further improved by introducing a three-dimensional coordinate attention mechanism to enhance the performance of the classification model. S6. Input the classification model features trained in S5 into the model in S4, repeat S4 and S5 for iteration, calculate the classification accuracy of the prediction model after each iteration, and retain the model with the highest accuracy up to the current iteration; after reaching the set number of iterations, output the optimal model. The counterfactual reasoning-based deep explanation model for Alzheimer's disease can output higher-level visual explanations, including: Based on a pre-trained Alzheimer's disease classification model, a visual explanation model based on counterfactual reasoning is constructed. This enables the counterfactual graph generator to transform image data into data of any target type through counterfactual mapping. When the counterfactual graph is added to the sMRI image of a subject with AD, it causes the input sMRI image category to change, thereby diagnosing the subject as a normal subject. Based on a recurrent generative adversarial network framework, a multi-path counterfactual reasoning explanation model for Alzheimer's disease is constructed, which mainly includes three modules: counterfactual graph generator, outcome evaluator, and discriminator. Counterfact graph generator is used to synthesize target labels Counterfactual diagram with conditions It consists of an encoder and generator Composition, encoder and generator The network design is a variant based on U-Net, which combines the target labels with the encoder after tiling. The data features extracted from each layer are concatenated and passed to the generator via skip connections. In the middle, record Indicates encoder The Middle The output feature maps of each convolutional layer, given the target label. The model is tiled to match the shape of the feature map. To better extract relevant information from the target label, 3D convolution is employed, which has learnable characteristics. The convolution kernel and stride in each dimension are set to 1 with zero padding, and then the non-linear LReLU activation function is applied to obtain the following results: in, Operators representing channel concatenation, followed by feature mappings. = The target label information is passed to the generator by skipping the connection. In the middle, then the generator Able to responsibly synthesize corresponding counterfactual maps from target labels and feature maps. The results are shown below: in Finally, by synthesizing the counterfactual graph... With input MRI images The transformation is produced by addition. ,Right now It will be classified as the target label t by the subsequent result evaluator module, i.e., the pre-trained diagnostic model.

2. The method for localizing and classifying Alzheimer's disease pathological regions according to claim 1, characterized in that: The sMRI image data preprocessing and resizing method includes the following steps: S11. Perform anterior-posterior merging and posterior-merging correction on the acquired ADNI image data to obtain the data in the required standard space. S12. Using the HD-BET brain dissection tool, the skull and neck tissues are dissected to obtain image data after dissection. S13. Using the FLIRT tool in FSL software, the dissected image is registered to the standard MNI152 1mm brain template space through FLIRT linear transformation. S14. Perform bias field correction on the registered image data using the N4BiasFieldCorrection function in the ANTs software, finally obtaining an image with the following dimensions: Image data; S15. The corrected data is normalized to zero mean and unit variance, and quantile normalization is set to 10% and 90% respectively. The image is then downsampled to reduce its size by a factor of 2. The final sMRI image size input into the model is [size missing]. .

3. The method for localizing and classifying Alzheimer's disease pathological regions according to claim 1, characterized in that: Using counterfactual maps as guidance for pathological region location information, the 2D coordinate attention method is improved to make it suitable for training 3D networks using 3D sMRI data, including: Using counterfactual graphs as guidance enhances the effectiveness of the diagnostic model, generating target labels for input images as follows: and The counterfactual mapping, i.e., the brain images most normal and most similar to AD for the input sample, is used to construct the guidance map by combining these two counterfactual maps. This indicates brain regions sensitive to AD pathology: in This represents the minimum-maximum normalization performed in voxel fashion; therefore, the guiding diagram... The absolute value in the figure refers to the attention value that is allowed to be used in the extreme cases of the most normal brain images and the most similar AD brain images. This guideline modulates the output of the layer through the subsequent three-dimensional coordinate attention mechanism, thereby enhancing the performance of the layer's output in the diagnostic model. sMRI images are mapped at each classification model layer using global attention, local attention, and 3D coordinate attention. The 3D coordinate attention performs global average pooling across the three spatial dimensions, transposes the pooled tensors, and concatenates them. Then, the images are processed... The convolution, batch normalization, and activation functions are applied, and then the tensor is split into three separate tensors along the spatial dimension, and each is processed separately. The convolution and sigmoid functions are multiplied together, and the outputs of global attention and local attention are multiplied with the output of 3D coordinate attention. After passing through the sigmoid function, the data is multiplied with the initially mapped data and then added together. Finally, the ReLU activation function is used to obtain the attention-enabled feature map.

Images