A dementia classification method based on knowledge-guided image diffusion reasoning

Abstract

This invention relates to the field of multimodal data fusion and neurodegenerative disease classification, specifically a dementia classification method based on knowledge-guided image diffusion reasoning. First, initial features are extracted from magnetic resonance imaging (MRI) data using a 3D neural network, and a conditional diffusion model is used to generate category-enhanced image features. Simultaneously, a dementia knowledge graph is constructed from clinical metadata to integrate domain prior knowledge. Then, the enhanced image features and the knowledge graph embedding representation are input into a multimodal learning framework. This framework aligns and fuses information from the two modalities through bimodal consistency learning and modality-specific learning achieved through contrastive learning. Finally, the fused features are fed into a classifier for dementia classification. This invention provides a novel knowledge-guided deep learning framework for dementia classification, effectively overcoming the challenges of limited data and poor interpretability faced by traditional models, and significantly improving the accuracy and robustness of classification.

Description

Technical Field

[0001] This invention relates to the fields of multimodal data fusion and neurodegenerative disease classification, and in particular to a dementia classification method based on knowledge-guided image diffusion reasoning. Background Technology

[0002] For the classification of Alzheimer's Disease (AD), existing classification techniques mainly rely on single-modality detection methods, such as: imaging methods using magnetic resonance imaging (MRI) or positron emission tomography (PET) to detect changes in brain structure and metabolic levels, but their sensitivity and specificity are limited; clinical scales, such as the Montreal Cognitive Assessment (MoCA) and Mini-Mental State Examination (MMSE), can reflect cognitive function status, but they are highly subjective; molecular and genetic data, such as apolipoprotein E (APOE) genotype and cerebrospinal fluid (CSF) biomarkers, can reveal potential risks, but they are costly to obtain and the data is incomplete.

[0003] To overcome these limitations, multimodal learning methods have been increasingly applied to AD classification tasks in recent years. By fusing magnetic resonance imaging (MRI), genetic information, clinical scores, and molecular omics data, models can improve classification accuracy and characterization of test samples. However, these methods still face the following challenges: large-scale acquisition of clinical multimodal data is difficult, and there are distributional differences between different modalities; relying solely on raw images and phenotypic features makes it difficult to capture causal relationships in the pathological process.

[0004] In recent years, knowledge-guided deep learning frameworks have gradually gained attention. By introducing disease-specific knowledge graphs, they combine biological prior information with data-driven models, demonstrating significant advantages in alleviating data scarcity and improving model interpretability. Meanwhile, diffusion models, as an emerging generative approach, have shown superior capabilities in medical image enhancement and feature extraction, generating category-specific structural features that help identify subtle early pathological changes.

[0005] Based on the above situation, this invention proposes a multimodal learning method that combines knowledge graphs and diffusion generation mechanisms to address the shortcomings of existing technologies in dementia classification. Summary of the Invention

[0006] This invention aims to address the shortcomings of existing Alzheimer's Disease (AD) classification methods, such as insufficient classification accuracy, poor model generalization ability, and lack of utilization of clinical knowledge. To this end, this invention proposes a Knowledge-Guided Imaging Diffusion Inference (KIDI) method. This method combines clinical knowledge graphs with 3D MRI image features and introduces a conditional diffusion model to generate category-specific features, thereby improving the early classification ability of AD and dementia.

[0007] This invention is achieved through the following technical solution: This invention provides a dementia classification method based on knowledge-guided and imaging diffusion reasoning, the method comprising the following steps: Step 1: Acquire three-dimensional magnetic resonance imaging data, and generate high-dimensional representation features of the three-dimensional image through a feature extraction network and a conditional denoising diffusion model; Step 2: Obtain clinical metadata corresponding to the 3D magnetic resonance imaging data, construct a dementia knowledge graph containing entities, relationships and attributes, and generate its knowledge graph embedding representation; Step 3: Input the image features obtained in Step 1 and the knowledge graph obtained in Step 2 into a multimodal learning framework. Perform feature fusion through bimodal consistency learning and modality-specific learning, and use a classifier to process the fused features to output the final dementia classification result.

[0008] Furthermore, in step one, the specific training process for the conditional denoising diffusion model is as follows: (1) Forward diffusion process: Gaussian noise is gradually added to the initial eigenvector; the initial eigenvector... Gradually introduce random Gaussian noise At any time step According to preset noise scheduling parameters ,Will With noise A linear combination based on the corresponding weights is used to obtain the time step. Noisy features ,in Controlling the original features and noise components in The proportion of noisy features at any time step t. The calculation formula is as follows: ; (2) Model training: A U-Net network is trained to learn to predict and remove noise given a class label. The added noise is predicted based on the noisy features, time step and class label. The loss function of the diffusion model is defined by minimizing the mean square error between the predicted noise and the real noise. (3) Backsampling process: Starting from a random Gaussian noise vector, new feature vectors under different class conditions are generated through iterative denoising; the backsampling process is as follows: taking the random Gaussian noise vector as the initial input, at each time step t, based on the current noisy features Time step index and conditional tags The noise estimate is output by the denoising network. Subsequently, based on the noise scheduling parameters and its cumulative form For the current features Weighted corrections are applied to the predicted noise to calculate the mean vector for the corresponding time step. Based on this, a random vector z following a Gaussian distribution is introduced, and the noise intensity parameter is used... After scaling it and Add them together to generate the feature vector of the previous time step. .

[0009] The denoising update formula for each step is: ; ; Furthermore, in step two, the knowledge graph is represented as a directed graph. The graph contains entity nodes, relational edges connecting these nodes, and each relational edge has a corresponding set of attributes used to describe the specific information of this relation.

[0010] Furthermore, in step three, bimodal consistency learning is achieved through a contrastive learning strategy, for each batch of samples... The model calculates its correlation with the corresponding positive sample. Similarity with other samples in the same batch The similarity was normalized for comparison. This was done using temperature parameters. Adjust the similarity scale and input the results into the cross-entropy contrastive loss function. The model is trained to maximize the matching of positive samples from the same modality and different modalities, while minimizing the similarity with other negative samples, thereby aligning the feature representations of the imaging modality and the clinical knowledge modality.

[0011] ; Where N is the batch size. It is the normalized feature of the k-th sample. It is the normalized feature of the j-th sample. These are the corresponding positive samples. It is a temperature parameter.

[0012] Furthermore, in step three, modality-specific learning extracts shared features using a multilayer perceptron with shared weights and a modality-specific multilayer perceptron, respectively. Modality-specific features To encourage representation decoupling, an alignment loss is introduced. This loss is calculated by sharing features. Reconstruction of representation The differences, and modality-specific features Reconstruction of representation By minimizing these mean square errors, the model can effectively separate modality-specific information from shared information while maintaining consistency of shared information, thereby achieving feature decoupling. .

[0013] Furthermore, in step three, feature fusion is achieved through a Transformer encoder, which converts modality-specific representations... After splicing, input the data into the encoder. These are mapped to a query matrix Q, a key matrix K, and a value matrix V, respectively; then the dot product of the query and the key is calculated and divided by the scaling factor. After passing The attention weights are obtained; finally, the value matrix V is weighted and summed using these weights to obtain the fused feature representation H. ; in, These represent the query, key, and value matrices, respectively. The transpose of the key matrix. The vector feature dimension of the key matrix is ​​obtained by multiplying the concatenated features by three different weight matrices. H is the feature representation after attention. The attention-fused feature H is input into the classification head multilayer perceptron and first undergoes a first-layer linear transformation. And through a nonlinear activation function The intermediate feature Z is obtained, and then a linear transformation is performed on the output layer. Then, the Softmax function is used to map the intermediate features into the final classification probability vector. Its form is: ; ; in, Z represents the parameters of the first-level linear transformation, and Z represents an intermediate calculation result. Here are the output layer parameters, and σ(⋅) is the non-linear activation function. This is the final classification probability vector, and the overall optimization objective function of the multimodal learning framework. It is a weighted sum of classification loss, alignment loss, and contrast loss.

[0014] Compared with existing technologies, the beneficial effects of this invention are as follows: By combining a dementia knowledge graph, structured clinical domain knowledge is integrated into a deep learning model, providing prior information beyond data-driven classification, effectively capturing the causal relationships between clinical features, and significantly improving classification performance; the use of a conditional diffusion model to generate and enhance MRI image features can capture subtle neuroanatomical and pathological changes that are crucial for early classification, alleviating the data sparsity problem caused by limited sample size; a multi-level fusion strategy including bimodal consistency learning and modality-specific representation learning is designed, which, compared with simple feature stitching, can more effectively integrate two heterogeneous data, neuroimaging and clinical knowledge, achieving more robust cross-modal information alignment and complementarity. Attached Figure Description

[0015] The invention will now be further described with reference to the accompanying drawings.

[0016] Figure 1 This is a flowchart of a dementia classification method based on knowledge-guided image diffusion reasoning in a specific embodiment of the present invention; Figure 2 This is a visualization of the knowledge graph in a specific embodiment of the present invention; Figure 3 This is a bar chart of ablation experiments with different loss functions in the NACC case, in a specific embodiment of the present invention; Figure 4 This is a visualization of the feature distribution of NACC cases in a specific embodiment of the present invention; wherein, Figure 4 In this context, t-SNE represents t-distributed random nearest neighbor embedding, NC represents normal cognition, MCI represents mild cognitive impairment, AD represents Alzheimer's dementia, and p represents the significance threshold. Figure 5 This is a receiver operating characteristic (ROC) curve of a NACC case in a specific embodiment of the present invention; wherein, Figure 5 In the middle, A represents the working characteristic curve of a normal cognitive subject. Figure 5 B represents the operating characteristic curve of subjects with mild cognitive impairment. Figure 5 C represents the operating characteristic curve of dementia subjects. Figure 5 D in the middle represents the macro average working characteristic curve; Figure 6 This is a precision-recall curve of NACC cases in a specific embodiment of the present invention; wherein, Figure 6In the middle, A represents the normal precision-recall curve. Figure 6 In section B, the precision-recall curve represents the result of mild cognitive impairment. Figure 6 In the middle, C represents the precision-recall curve for dementia. Figure 6 D represents the macro average precision-recall curve. Detailed Implementation

[0017] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0018] Example 1 like Figure 1 As shown, a dementia classification method based on knowledge-guided and imaging diffusion reasoning includes the following steps: Step 1: Acquire three-dimensional magnetic resonance imaging (3D-MRI) data. Generate image features related to specific disease categories using a feature extraction network and a conditional denoising diffusion model. Let the original 3D-MRI volume data be... Where H, W, and D are the height, width, and depth of a voxel, respectively, the MRI data is input into a 3D feature extraction convolutional network: ; in Representing the 3D convolution of layers, This represents the output of the l-th 3D convolution layer. represent The input of the 3D convolutional layers is used to obtain the extracted features. ; Step 2: Obtain clinical metadata corresponding to the 3D-MRI data, construct a dementia knowledge graph (DementiaKG) containing entities, relationships and attributes, and generate its knowledge graph embedding representation; Step 3: Extract the image features obtained in Step 1. The knowledge graph embedding obtained in step two is input into a multimodal learning framework. Feature fusion is performed through bimodal consistency learning and modality-specific learning. A classifier is then used to process the fused features, outputting the final dementia classification result. .

[0019] In detail, the specific training process of the conditional denoising diffusion model in step one is as follows: (1) Forward diffusion process: For the initial eigenvector By gradually adding Gaussian noise through a series of steps, the noisy features are obtained at any time step t. The calculation formula is as follows: ; in, This represents the percentage of signal remaining at each step. The cumulative signal hold ratio is the product of the signal percentages at each step from 1 to t. For linearly increasing variance scheduling, Standard Gaussian noise; (2) Model training: Train a U-Net network to learn to predict and remove noise given class labels, based on noisy features. The added noise is predicted using time step t and category label y. Loss function of diffusion model Defined by minimizing the mean square error (MSE) between the predicted noise and the actual noise: ; in, Represents predicted noise; (3) Backsampling process: Starting from a random Gaussian noise vector, new feature vectors under different class conditions are generated through iterative denoising. The denoising update formula for each step is: ; ; in, It is the estimated mean. It is the prediction noise of the U-Net model. , It is the posterior variance. For the first Step data status, The noise standard deviation at time step t, This represents the percentage of signal remaining at each step. The cumulative signal hold ratio is the product of the signal percentages at each step from 1 to t. This represents the predicted noise.

[0020] In step two, the dementia knowledge graph It is formalized as a directed edge-labeled graph: ; in, It is a collection of entities (such as patients, assessment items). It is a set of edges representing relationships. It is a collection of attributes. For attribute mapping, each relation in the graph is represented as a triple from the Resource Description Framework (RDF). Existence, in which It is the main body. It is a predicate. It is the object.

[0021] In step three, bimodal consistency learning is achieved through a contrastive learning strategy, which employs a normalized temperature-scaled cross-entropy contrastive loss function. Aligning the representations of the imaging modality with the clinical knowledge modality: ; Where N is the batch size. It is the normalized feature of the k-th sample. It is the normalized feature of the j-th sample. These are the corresponding positive samples. It is a temperature parameter.

[0022] In step three, modality-specific learning extracts shared features through a multilayer perceptron with shared weights and a modality-specific multilayer perceptron. ) and comprehensive characteristics ( And introduce a mean square error alignment loss. To encourage representation decoupling: .

[0023] In step three, feature fusion is achieved through a Transformer encoder, which converts modality-specific representations... After being concatenated, the data is input into the encoder, where information is fused using a self-attention mechanism. ; in, These represent the query, key, and value matrices, respectively. The transpose of the key matrix. The vector feature dimension representing the key matrix is ​​obtained by multiplying the concatenated features by three different weight matrices. H is the feature representation after attention. H is input to the classification head MLP, and its form is: ; ; in, Z represents the parameters of the first-level linear transformation, and Z represents an intermediate calculation result. Here are the output layer parameters, and σ(⋅) is the non-linear activation function. This is the final classification probability vector, and the overall optimization objective function of the multimodal learning framework. The weighted sum of classification loss, alignment loss, and contrast loss: ; in For the prediction results With real labels Cross-entropy loss, and To control the hyperparameters contributing to each loss term, This represents the cross-entropy comparison loss function.

[0024] Finally, four metrics—accuracy (ACC), F1 score, recall, and precision—are used to evaluate the model's accuracy. Accuracy refers to the proportion of correctly predicted samples out of the total number of samples, providing a direct assessment of the model's overall performance. Recall, also known as the full detection rate, represents the ratio of correctly predicted positive instances to the total number of actual positive instances, evaluating the model's ability to discover all relevant instances. The F1 score is the harmonic mean of precision and recall, comprehensively considering both the model's accuracy and completeness. Precision represents the proportion of correctly predicted positive instances out of the total number of correctly predicted positive instances, focusing on the accuracy of the model's positive predictions. The formulas are as follows: ; ; ; ; TP represents a true positive, TN represents a true negative, FP represents a false positive, and FN represents a false negative. The higher these indicators are, the more accurate the prediction results.

[0025] Example 2 In this embodiment, the visualization of the knowledge graph is as follows: Figure 2 As shown, a dementia classification method based on knowledge-guided and imaging diffusion reasoning includes the following steps: Step 1: Neuroimaging Feature Extraction and Generation. A 3D neural network is used to process 3D-MRI scan data from dementia patients to extract preliminary image feature vectors. These feature vectors are then input into a conditional denoising diffusion model. At any time step t, the noisy features... The calculation formula is as follows: ; in, , , For linearly increasing variance scheduling, It is standard Gaussian noise.

[0026] The model progressively adds noise to features during the forward pass and learns to predict and remove this noise given a class label by training a U-Net network. The added noise is predicted using time step t and category label y. Loss function of diffusion model Defined by minimizing the mean square error (MSE) between the predicted noise and the actual noise: ; After training, through a backsampling process, the model can generate high-quality enhanced features for specific disease categories from random noise, which are then used for subsequent classifier training. The denoising update formula for each step is: ; ; in, It is the estimated mean. It is the prediction noise of the U-Net model. , It is the posterior variance.

[0027] Step Two: Dementia Knowledge Graph Construction and Representation. Collect and integrate multimodal clinical metadata such as patient demographics, clinical classification results, and neuropsychological assessment scores. Based on ontology methods, transform this data into a structured knowledge graph (DementiaKG). This knowledge graph, in the form of RDF triples, explicitly represents the causal and hierarchical relationships between different clinical entities (such as patients, assessment items, and classification conclusions), thereby transforming unstructured clinical experience into a machine-readable format. The construction process of the structured knowledge graph (DementiaKG) is as follows: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Formalize it as a directed edge-labeled graph: in, It is a collection of entities (such as patients, assessment items). It is a set of edges representing relationships. It is a set of attributes. Each relation in the graph is represented as a triple in Resource Description Framework (RDF). Existence, in which It is the main body. It is a predicate. It is the object.

[0028] Step 3: Knowledge-guided multimodal fusion and classification. The image features extracted and generated in Step 1, along with the knowledge graph embedding representation constructed in Step 2, are input into a knowledge-guided multimodal learning framework.

[0029] The framework achieves information fusion and classification through the following mechanism: First, it employs a contrastive learning strategy by minimizing a normalized temperature-scaled cross-entropy contrastive loss function. To align shared semantic representations between the imaging modality and the clinical knowledge modality, a cross-entropy contrastive loss function with normalized temperature scaling is used. : ; Where N is the batch size. It is the normalized feature of the k-th sample. These are the corresponding positive samples. It is a temperature parameter.

[0030] Secondly, by designing neural network pathways that support both modality sharing and modality specificity, and introducing an alignment loss, we can achieve the desired results. This approach promotes the decoupling of shared features while preserving the unique information of each modality. ; in, Multilayer perceptron (MLP) representing shared weights. This represents a modality-specific MLP. Shared features are extracted as follows: The comprehensive characteristics are .

[0031] Finally, the aligned and enhanced features are input into a Transformer encoder for deep fusion, and the final classification prediction is output through a fully connected classification layer. The overall optimization objective function of the framework is... The weighted sum of classification loss, alignment loss, and contrast loss: ; in, and Hyperparameters used to control the contribution of each loss term.

[0032] Example 3 This embodiment aims to illustrate the specific application and effects of the present invention on a set of real-world datasets.

[0033] The hardware environment for this embodiment is as follows: one server, CPU: i9-14900K, GPU: RTX 4090 (24GB); software environment: Python 3.11; operating platform: Ubuntu 24.04.

[0034] This embodiment tests the effectiveness of the proposed method on two publicly available large-scale, multi-center datasets: the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset and the National Center for Alzheimer's Disease Coordination (NACC) dataset. Detailed information about the experimental datasets is shown in Table 1.

[0035] Table 1. Detailed information on the experimental dataset .

[0036] The feature distribution visualization of the NACC dataset (NACC cases) is shown in the figure below. Figure 4The implementation process of this method is illustrated using the example shown below, and specifically includes the following steps: Step 1: MRI Data Preprocessing and Feature Extraction. First, the raw MRI scan data from 4023 NACC samples underwent standardized preprocessing, including: aligning the scan images with the MNI standard template (Montreal Neurological Institute) to correct for orientation differences; using the robustfov and fslmaths tools to crop and preserve brain regions; using the BET tool (Brain Extraction Tool) to perform skull dissection to remove extracranial tissue; and using the FLIRT tool (FMRIB's Linear Image Registration Tool) for linear registration to the MNI standard space.

[0037] After preprocessing, following the steps described in Example 1, a 3D-ResNet network is used to extract the feature vector of each MRI sample, and a conditional diffusion model is used to generate class-enhanced synthetic features.

[0038] Step 2: Construction and Representation of the Dementia Knowledge Graph (DementiaKG). Using clinical tabular data associated with 4023 NACC samples, including demographics, Mini-Mental State Examination (MMSE) scores, and Clinical Dementia Rating Scale (CDR) scores, a dementia knowledge graph (DementiaKG) was constructed. Through knowledge graph embedding technology, a low-dimensional vector representation capable of capturing complex clinical relationships was generated for each patient.

[0039] Step 3: Multimodal Fusion Training and Classification. The MRI image features obtained in Step 1 and the knowledge graph obtained in Step 2 are embedded into the multimodal learning framework. Five-fold cross-validation is used for model training and evaluation to ensure robustness and reduce overfitting. The model's total loss function is a weighted average of classification loss, contrastive loss, and alignment loss. Joint optimization ensures that the model learns well-aligned and information-rich multimodal representations while accurately classifying the data.

[0040] Four metrics are used to evaluate the model's accuracy: Accuracy (ACC), F1-score, Recall, and Precision. Accuracy is the proportion of correctly predicted samples out of the total number of samples, providing a direct assessment of the model's overall performance. Recall, also known as the full detection rate, represents the ratio of correctly predicted positive instances to the actual number of positive instances, evaluating the model's ability to find all relevant instances. The F1-score is the harmonic mean of precision and recall, comprehensively considering both the model's accuracy and completeness. Precision represents the proportion of correctly predicted positive instances out of the total number of predicted positive instances, focusing on the accuracy of the model's positive predictions. Its calculation formula is: ; ; ; ; TP represents a true positive, TN represents a true negative, FP represents a false positive, and FN represents a false negative. The higher these indicators are, the more accurate the prediction results.

[0041] The proposed method (KIDI) was compared with various baseline models and state-of-the-art (SOTA) methods, including single-modal models using only images (such as 3D residual networks ResNet-3D and Squeeze-and-Excitation Net) and multimodal models employing simple stitching strategies (Concat-baseline Models), as well as general multimodal models (CLIP) designed for image-text tasks, and HCCT (Hybrid CompactConvolutional Transformer) and AddFormer (Alzheimer's Disease Detection Transformer). Experimental results are shown in Table 2, with all metrics representing the average values ​​after 5-fold cross-validation on the test set.

[0042] Table 2 Comparison of experimental results for the NACC dataset .

[0043] Table 3 Comparison of experimental results for the ADNI dataset .

[0044] As can be seen from the results in Tables 2 and 3, the method proposed in this invention significantly outperforms traditional modal splicing methods and multimodal baseline models in all evaluation metrics (especially accuracy and F1 score) on both the NACC and ADNI datasets. This indicates that the present invention, by enhancing features through knowledge graph guidance and conditional diffusion models, can more effectively fuse heterogeneous data, thereby achieving more accurate and reliable dementia classification.

[0045] For modal concatenation strategies, concatenation occurs before the final classification layer, where features are simply joined with the features of the classification data. While these models achieve reasonable performance, they often struggle to effectively capture complex intermodal relationships. For example, on the NACC dataset, their ACC (accuracy) ranges from 0.8056 to 0.8198, and their AUC (area under the curve) is approximately 0.93. On the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, performance is significantly lower, with ACC ranging from 0.7156 to 0.7385. KIDI (Knowledge-Guided Imaging Diffusion Inference) significantly outperforms all these baselines across all metrics on both datasets. On NACC, KIDI achieves an ACC of 0.8435 and an F1 score of 0.8168, clearly demonstrating that its knowledge-guided fusion strategy is superior to simple concatenation. The performance difference is even more pronounced on ADNI, with KIDI achieving an ACC of 0.7976 and an F1 score of 0.7835, demonstrating its robustness and generalizability across different cohorts. This advantage highlights the effectiveness of KIDI's multimodal consistency learning and modality-specific representation learning, which outperforms direct feature concatenation and can thoroughly integrate heterogeneous representations of neural image clinical pairs.

[0046] CLIP (Contrastive Language-Image Pretraining) is a representative multimodal model designed for general image-text understanding. While not tailored for medical images and tabular clinical data, its ability to learn joint representations across modalities makes it a relevant comparison. CLIP achieved competitive results on the National Center for Alzheimer's Disease Coordination (NACC) dataset (the receiver operating characteristic curves for the NACC cases are shown in the figure). Figure 5 As shown in the figure, the precision-recall curves for NACC cases are as follows: Figure 6As shown in the figure, KIDI achieved an ACC (accuracy) of 0.8254 and an F1 score of 0.7923, with a particularly high AUC (area under the curve) of 0.9478. However, KIDI still outperformed CLIP in ACC, F1 score, recall, and NACC precision. KIDI's slightly lower AUC (0.9343 vs. 0.9478) compared to CLIP on NACC may indicate that CLIP's more extensive pre-training on different image-text pairs allows for more robust classification boundary separation, while KIDI's strength lies in its precision and recall, which are crucial for clinical classification tasks where identifying true positives and minimizing false positives is essential. On the ADNI dataset, KIDI significantly and consistently outperformed CLIP across all metrics, with KIDI's ACC (0.7976) and F1 score (0.7835) significantly higher than CLIP's (0.7542 and 0.7269, respectively). This highlights KIDI’s specialized design, including a conditional diffusion module for enhancing MRI features and explicit integration of domain-specific DementiaKG, which makes it better suited to handling the nuances of medical classification data compared to general multimodal models.

[0047] Overall, the findings consistently demonstrate that KIDI's innovative architecture, combining knowledge-guided deep learning and conditional diffusion modules, delivers superior and more reliable classification performance for AD on both datasets. Subtle fusion strategies and domain-specific knowledge injection enable KIDI to extract more distinctive features and establish more robust intermodal relationships.

[0048] The main contribution of this invention is to propose a novel Knowledge Guided Instruction (KIDI) framework for dementia classification. This framework effectively combines domain knowledge with multimodal image data through a diffusion-based learning paradigm, thereby improving prediction accuracy and enhancing model interpretability.

[0049] Example 4 To investigate the contribution of key blocks, ablation studies were conducted on three key blocks in the KIDI method: Table 4 Comparison of Modal and KG Ablation Results .

[0050] According to Table 4, the KG-guided classification strategy proposed in this invention outperforms all other combinations, demonstrating the effectiveness of knowledge-guided learning. In the case of NACC, the evaluation metrics of the configuration without DementiaKG are significantly lower than those achieved by the KIDI method. This indicates that the constructed DementiaKG is well-suited for integration into a multimodal learning framework for AD classification. The causal relationships between entities extracted from clinical metadata in tabular form provide unique information, improving accuracy and reliability.

[0051] like Figure 3 As shown, this chart is a bar graph comparing F1 scores of different datasets (NACC and ADNI) under different conditions. This condition indicates that a specific loss term will not be used. Model performance at that time; w / o This condition excludes another loss term. ;w / o This condition indicates that the third loss term will not be used. .

[0052] On the NACC dataset, certain loss terms (such as...) are not present. and Under the given conditions, the F1 score is generally higher than the KIDI baseline model. On the ADNI dataset, the overall F1 score is lower, but a similar trend is observed, where models that remove certain loss terms outperform KIDI. KIDI effectively combines domain knowledge with multimodal imaging data through a diffusion-based learning paradigm, improving prediction accuracy and interpretability. By integrating structured priors from medical knowledge graphs and leveraging the generative capabilities of diffusion models, it provides robust and interpretable classification decisions in heterogeneous neuroimaging clinical data.

Claims

1. A dementia classification method based on knowledge-guided and imaging diffusion reasoning, characterized in that: The method includes the following steps: Step 1: Acquire three-dimensional magnetic resonance imaging data, and generate high-dimensional representation features of the three-dimensional image through a feature extraction network and a conditional denoising diffusion model; Step 2: Obtain clinical metadata corresponding to the 3D magnetic resonance imaging data, construct a dementia knowledge graph containing entities, relationships and attributes, and generate its knowledge graph embedding representation; Step 3: Input the image features obtained in Step 1 and the knowledge graph obtained in Step 2 into a multimodal learning framework. Perform feature fusion through bimodal consistency learning and modality-specific learning, and use a classifier to process the fused features to output the final dementia classification result.

2. The dementia classification method based on knowledge guidance and imaging diffusion reasoning according to claim 1, characterized in that: In step one, the specific training process for the conditional denoising diffusion model is as follows: (1) Forward diffusion process: Gaussian noise is gradually added to the initial features through noise scheduling parameters to generate noisy features; (2) Model training: The conditional U-Net network is used to predict the noise in the noisy features and is optimized by mean square error; (3) Back sampling process: Starting from random Gaussian noise, the feature vector of the target category is gradually generated through iterative denoising. 3.The dementia classification method based on knowledge guidance and imaging diffusion reasoning of claim 1, wherein: In step two, the knowledge graph is represented as a directed graph containing entity nodes, relational edges connecting the entity nodes, and a corresponding set of attributes for each relational edge to describe the specific information of that relation.

4. The dementia classification method based on knowledge guidance and imaging diffusion reasoning according to claim 1, characterized in that: In step three, bimodal consistency learning is achieved through a contrastive learning strategy. For each sample in a batch, the model calculates its similarity to the corresponding positive sample and compares it with the similarity of other samples in the same batch in a normalized manner. The similarity scale is adjusted by using a temperature parameter, and the result is input into the cross-entropy contrastive loss function. The model is trained to maximize the matching of positive samples of the same modality and different modality, while minimizing the similarity with other negative samples, thereby aligning the feature representations of the imaging modality and the clinical knowledge modality.

5. The dementia classification method based on knowledge-guided and image diffusion reasoning of claim 1, wherein: In step three, modality-specific learning extracts shared features and modality-specific features through a multilayer perceptron with shared weights and a modality-specific multilayer perceptron, respectively. To encourage representation decoupling, an alignment loss is introduced: this loss calculates the difference between the shared feature and its reconstructed representation, as well as the difference between the modality-specific feature and its reconstructed representation, and minimizes these mean square errors. This allows the model to effectively separate modality-specific information from shared information while maintaining the consistency of shared information, thereby achieving feature decoupling.

6. The dementia classification method based on knowledge-guided and imaging diffusion reasoning according to claim 1, characterized in that: In step three, the feature fusion is implemented by a Transformer encoder, which fuses the modality-specific representations into a unified representation. The fused representation is then input into a classification head multi-layer perceptron (MLP) to obtain the final classification probability vector. The overall optimization objective function of the multi-modal learning framework is the weighted sum of the classification loss, alignment loss, and contrastive loss.

Images