Disease Diagnosis Method and System Based on Multimodal Spatial-Frequency Domain Adaptive Fusion

Abstract

This invention discloses a disease diagnosis method and system based on multimodal spatial-frequency domain adaptive fusion, relating to the fields of artificial intelligence and biomedical engineering. The method standardizes multimodal data by encoding the standardized multimodal data separately to obtain initial feature representations. After projection and gating alignment, and cross-modal interactive attention alignment, the initial feature representations of each modality are obtained, resulting in enhanced representations for each modality. These enhanced representations are then fused into a shared feature representation. Deep feature extraction is performed on the enhanced representations of each modality to obtain deep features for each modality. These deep features are then subjected to adaptive multi-domain feature enhancement processing to obtain multi-domain enhanced features for each modality. The multi-domain enhanced features of each modality are semantically aligned and fused using a hierarchical attention mechanism to obtain fused features. The fused features are then input into a diagnostic network for prediction to obtain disease diagnosis results, improving the accuracy and robustness of intelligent diagnosis of papillary thyroid carcinoma.

Images