The invention relates to the technical field of data fusion, and discloses a multi-
modal multi-source heterogeneous data fusion method and
system.The multi-
modal multi-source heterogeneous data fusion method comprises the steps that a multi-
modal prototype network is constructed, different modal features are extracted, and a
semantic mapping relation between modals is established; a
memory module is constructed, and multi-modal characterization of historical key samples is stored through sample value evaluation; executing diversity sampling to reserve a rare abnormal mode; modal balance
processing is implemented, and contribution weights of different modal gradients are dynamically adjusted; knowledge
distillation is applied, and general fusion knowledge is extracted from a data-rich scene to assist
small sample decision making; the method solves the problems of low fusion efficiency of
small sample and
cold start scenes, disastrous forgetting in
incremental learning, unbalanced modal learning rate and rare mode retention under long-
tail distribution, and is suitable for
predictive maintenance, personalized recommendation and
medical health monitoring in the manufacturing industry and scenes requiring
small sample learning and incremental updating.