Data processing method and device based on multi-modal contrast fusion and graph neural network, and medium

By mapping multimodal biomedical data to a unified space and performing cross-modal semantic alignment, and combining Transformer and graph convolutional networks, the problem of difficulty in mining complementary information between modalities in multimodal data processing is solved, achieving efficient data fusion and improved classification performance.

CN121034429APending Publication Date: 2025-11-28RUIJIN HOSPITAL AFFILIATED TO SHANGHAI JIAO TONG UNIV SCHOOL OF MEDICINE
0 Cites 0 Cited by

Patent Information

Application Number
CN202510894605.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-30
Publication Date
2025-11-28

AI Technical Summary

Technical Problem

Existing multimodal biomedical data processing methods struggle to effectively integrate heterogeneous data, lack intermodal nonlinear complementary information mining, and are difficult to construct consistent representations across modalities. Consequently, the fused features fail to fully preserve the semantic correlations and sample association patterns of each modality.

Method used

By mapping three modalities of data—mRNA expression, DNA methylation, and miRNA expression—to a unified latent space, cross-modal semantic alignment is performed using the NT-Xent contrastive learning mechanism. Furthermore, the Transformer multi-head self-attention mechanism is combined to model high-order interactions between modalities, constructing a sample similarity graph structure. Finally, a graph convolutional network is used to extract high-order adjacency features of nodes for classification.

Benefits of technology

It achieves efficient semantic fusion and complementary information mining of multimodal data, improves the deep analysis capability and classification performance of medical data, and significantly enhances the robustness and generalization ability of the model, especially in scenarios with scarce labels.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to a data processing method and device based on multi-modal contrast fusion and a graph neural network, and a medium. The method comprises the following steps: collecting multi-modal biomedical data set samples of mRNA expression data, DNA methylation data and miRNA expression data; embedded matrixes of three modes are obtained through an independent feedforward neural network encoder; adopting a contrast learning mechanism based on NT-Xent to carry out unsupervised contrast alignment on the embedded matrixes of different modalities, and generating aligned embedded matrixes of three modalities; embedding and stacking the aligned modalities into a sequence, and inputting the sequence into a multi-layer Transform encoder to obtain a unified fusion representation; normalizing the fusion representation, and constructing a sample similarity graph structure; and based on the graph structure and the fusion representation, node high-order adjacency features are extracted through a three-layer graph convolutional network, sample classification is completed in combination with two full-connection layers, and a sample classification result is output. Compared with the prior art, the method has the advantages of cross-modal data fusion optimization, medical data discrimination enhancement, high efficiency and the like.
Need to check novelty before this filing date? Find Prior Art