Multi-modal denoising representation learning method and system for dynamic multi-granularity interaction

CN122153306APending Publication Date: 2026-06-05LIAONING UNIVERSITY

Patent Information

Authority / Receiving Office: CN · China
Patent Type: Applications(China)
Current Assignee / Owner: LIAONING UNIVERSITY
Filing Date: 2026-02-27
Publication Date: 2026-06-05

AI Technical Summary

Technical Problem

Existing multimodal representation learning methods struggle to simultaneously address both single-modal and cross-modal noise, and neglect the strong semantic connections between visual and textual modalities, resulting in information loss and insufficient robustness.

Method used

We employ a dynamic multi-granularity interactive multimodal denoising representation learning method (DMIM-DRL), which achieves semantic complementarity and cross-validation between visual and textual modalities through multi-granularity spatiotemporal attention mechanisms, cross-attention mechanisms, and gated linear units. Combined with an adaptive cross-modal dynamic interaction method, we optimize feature weights and reduce redundant noise interference.

Benefits of technology

It significantly improves the robustness and accuracy of multimodal representation, maintains semantic consistency in complex noisy environments, and enhances the robustness and accuracy of downstream tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

Figure CN122153306A_ABST

Patent Text Reader

Abstract

The application relates to a multi-modal denoising representation learning method and system of dynamic multi-granularity interaction, and belongs to the technical field of multi-modal data processing and representation learning. First, a multi-granularity feature interaction denoising method is proposed. By introducing a multi-granularity space-time attention mechanism and a cross attention mechanism and combining a gate linear unit, semantic complementation and cross verification between internal modalities of a visual end and a text end are realized, damaged modal features caused by noise interference are effectively repaired, and noise suppression capability is improved. Secondly, an adaptive cross-modal dynamic interaction denoising method (ACID) is proposed. A compression-excitation mechanism and a cooperative attention mechanism are combined to adaptively and dynamically optimize feature weights, realize cross-modal deep semantic alignment, and reduce redundant noise interference. Finally, multi-task loss functions such as image-text alignment and mask language modeling are used for optimization training, the robustness and effectiveness of multi-modal representation are significantly improved, and the application service purpose is better achieved.

Need to check novelty before this filing date? Find Prior Art