Multimodal feature interaction method and apparatus based on information filtering

By introducing semantically guided adaptive information filtering and bidirectional interaction mechanisms into multimodal analysis, the problem of spurious associations caused by noise interference is solved, improving the model's discriminative performance and robustness, especially its generalization ability in complex scenarios.

CN122173771APending Publication Date: 2026-06-09NAT UNIV OF DEFENSE TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202610625966.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-05-08
Publication Date
2026-06-09

AI Technical Summary

Technical Problem

Existing multimodal analysis methods tend to incorporate noise along with valid information into the model when dealing with complex scenarios, leading to the model learning spurious associations that are irrelevant to the task, resulting in insufficient robustness and generalization ability.

Method used

By introducing semantically guided adaptive information filtering, image and text data are acquired, semantic conditions are constructed, adaptive filtering weights are generated, image features are weighted and filtered, and text and image features are updated through a two-way interaction mechanism to suppress noise and enhance key information.

Benefits of technology

It significantly improves the model's discrimination performance and robustness, and enhances its generalization ability in noisy and key information-sparse scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122173771A_ABST
    Figure CN122173771A_ABST
Patent Text Reader

Abstract

The application relates to a multi-modal feature interaction method and device based on information filtering. The method comprises the following steps: extracting image local features and text features; constructing a text semantic condition; generating adaptive weight filtering image noise based on the condition, and obtaining filtered features; updating the text features by using the filtered features; and guiding the image features again by using the updated text features. The filtering and interaction processes are jointly optimized by using a loss function containing a regularization term. The method can effectively suppress noise in multi-modal data, and improve feature consistency and task robustness through bidirectional closed-loop interaction.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Multi-modal semantic collaborative interaction image-text joint named entity recognition method

    CN115455970A

  • Text-guided image feature selection and bidirectional attention fused multi-modal relation extraction method

    CN121581176A

  • Joint multi-modal entity relationship extraction method and system based on information representation and semantic alignment

    CN121859903A

  • Knowledge fusion multi-modal interaction method and apparatus based on improved alignment method

    WO2025025290A1