Visual question answering method and device based on knowledge verification and interactive reasoning

CN117763125BActive Publication Date: 2026-07-10TIANJIN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
TIANJIN UNIV
Filing Date
2024-01-10
Publication Date
2026-07-10

AI Technical Summary

Technical Problem

Existing visual question answering models tend to introduce a lot of noise unrelated to the image question when integrating external knowledge, and lack effective knowledge-to-image-region-level fine-grained association, resulting in insufficient cross-modal reasoning accuracy.

Method used

A method based on knowledge verification and interactive reasoning is designed. By evaluating the confidence of knowledge and constructing intra-domain and inter-domain attention masking mechanisms, high-quality knowledge is screened and the alignment relationship between image regions and knowledge is established. Interactive reasoning is performed using the Transformer model.

Benefits of technology

It improves the denoising effect of knowledge screening and the accuracy of answers, and enhances the accuracy of cross-modal reasoning in complex scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117763125B_ABST
    Figure CN117763125B_ABST
Patent Text Reader

Abstract

The application discloses a visual question answering method and device based on knowledge verification and interactive reasoning. The method comprises the following steps: for the extracted relevant knowledge, the confidence of each knowledge item is evaluated according to the keyword type and the organization form of the knowledge source; the knowledge to be introduced and the image information are combined to interactively learn the receptive field control process and the image domain mask generation process; the mask generation process of the knowledge domain part in the domain interactive reasoning method is executed; the inter-domain mask generation process between the image domain and the knowledge domain in the domain interactive reasoning method is executed; the optimization process of the intra-domain mask and the inter-domain mask in the domain interactive reasoning is executed; the image features, the knowledge features and the current question features are integrated and sent into a decoder, meanwhile, a loss function is minimized, network parameters are optimized, and finally the correct answer to the question raised by a human being for the current scene is generated. The device comprises a processor and a memory.
Need to check novelty before this filing date? Find Prior Art