Visual question and answer method and device, computer equipment and computer readable storage medium

CN119513346APending Publication Date: 2025-02-25CHANGSHU INSTITUTE OF TECHNOLOGY
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411583399.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-07
Publication Date
2025-02-25

AI Technical Summary

Technical Problem

In the prior art, when processing image features and problem features, the joint embedding method cannot effectively model the complex relationship between the two modes, resulting in a low accuracy of the answer prediction of visual question and answer.

Method used

By introducing an adaptive attention mechanism into the visual question-and-answer model, dynamically adjusting the attention receptive field, combining the guidance attention mechanism, forming problem-guided image attention and refining visual features, thereby achieving cross-modal feature fusion.

Benefits of technology

The visual question-and-answer model's understanding of images and problems is improved, the accuracy of answer prediction is enhanced, and the problem of insufficient interaction capabilities within and between modals is solved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119513346A_ABST
    Figure CN119513346A_ABST
Patent Text Reader

Abstract

The invention discloses a visual question and answer method and device, computer equipment and a computer readable storage medium, and aims to realize cross-modal interaction by using a question guide image, so that generated image features are integrated with question information, and the intra-modal and inter-modal interaction capability is improved. The method comprises the steps of determining an adaptive attention feature map of a sample image and question feature representation of a sample question based on a visual question and answer model; performing cross-modal feature fusion on the adaptive attention feature map and the question feature representation, and determining a sample prediction answer; according to the loss between the sample prediction answer and the sample actual answer, optimizing model parameters of a visual question-answering model, and re-obtaining a training sample to optimize the visual question-answering model until the visual question-answering model reaches a standard of stopping model training, and obtaining a target model; and in response to the visual question and answer instruction, inputting a to-be-predicted image and a to-be-predicted question into the target model for prediction to obtain a predicted answer corresponding to the to-be-predicted question.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Cited By

  • Sonar image visual question and answer method and system

    CN121415084A