Training method of visual question answering model and visual question answering method
By performing text recognition on images and adding phrases to a general answer set before training the visual question answering model, and then combining this with BUTD attention model training, the accuracy problem of visual question answering models when the answers come from the images themselves is solved, thus improving the accuracy of the answers.
CN116311312BActive Publication Date: 2026-06-09TD TECH LTD +1
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TD TECH LTD
- Filing Date
- 2021-12-21
- Publication Date
- 2026-06-09
AI Technical Summary
Technical Problem
Existing visual question answering models have low accuracy when the answer comes from the image itself and does not exist in a general answer database.
Method used
Before training the visual question answering model, text recognition is performed on the image to be recognized. Multiple recognized word groups are added to a general answer set to obtain the target answer set. The model is then trained using the BUTD attention model to obtain the visual question answering model.
Benefits of technology
It improves the accuracy of visual question answering models in outputting answers, especially in handling questions with specific words in images.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN116311312B_ABST
Abstract
The application provides a visual question and answer model training method and a visual question and answer method. The visual question and answer method comprises the following steps: obtaining an image to be recognized and a corresponding question to be answered, performing text recognition on the image to be recognized, adding a plurality of word groups recognized to a general answer set of a visual question and answer model, obtaining a target answer set, respectively performing feature extraction on the image to be recognized and the question to be answered to obtain image features and first text features, inputting the image features and the first text features into the visual question and answer model to obtain a first answer. By adding the plurality of word groups in the image to be recognized to the general answer set, when the visual question and answer model is used to obtain an answer corresponding to the question to be answered, not only the answers in the general answer set are considered, but also the influence of the word groups in the image to be recognized on the question to be answered is considered, so that the accuracy of the output answer is effectively improved.
Need to check novelty before this filing date? Find Prior Art