VQA Training Data Generation via Inference Question Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current visual question answering (VQA) systems rely on costly and time-consuming manual labeling for training data, limiting the amount and complexity of questions, which restricts the model's ability to understand image content and answer complex inference questions.
Innovation Solution
Automatically generate training data with inference questions using pre-labeled object relations and attributes, expanding the dataset to include more complex queries, thereby improving model training efficiency and capability to answer complex questions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If manual labeling is used to obtain training data, then the training data can be obtained with direct questions related to target objects, but the process is costly, slow, and produces limited training data with simple question forms
Solution Approach 1:
The system performs preliminary action by automatically generating multiple question variations and inference questions in advance during the data preparation phase. Instead of manually creating questions one by one, the system pre-generates training data with diverse question forms and complex inference questions that can guide the model to focus on key information, thereby reducing the time and cost of subsequent training while increasing the amount of training data available.
2Adaptability or versatility
If manual labeling is used, then questions can be directly related to target objects, but the question forms remain simple without complex descriptions and inferences
Solution Approach 1:
The system introduces an intermediary component that automatically generates complex question forms and inference questions based on the training data. This intermediary layer transforms simple target object references into diverse question structures involving descriptions, inferences, and multiple reasoning steps, thereby increasing question complexity and adaptability without requiring manual effort for each complex question creation.
3Reliability
If more training data is obtained through manual labeling, then the model can be better trained, but the cost and time increase significantly
Solution Approach 1:
The system implements self-service by automatically generating its own training data with diverse question forms and complex inference questions without requiring external manual labeling. The system uses algorithms to create question variations, generate inference questions, and structure training data autonomously, thereby maintaining high model training effectiveness while dramatically improving data preparation efficiency and reducing costs.
Data Source
AI summary
Embodiments of the present disclosure are directed to a method and an apparatus for generating training data for a visual question answering (VQA) system, and a computer readable medium. The method for generating training data for a visual question answering system includes: obtaining a first set of training data of the visual question answering system, the first set of training data comprising a first question for an image in the visual question answering system and a first answer corresponding to the first question; obtaining information related to the image; generating a second question corresponding to the first answer based on the information to obtain a second set of training data for the image in the visual question answering system, the second set of training data comprising the second question and the first answer.