Visual question-answering method and system of visual question-answering model based on labor division decision

Abstract

The invention belongs to the field of image question answering, and particularly relates to a visual question answering method and system of a visual question answering model based on division of labor decision, and the method comprises the steps: obtaining a visual image and a to-be-answered question, inputting the visual image and the to-be-answered question into an LRBNet model, and obtaining a question answering result; the LRBNet model comprises a visual understanding module, a text understanding module and an exchange module; the visual understanding module is used for obtaining a visual feature map, the text understanding module is used for obtaining a text feature map, and the exchange module is used for carrying out data interaction on the visual feature map and the text feature map and updating nodes according to interaction data; associating and updating the visual spatial feature map and the text semantic information to obtain a final question and answer result; according to the method, the text semantic information and the visual space information are separately processed, and only the processed results are finally fused, so that the reasoning difficulty of other VQA models due to cross-modal fusion is reduced.

Description

technical field [0001] The invention belongs to the field of image question answering, and in particular relates to a visual question answering method and system based on a division of labor decision-making visual question answering model. Background technique [0002] Deep learning algorithms have achieved great success in both vision-related and language-related tasks, and visual question answering is a task that tests cross-modal understanding of vision and language. In the most common form of visual question answering, a computer is presented with an image and a textual question about the image, and a visual question answering algorithm needs to answer the question based on the image. [0003] Most current image question answering models (VQA) use neural networks such as recurrent neural networks (RNNS) and long short-term memory networks (LSTM) to learn encoded representations of questions. In order to encode the image, the early VQA model used Resnet or VGG to extract...

Images