Visual question-answering model training method and device

Abstract

The invention provides a visual question-answering model training method and device, and relates to the technical field of computers. The visual question-answering model training method comprises thesteps of obtaining a training sample and a sample label; extracting sample image feature information and sample problem feature information; performing feature cross processing on the sample image feature information and the sample problem feature information to obtain a sample image feature vector carrying the sample problem information and a sample problem feature vector carrying the sample image information; inputting the sample image feature vector carrying the sample question information and the sample question feature vector carrying the sample image information into the visual question-answering model to obtain a prediction answer through the visual question-answering model; determining a loss value of a loss function based on the real answer and the predicted answer; and updating the visual question and answer model through the loss value of the loss function.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a visual question answering model training method and device, a computing device and a computer-readable storage medium. Background technique [0002] Visual Question Answering (VQA) is a comprehensive task involving computer vision and natural language processing. A VQA system takes a picture and a free-form, open-ended natural language question about the picture as input. Generate a natural language answer as output. [0003] At present, the existing visual question answering model training methods generally first extract the image features to be answered through a pre-trained deep convolutional neural network model (CNN), convert the questions into several word vectors, and then convert the image features into The question words of the word vector are input into the long short-term memory network (LSTM) together, and the LSTM network is used to generate the answer...

AI Technical Summary

Problems solved by technology

[0004] However, the existing visual question answering model training methods are too simple to extract and process image features and question features, without considering the spatial semantic context information between image regions, and the current visual question answering VQA question data set is limited, and the model is generally overfitting. combined state, which affects the semantic context similarity between the obtained answer and the real answer

Images