Model training method, and task type visual dialogue problem generation method and device

Abstract

The invention discloses a model training method, and a task type visual dialogue problem generation method and device. The model training method comprises the following steps: obtaining human dialoguedata and the feature data of a corresponding image, determining the question category of each round of question and answer data, and generating a question category label; and traversing each round ofquestion and answer data according to a dialogue sequence, and training a preset task type visual dialogue question generation model by utilizing the round of question and answer data, the corresponding question category label and the feature data of the image. The training comprises the steps of generating a context vector and image features with text guidance based on a round of question and answer data currently input to the model and feature data of an image; and predicting the question category of the next round of question and answer data based on the context vector and the image features, predicting the question of the next round of question and answer data within the category range, and adjusting the network parameters of the model based on the prediction result. Dialogue interaction turns can be reduced, and the task success rate is improved.

Description

technical field [0001] The invention relates to mobile communication technology, in particular to a model training method and a method and device for generating task-type visual dialogue problems. Background technique [0002] At present, the visual dialogue task system is one of the hot research technologies of artificial intelligence. The visual dialogue task system determines the user's target object in the relevant scene based on the question-and-answer information between the agent and the user, so as to help the user complete certain tasks for the target object. Visual dialogue is an interdisciplinary study of multi-disciplinary knowledge and methods in natural language processing, computer vision, knowledge representation learning, and reasoning. At present, the existing visual dialogue systems can be roughly divided into two types: task-based visual dialogue and chatting visual dialogue. Among them, how to generate effective questions is one of the difficult proble...

AI Technical Summary

Problems solved by technology

In this way, training based on the above supervision signals will guide the generated questions to be consistent with the questions in real-person dialogues

This way, simple questions are easier to generate

However, due to the small amount of words contained in simple questions and the small amount of effective information that is conducive to accurately guessing the target object, more rounds of interaction are required to guess the target object

In practical applications, the maximum number of interactions is limited. In this way, if the maximum number of interactions is less than the actual number of interactions required, it will happen that the target object cannot be accurately guessed after completing the maximum number of interactions. Execution failed

Moreover, the expression of natural language is diverse, and different expressions can have similar effects. However, the model training in the above method is driven only by the difference in the expression form of the generated sentences, that is, it is only based on real dialogues and predictions. The feedback adjustment of the supervision signal generated by the difference between the dialogues will lead to the inaccuracy of the adjustment, thereby reducing the accuracy of the final trained model, and then affecting the success rate of the visual dialogue task based on the model.

[0005] In addition, the inventor found in the process of implementing the present invention that: using the existing end-to-end supervised learning method, the same problem may be raised repeatedly

In this method, when the encoder is trained, the self-training and fitting is based on a single supervisory signal, making the model training process a black box, which in turn makes the above problems insufficiently interpretable, that is, it is impossible to analyze the above problems based on the training process. s reason

For this reason, the inventors have found through research and analysis and simulation verification that one of the reasons for generating repetitive problems in the above-mentioned dialogue process is that in the existing end-to-end supervised learning method, the dialogue text information is not hierarchically encoded, so, for the The answer information that is valued will not be allocated enough attention, which may cause the model to be unable to effectively integrate different text features, so that the information answered by the user will be forgotten, and the question will be repeatedly asked. Happening

Images