Image question and answer method based on multi-objective association deep reasoningmulti-target association deep reasoning

Abstract

The invention discloses an image question and answer method based on multi-objective association deep reasoning. The method comprises the following steps of 1, carrying out data preprocessing on an image and a text described by a natural language of the image; 2, carrying out attention mechanism reordering on each target based on an adaptive attention module model enhanced by geometric features of a candidate box; 3, constructing a neural network structure based on an AAM model; and 4, model training: training neural network parameters by using a back propagation algorithm. The invention provides a deep neural network for image question answering, in particular to a method for performing unified modeling on image-question text data, performing reasoning on each target feature in an image, and reordering attention mechanisms of the targets so as to answer questions more accurately, and a better effect is obtained in the field of image question answeringThe invention discloses an image question and answer method based on multi-target association deep reasoning. The method comprises the following steps: 1, carrying out data preprocessing on an image and a text described in a natural language of the image, and 2, carrying out attention mechanism reordering on each target based on an adaptive attention module model with enhanced geometrical characteristics of a candidate box. And 3, a neural network structure based on an AAM model. And 4, model training: training neural network parameters by using a back propagation algorithm. The invention provides a deep neural network for image question answering, and particularly provides an image-image question answering method. According to the method, the data of question texts are subjected to unified modeling, reasoning is carried out on the characteristics of all the targets in the image, attention mechanisms of all the targets are reordered, so that questions are answered more accurately, and a good effect is obtained in the field of image questions and answers.

Description

technical field [0001] The present invention relates to a deep neural network structure for visual question answering (Visual Question Answering) tasks, in particular to a unified modeling of image-question answering data to find the relationship between each entity feature in the image and the geometric feature of the corresponding spatial position. The interaction relationship between them, by modeling the positional relationship between them, the method of adaptively adjusting the attention weight is achieved. Background technique [0002] Image question answering is an emerging task at the intersection of computer vision and natural language processing. This task aims to allow the machine to automatically answer the corresponding answer by giving a question related to the image. Compared with image description, another cross task of computer vision and natural language processing, it is necessary for the machine to understand images and questions and get the correct res...

AI Technical Summary

Problems solved by technology

[0007] Due to the complexity of image content and diverse subjects in natural scenes, the high degree of freedom of description based on natural language makes the description of image content a huge challenge.

Images