Visual question and answer method based on network structure search

Abstract

The invention provides a visual question and answer method based on network structure search. The method comprises the following steps: performing feature extraction on an original picture by adopting a first artificial neural network model; performing feature extraction on the text information by adopting a second artificial neural network model; a to-be-searched network structure framework being a coding-decoder framework, defining three search operators for framework network search, and the input of the search operators being image features or text features extracted based on the original picture or / and the text information; searching the architecture weight of the network structure and the operation weight of an operator by using a gradient-based alternative optimization strategy; and enabling the search network to output candidate word vectors according to a multi-classification method, and selecting the word vector with the maximum probability as an answer to be output. The visual question and answer method based on network structure search has the beneficial effect that a better effect can be searched in a larger space.

Description

technical field [0001] This application relates to the field of visual question answering, in particular, to a visual question answering system method based on network structure search. Background technique [0002] With the development of deep learning, visual question answering has been widely used. However, the traditional visual question answering system still has certain defects. The multimodal feature fusion part of the traditional visual question answering system is a very skillful network structure designed by experts. Through the network structure Only when the search technology automatically designs the network structure can it find the optimal network structure in a sufficiently large space. [0003] Specifically, in recent years, with the rapid development and important success of artificial intelligence, Visual Question Answering (VQA), as an intersection of computer vision and natural language processing, has attracted widespread attention. VQA tasks widely ex...

AI Technical Summary

Problems solved by technology

The current network is artificially designed by experts. These deep learning network designs have obvious "prior" traces, such as the RCNN series network in the field of image classification, from RCNN network to Fast-RCNN network, to Faster-RCNN network, and then to Mask- RCNN network, each upgrade integrates the "prior" design advantages of the previous network, and then improves it, but as the network structure becomes more and more complex, the cost of manual design network trial and error is getting higher and higher

Images