Image description generating method based on neural network and image attention focuses

Abstract

The invention provides an image description generating method based on a neural network and image focuses. The method is characterized in that an original one-layer word embedding structure is replaced by a two-layer word embedding structure, and accordingly word expression can be learned effectively; image feature expression is directly used as the input of an m-RNN model, and accordingly the capacity of a circulating layer can be fully utilized, and the small-dimension circulating layer can be used; by a decision soft attention mechanism, the attention of an image salient region is reflected and used as one input of a multi-modal layer. By the method, the focus relation between targets or scenes is utilized effectively, and the semantic features of the image is described in a targeted manner.

Description

technical field [0001] The present invention relates to the field of computer vision, and more specifically, to an image description generation method based on a neural network and image attention points. Background technique [0002] Obtaining text-level image descriptions has become an important research topic in the current field of computer vision, and in real life, it has many application scenarios. Examples include early childhood education, image retrieval, and navigation for the blind. With the rapid development of computer vision and natural language processing technology, a large number of effective works on this topic appear, many of which regard it as a retrieval problem. The researchers project the features of text sentences and images into the same semantic space by learning a node embedding layer. These methods generate image descriptions by retrieving similar descriptions from text sentence datasets, but they lack image descriptions that can effectively com...

AI Technical Summary

Problems solved by technology

However, RNNs rely on good semantic expression input. For the field of image understanding, the degree of attention between image objects or attributes cannot be reflected by RNNs, and for neural networks, it is a high-dimensional information processing with high computational complexity.

Images