Picture description method of guiding attention mode on the basis of attribute probability vector

Abstract

The invention discloses a picture description method of guiding an attention mode on the basis of an attribute probability vector. The method comprises the following steps: inputting an image to obtain a feature graph through a fully convolutional neural network, and then obtaining the attribute probability vector through a multi-instance learning algorithm layer; selecting certain threshold values for the obtained attribute probability vector to initialize hidden states of c0 and h0 of a long short-term memory (LSTM) unit; guiding the attention mode through the attribute probability vector, and combining a state of ht-1 of a description statement LSTM of a previous moment to generate an encoding vector, which currently needs to be attended, at a region which is on the feature graph and attended by the attention mode of a current moment; outputting an output state of ht of the current moment by the description statement LSTM according to the current encoding vector; and becoming the state of the previous moment by the output state of the current moment, and repeating the previous operations until generation of a description language is completed. Compared with other methods, the picture description method of guiding the attention mode on the basis of the attribute probability vector of the invention obviously improves effects, is better in comprehensive performance of evaluation indexes, and can basically be competent for general picture description needs.

Description

technical field [0001] The present invention designs a picture description method based on an attribute probability vector to guide an attention pattern, and relates to the technical fields of deep learning and computer vision. Background technique [0002] An important feature of human perception of the world is that they do not process the entire scene at one time, but focus on certain parts of the visual space to obtain the required time and place information, and as time progresses, Human beings will build the internal representation of the scene based on the information of different fixed points to guide a series of cognition and actions in the future; due to the simplicity of some scenes compared to the whole scene, this kind of "limited" brain resources are concentrated on the perception part The mechanism on important scenes directly leads to the reduction of the complexity of human processing scenes, because it allows humans to always place the objects of interest i...

AI Technical Summary

Problems solved by technology

[0005] Among the image description models based on attention mode, the Soft Attention (Soft-ATT) model proposed by Xu et al. is very representative, but the obtained image description results are still not accurate enough.

Images