An Image Description Method Based on Conditional Random Fields and Intrinsic Semantic Attention

Abstract

The invention relates to an image description method based on conditional random fields and internal semantic attention, and belongs to the cross technical field of computer vision and natural language processing. First process the training data, then design the network structure model, use the existing convolutional neural network and target detection network to extract image features, and design a cyclic neural network with internal semantic attention mechanism and attention feature residual structure to generate images corresponding to Description; Next, the combination of cross-entropy loss function and conditional random field loss function is used as the training target, and the processed training data is used to train the network model to obtain a network with image semantic description generation function; finally, input any image The network, get the corresponding description. Compared with the prior art, this method not only ensures the accuracy of the sentence structure of the generated description, but also solves the problem of repeated phrases in the generated description, so that the generated description can better capture the key information in the image.

Description

technical field [0001] The present invention relates to an image description method based on conditional random field and internal semantic attention, in particular to a deep network model based on conditional random field, using a unique internal semantic attention mechanism combined with a network model of attention feature residual structure To generate a semantic description corresponding to an image, which belongs to the cross-technical field of computer vision and natural language processing. Background technique [0002] With the explosive growth of image data on the Internet, it has become unrealistic to identify and retrieve image semantic information manually. The data structure of the image itself is relatively abstract, but it contains a wealth of information. The method of deep learning to generate descriptions for images and mine the semantic information in them has a wide range of application scenarios in image retrieval, AI question answering, intelligent rec...

AI Technical Summary

Problems solved by technology

[0003] Traditional image semantic description methods include image description methods based on template filling and image description methods based on retrieval, the results of these methods are not accurate enough, and consume a lot of time and extra work

For several years, the deep learning method based on the encoder-decoder architecture has also been applied to the field of image semantic description generation, but the inconsistency between the image description training and generation process has always made the generated image description too single, and also Not accurate enough

However, the current method still cannot effectively solve the problem of inaccurate sentence structure and repeated phrases in the generated description.

Images