Image description generation method and device based on deep residual network and attention

Abstract

The invention discloses an image description generation method and device based on a deep residual network and attention, which solves the problem of decreased accuracy of the deep neural network, uses the deep residual network to learn image features from the bottom layer to the high layer, and generates rich The input image representation is then combined with an attentional recurrent long-short-term memory network to generate natural and fluent description sentences. The method includes the following steps: acquiring a large amount of image sample data and preprocessing it; extracting image features of the preprocessed image sample data; using a residual neural network model to process the extracted image features to generate an image representation; The image representation is mapped to the input of the attention-based cyclic long-short-term memory network language model, and the attention-based cyclic long-short-term memory network language model is used to predict the word vector of the image and generate a description sentence for the image.

Description

technical field [0001] The present disclosure relates to the field of image processing, in particular to an image description generation method and device based on a deep residual network and attention. Background technique [0002] Image description generation technology is closely related to technologies such as image semantic analysis, image annotation and image advanced semantic extraction. Deep learning has shown promising performance in recent years on both image and natural language processing tasks. [0003] In recent years, deep convolutional networks have made a series of breakthroughs in image classification and image recognition. The deep network makes the features richer by superimposing the depth of the layer. Many important visual recognition tasks also benefit from deep models. However, as the depth of the network increases, the accuracy begins to saturate, and then declines rapidly, and the problem of model degradation occurs. During the research and dev...

AI Technical Summary

Problems solved by technology

[0006] In order to overcome the deficiencies of the above-mentioned existing technologies, the present disclosure provides a method and device for generating image descriptions based on deep residual networks and attention, which solves the problem of decreased accuracy of deep neural networks, and uses deep residual networks to learn images from Low-level to high-level image features, by embedding the input image into a fixed vector to generate a rich input image representation, and then combined with an attentional recurrent long short-term memory network to generate natural and smooth description sentences

Images