A Method for Automatic Generation of Image Content Description Based on Joint Neural Network Model

Abstract

The invention discloses a method for automatically generating image content descriptions based on a joint neural network model, including four steps: building a joint neural network, training joint neural network multi-layer mapping parameters, preprocessing pictures and corresponding text, and inputting a model for description result detection. step. The present invention constructs a joint neural network by using a multi-layer offset-improved convolutional deep neural network and a cyclic neural network with learning long-term dependence information; uses the error backpropagation algorithm to train the multi-layer mapping parameters of the neural network; and pre-images Processing, use the convolutional neural network to extract features and perform feature mapping; input the model for description result detection, and use the feature vector input of the image in the test data set to the trained joint neural network to generate the corresponding text description, which can be significantly effective Improved automatic description of image content.

Description

technical field [0001] The invention relates to the technical field of machine learning and image processing, in particular to a method for automatically generating image content descriptions based on a joint neural network model. Background technique [0002] How to automatically describe image content is an important research topic of artificial intelligence in the field of computer vision and natural language processing. Image description generation is a comprehensive problem that combines computer vision, natural language processing and machine learning. It is similar to translating an image into a description text. This problem is not challenging for humans, but it is very challenging for machines, because the task requires the use of The machine model understands the relationship between the content of the image and the content expressed in natural language, and at the same time extracts the semantic information in the image and generates human natural language. [00...

AI Technical Summary

Problems solved by technology

However, the ordinary RNN network (Recurrent Neural Network) has the problem of gradient descent. RNN can only memorize the limited unit content before the neural network unit, which makes the effect of automatically describing the image content not good.

Images