Image description method

Abstract

The invention discloses an image description method, and the method comprises the steps: extracting global image features in a picture through a VGG convolution neural network, and extracting local image features in picture by using Faster R-CNN network, fusing global image features and local image features through a global-local feature fusion algorithm to acquire the image fusion features; processing image fusion features through a bidirectional long-short-term memory network with an attention mechanism; and generating a preliminary image description statement, performing WordNet word vectorsimilarity calculation based on the image target information obtained during local image feature extraction and nouns in the preliminary image description statement, correcting the image descriptionstatement, and generating a final image description statement. According to the method, the influence of useless information is reduced, the expression of key information is enhanced, the fault tolerance and generalization ability of the model are enhanced, and the accuracy of statement description is improved.

Description

technical field [0001] The invention relates to the field of image recognition processing, in particular to a method for object detection and description sentence generation in an image based on a codec and multiple feature fusion processing. Background technique [0002] With the rapid development of technology, smart phones are becoming more and more popular among people, and selfies and casual photos have gradually become a mainstream way of socializing, so images are growing at an exponential rate. As of 2014, Facebook alone has more than 250 billion images. Conventional image retrieval methods, such as manually labeling images and performing brief image descriptions, are no longer able to bear pictures of this magnitude. It becomes impossible to deal with it in a certain way, so it is conceived to use machines to automatically label and describe images. [0003] Image description thrives against the background of the rapid development of machine learning and deep learn...

AI Technical Summary

Problems solved by technology

Image description is relatively convenient for humans, but it is quite difficult for computers, because computers not only need to find the target and background in the picture, but also need to understand the relationship between them, which is a relatively complicated thing

[0004] Most of the existing image description methods often use global features for image features. From the description results, it can be seen that the description accuracy of the relationship between target parts is low, and there will be errors in the description results.

None of the current image description methods have effective corrections for the above problems

Images