Automatic image semantic description method

Abstract

The invention discloses an automatic image semantic description method. the method comprises the following steps: utilizing a clustering algorithm to cluster the image set according to the visual features; dividing the clustered image set into a plurality of categories; performing image pre-description processing by using the CNN; the image is labeled with a plurality of categories, and the pre-description of the first layer is called as the category labeling of the image; constructing a classifier for each type of image by using an SVM; judging whether such descriptions are added to the imageor not by utilizing a classifier; utilizing an MBRM model labeling algorithm; and obtaining image semantics through the joint of the image areas obtained by the related training set. The invention provides an automatic image semantic description method. The method can effectively fuse the low-level features of the image and the semantic description high-level semantic information of the image, ishigh in precision and accuracy, has the characteristics of definiteness, formalization, sharing, summarization and the like, can be widely applied to a plurality of fields including information retrieval, information extraction, semantic networks and knowledge management, and is high in applicability.

Description

technical field [0001] The invention relates to the technical field of image semantic description, in particular to an automatic image semantic description method. Background technique [0002] Automatic description of image content (image captioning), that is, to automatically describe the content of images in natural language, has a wide range of application prospects for automatic description of image content, such as human-computer interaction and blind guide systems, and has recently become a field of computer vision and artificial intelligence. A new focus, unlike image classification or object detection, automatic image description aims at comprehensive description of objects, scenes and their relationships, involving visual scene parsing, content semantic understanding, and natural language processing, which is a cutting-edge technology in mixed tasks integrated design; [0003] In the existing technology, the image features extracted by CNN are used as the input of...

AI Technical Summary

Problems solved by technology

[0003] In the existing technology, the image features extracted by CNN are used as the input of the recurrent neural network (RNN), and the image semantic description information is used as the output of the RNN, and the image semantic description problem is regarded as the translation process from the image to the semantic description. An automatic image semantic description model based on CNN and RNN, but the accuracy of this method's understanding of image semantics is not high, the sentences marked with this model are not smooth enough, and the accuracy of the marked content is not high