Adaptive generation system for image semantic description

Abstract

The invention relates to an adaptive generation system for image semantic description, and belongs to the technical field of image semantic description. The system comprises an image reader, an encoder, a decoder and a semantic description display. The image reader is used for acquiring an image to be described semantically. The encoder comprises a global feature extraction unit, a local feature extraction unit and an image feature combination unit. The decoder comprises a single-layer or multi-layer neural network. The neural network comprises an LSTM part, an Attention part and an MLP part,and a semantic description model is generated. The Attention part adopts a self-adaptive attention mechanism. The decoder generates words and sentences of image semantic description by utilizing the semantic description model according to the combination information output by the encoder. The semantic description display is used for outputting and displaying words and phrases of the image semanticdescription. The focus key point of the image is determined, higher-level semantic information is mined, and the detail information problem describing words or sentences is perfected.

Description

technical field [0001] The invention relates to the technical field of image semantic description, in particular to an adaptive generation system for image semantic description. Background technique [0002] With the rapid development of artificial intelligence and breakthroughs in deep learning technology, computer vision technology based on deep learning is becoming more and more mature. Researchers are trying to make machines understand more complex semantic information in visual information. Therefore, at the intersection of computer vision and natural language processing The research direction of image semantic description appears in the field. The image semantic description technology was first proposed by Farhadi et al. Its goal is to realize the transformation from the image in the visual space to the text description in the semantic space. The method realizes the mapping from the image to the text description sentence, and gives semantic interpretation to the visual...

AI Technical Summary

Problems solved by technology

[0004] In view of the above analysis, the present invention aims to provide an adaptive generation system for image semantic description, to solve the current image semantic description technology to determine the focus of attention of the image, to mine higher-level semantic information, and to improve the description of words or sentences. details, etc.

Images