An image description method constructed based on a hierarchical feature relation graph

Abstract

The invention relates to the field of computer vision recognition, and provides an image description method constructed based on a hierarchical feature relation graph, which comprises the following steps: constructing a training data set; inputting the image into a block detection module, and outputting block visual information; inputting the image into a target detection module, and outputting target visual information; inputting the image into a text detection module, and outputting text visual information; Respectively inputting the three visual information into a description generator, respectively constructing a relational graph of each type of visual information and training image coordinate information, and optimizing the three visual information; Screening and fusing the three types of visual information to obtain multi-modal characteristics; inputting the description words into a recurrent neural network to extract feature information, and predicting the next description wordto generate a complete description sentence. According to the method, various types of visual information are optimized, screened and fused, so that any input test image is described, and the image description accuracy can be effectively improved.

Description

technical field [0001] The present invention relates to the field of computer vision recognition, and more specifically, relates to an image description method constructed based on a hierarchical feature relationship graph. Background technique [0002] In recent years, image description, as an interdisciplinary subject between computer vision and natural language processing, has received more and more attention. Image description means that given an input image, an algorithm needs to be designed so that the computer learns to describe the content in the image with grammatically and semantically correct sentences. Therefore, image description is very difficult for machines, because machines need to learn to recognize information such as objects and scenes in images, and at the same time need to understand the internal relationship between different objects, objects and backgrounds. But this research direction is valuable, because image description has a wide range of applic...

AI Technical Summary

Problems solved by technology

However, these methods still have shortcomings. Most of the image description methods in the prior art only use the information of a single object, without considering the relationship information between objects, so the relationship between objects still needs to be improved.

Images