Image scene map generation method based on iteration refinement

Abstract

The invention discloses an image scene map generation method based on iteration refinement, and relates to the field of image scene maps. The method comprises the following steps of: inputting an image scene map data set and parameters thereof, and extracting visual features of an image; in combination with the image data and the scene map data in the training set, classifying the visual relationship between the entity pairs by using the visual information and the semantic information to generate an image scene map; generating a plurality of region bounding boxes and corresponding feature vectors of each region, and fusing the region feature vectors to obtain feature vectors of the image description text segments; inputting the feature vectors of the image description text segments into animage description text segment generator to generate image description text segments; constructing a refined graph, defining feature vectors and regional feature vectors of refined entity pairs, anditeratively executing image scene graph generation, image segmented image description file generation and refined feature vector until the maximum number of iterations is reached. According to the invention, the image scene map generation effect is improved.

Description

technical field [0001] The invention relates to the field of image scene atlas representation learning, and specifically discloses a method for generating an image scene atlas based on iterative refinement. Background technique [0002] An image scene graph is an abstract and structured representation of the scene depicted by an image. Specifically, the image scene graph records the entities, The attributes of entities and the relationship between entities, and each entity has a corresponding bounding box in the image. Image scene atlas was first proposed by Johnson et al. Recently, it has received attention in the fields of computer vision and artificial intelligence, and has related research results, such as: using image scene atlas to retrieve images; retrieving entities from word vectors learned from external corpora in advance Word vectors to assist visual question answering; use object detectors and relationship detectors to generate image scene graphs. [0003] In ...

AI Technical Summary

Problems solved by technology

[0009] Aiming at the problems existing in the generation of existing image scene atlases, the present invention proposes a method for generating image scene atlases based on iterative refinement, which utilizes the information contained in image description paragraphs to iteratively refine the visual relationship classification accuracy of image scene atlases, thereby Achieve the effect of improving the image scene map generation

Images