Scene map generation method

Abstract

The invention discloses a scene graph generation method, which is characterized in that three different levels of semantic tasks of object detection, visual relationship detection and image region description are mutually connected, and the visual tasks of different semantic levels of scene understanding are jointly solved in an end-to-end mode. Firstly, an object, a visual relationship and imageregion description are aligned with a feature information transfer graph according to spatial features and semantic connection of the object, the visual relationship and the image region description,and then feature information is transferred to three different levels of semantic tasks through the feature information transfer graph so as to achieve simultaneous iterative updating of semantic features. According to the method, object detection and visual relation detection are realized by utilizing semantic feature connection of different levels of scene images so as to generate scene images corresponding to the scene images; and the main region of the scene image is subjected to image description by using a natural language, Meanwhile, the image region description is used as a supervisionmethod for scene image generation so as to improve the accuracy of scene image generation.

Description

technical field [0001] The invention relates to a method for generating a scene graph, in particular to a method for generating a scene graph based on multi-level semantic tasks, belonging to the fields of object detection, visual relationship detection and image region description. Background technique [0002] Scene understanding is one of the hot issues in computer vision research. Visual scene understanding includes multiple semantic tasks at different levels: object detection and recognition, prediction of the visual relationship between detected objects, and the use of natural language to express the content of the scene image. description etc. Due to the complexity and diversity of object categories and their visual relationships, visual scene understanding is still a difficult problem. A good visual scene understanding system can not only identify the differences in the content represented by different images, but also focus on or represent the differences. . The k...

AI Technical Summary

Problems solved by technology

The existing traditional scene graph generation methods only perform object detection and relationship prediction on part of the content of the entire scene image, but cannot understand the main content of the scene image in a targeted manner.

At this stage, convolutional neural networks and recurrent neural network structures are used to generate image descriptions. The existing image description methods are only based on the target objects in the scene image and combined with natural language processing to generate an overall language description of the image content, and do not take advantage of the scene The relevant information of the image does not highlight the visual relationship between the objects to be expressed in the image

Images