Scene graph generation method, system and equipment based on global context interaction

Abstract

The invention discloses a scene graph generation method, system and equipment based on global context interaction. The method comprises the following steps: 1) carrying out vector joint representation based on fusion of various features such as object visual features, space coordinates and semantic tags; 2) global feature generation based on a bidirectional gating recurrent neural network; 3) a message iteration transmission mechanism based on a global feature vector; and 4) generating a scene graph based on target and relation state representation. Compared with the existing scene graph generation method, the scene graph generation method based on the global context interaction disclosed by the invention has the advantages that the global features of the image are fully utilized through the context interaction, and the application universality is better; meanwhile, after the global features after context interaction are obtained, message passing between the target pair and the relation of the target pair is carried out, the existing state is updated by utilizing the potential relation between the targets, more accurate scene graph generation is carried out, and the method has the advantage of practical application.

Description

technical field [0001] The invention belongs to the field of computer vision, and in particular relates to a method, system and device for generating a scene graph based on global context interaction. Background technique [0002] The scene graph composed of <subject-relation-object> triples can describe the scene structure relationship between objects and object pairs in the image. The scene graph has two main advantages: First, the <subject-relation-object> triplet of the scene graph has structured semantic content. Compared with natural language text, in the process of fine-grained information acquisition and processing There are obvious advantages; secondly, the scene graph can fully represent the objects in the image and the structure relationship of the scene, and has a wide range of application prospects in a variety of computer vision tasks. The decision-making system provides more comprehensive environmental information; in the semantic image retrieval ...

AI Technical Summary

Problems solved by technology

[0006] The scene graph generation method in prior art 1 does not consider making full use of feature vectors in the way of feature fusion; the method in prior art 2 does not use a message passing mechanism, does not consider information interaction between target pairs and their relationships, and cannot perform context transfer status update for

And both of them do not use the implicit constraints between all the targets in the image to build the context, which has certain shortcomings

Images