A Scene Graph Generation Method Based on Hyper-Relational Learning Network

Abstract

The invention discloses a method for generating a scene graph based on super-relational learning. The present invention includes the following steps: 1. Strengthen target interaction through target self-attention network, and integrate target features. 2. Strengthen the interaction between the target and the relationship through the target-relational attention network, and integrate the features between the target and the relationship. 3. Integrating hyper-relational transitive reasoning through a hyper-relational attention network. 4. Model training, put the target loss function and relationship loss function into the optimizer, and use the backpropagation algorithm to carry out gradient return and update of the network parameters. The present invention proposes a deep neural network for scene graph generation, especially a super-relational learning network, which makes full use of the interaction and transfer reasoning between objects and relationships, improves the reasoning ability of relationships in scene graph generation, and The performance in the field of scene graph generation has been greatly improved.

Description

technical field [0001] The present invention proposes a scene graph generation method (HLN) based on Hyper-relationship Learning Network. HLN originates from a hypergraph, using a multi-layer attention network to interact between objects. We further propose the Object-Relation Attention Network (OR-GAT) to autonomously fuse features from the interactions between objects and relations. The present invention proposes a super-relational attention network for the first time to integrate super-relational transfer reasoning. where super-relations refer to a subset of the relations among the three objects. Through the interaction between objects, the interaction between objects and relations, and the transitive reasoning of super-relations, the predictive effect of relations is obviously improved. Background technique [0002] Scene Graph Generation (SGG) aims to detect objects and predict object relationships. These detected objects and relationships then constitute the scene ...

AI Technical Summary

Problems solved by technology

[0010] (3) Currently, there is no method that considers high-level connections of relations, i.e. transitive inference

However, the high-level connectivity of relations has a particularly significant impact on the performance of relation prediction

Images