Image visual relationship anaphora positioning method based on attention pyramid graph network

Abstract

The invention relates to an image visual relationship anaphora positioning method based on an attention pyramid graph network. The image visual relationship anaphora positioning method comprises the following step of processing an input picture through using an attention pyramid graph network model to obtain a corresponding image visual relationship anaphora entity positioning graph, so as to realize anaphora positioning, wherein the attention pyramid graph network model comprises an attention feature pyramid network and a relationship conduction graph network, the attention feature pyramid network is used for acquiring a multi-scale attention feature graph from the input picture, and the relation conduction graph network obtains a final visual relation referring entity positioning graph based on the multi-scale attention feature graph. Compared with the prior art, the image visual relationship anaphora positioning method has the advantages of high positioning precision, high robustness and the like.

Description

technical field [0001] The invention relates to a method for referring to and locating visual relationship of an image, in particular to a method for referring and locating visual relationship of an image based on an attention pyramid network. Background technique [0002] In recent years, expressive reference understanding has received increasing attention in the fields of computer vision and natural language processing. This task is aimed at locating specific referent entities. This technology can help eliminate the ambiguity between entities of the same category, thus playing an important role in application scenarios such as image retrieval in the security field and robot human-computer interaction. However, traditional expressive reference understanding tasks need to evaluate the model's natural language and computer vision modules separately, so it is difficult to judge whether the error is caused by the language module or the vision module. To alleviate the need for ...

AI Technical Summary

Problems solved by technology

Symmetric stacked attention transfer methods may not be able to effectively localize these small-sized entities using a single small-scale attention feature map

In addition, the symmetric stacking attention transfer method models the relationship based on the visual features in the image, and the intra-class variance of the visual features of a specific relationship category in the visual relationship triplet is large, and the use of visual features makes the learning of the model difficult. Larger and difficult to distinguish visual relationships

Images