Hierarchical fusion combined query image retrieval method

Abstract

The invention discloses a hierarchical fusion combined query image retrieval method, which innovatively uses an image scene graph to comprehensively analyze entity information, entity attribute information and relationship information among entities in a reference image and a target image, and uses a graph structure to represent entities in the images and semantic relationships among the entities, so that a gap between low-level visual features and high-level semantics is overcome, the semantic representation capability of the image is improved, and the problem of low image retrieval accuracy is well solved. In addition, a hierarchical visual representation and fusion mode is innovatively designed, and visual features are coded into global features, local entity features and scene graph features; then, a multi-modal feature hierarchical fusion mode is designed; finally, a cluster matching mode is designed, image matching is carried out in three feature spaces including a global space, a local entity space and a scene graph space, and the accuracy of combined query image retrieval is further improved.

Description

technical field [0001] The invention belongs to the technical field of image retrieval, and more specifically relates to a combined query image retrieval method of hierarchical fusion. Background technique [0002] In the traditional image retrieval method, after the reference image is input, the retrieval system returns some results most similar to the reference image in the database. However, there is an intention gap problem in traditional image retrieval, that is, it is difficult for users to find a reference image that accurately reflects their retrieval intention. [0003] The goal of combined query image retrieval is to retrieve target images from large-scale data by simultaneously using reference images and textual descriptions. The combined query image retrieval can modify the reference image through additional text descriptions to reflect the user's real retrieval intention, and the retrieval method is more flexible and accurate. [0004] There are mainly two kin...

AI Technical Summary

Problems solved by technology

It is inaccurate to represent only by global features or local entity features. It cannot fully represent the semantic information of visual images, and lacks the ability to describe and summarize visual image scenes, resulting in combined queries based on image global features or local entity features. Image retrieval methods have low accuracy

[0006] The current method only encodes the image in a single visual dimension. Considering that the text description is complex and diverse, it is not only associated with abstract global visual features and specific local entity features, but also with the semantic features of the image, which also affects The accuracy of combined query image retrieval

Images